feat: Make file type detection and binary checks asynchronous (#3286)

[Han5991]

Refactored isBinaryFile and detectFileType functions to be asynchronous, leveraging promises for file operations. Updated corresponding test cases to handle the asynchronous changes. This enhances performance and ensures non-blocking behavior during file handling.

TLDR

What: Converted synchronous file binary detection operations (isBinaryFile, detectFileType) to asynchronous implementations using fs.promises API.

Why: The original sync file operations were blocking the Node.js event loop, causing UI freezes and poor performance when processing multiple files.

Impact: Eliminates event loop blocking, improves UI responsiveness, and enables concurrent file processing for significantly better performance in large codebases.

Dive Deeper

The Problem

The isBinaryFile function in packages/core/src/utils/fileUtils.ts was using synchronous file system operations (fs.openSync, fs.fstatSync, fs.readSync, fs.closeSync) which completely block the Node.js event loop during execution. This created severe performance bottlenecks, especially when the read-many-files tool
processes multiple files sequentially.

Root Cause Analysis

• Event Loop Blocking: Each file binary check froze the entire application
• Sequential Processing: Files had to be processed one-by-one instead of concurrently
• UI Unresponsiveness: Terminal interface became frozen during file operations
• Poor Scalability: Performance degraded dramatically with larger codebases

The Solution

1. Async Conversion: Converted isBinaryFile() from sync to async using fs.promises.open()
2. Proper Resource Management: Implemented try/finally blocks with fileHandle.close()
3. Cascade Updates: Updated detectFileType() and processSingleFileContent() to handle async calls
4. Test Modernization: Updated all test cases to use async/await patterns

Technical Details

Before (Blocking):

  export function isBinaryFile(filePath: string): boolean {
    const fd = fs.openSync(filePath, 'r');          // BLOCKS EVENT LOOP
    const fileSize = fs.fstatSync(fd).size;         // BLOCKS EVENT LOOP
    const bytesRead = fs.readSync(fd, buffer, 0, buffer.length, 0);  // BLOCKS EVENT LOOP
    fs.closeSync(fd);                               // BLOCKS EVENT LOOP
  }

After (Non-blocking):

  export async function isBinaryFile(filePath: string): Promise<boolean> {
    const fileHandle = await fs.promises.open(filePath, 'r');
    try {
      const stats = await fileHandle.stat();
      // ... async implementation with proper resource cleanup
    } finally {
      await fileHandle.close();  // Guaranteed cleanup
    }
  }

Reviewer Test Plan

Setup

git checkout feature/async-file-binary-check
npm run build
npm run test

Manual Testing Scenarios

1. Basic File Processing:
   npm start

Read this README file and summarize it

1. Expected: Should work seamlessly without UI freezing
2. Multiple File Processing (Performance Test):

Read all TypeScript files in the src directory

2. Expected: UI should remain responsive, files processed efficiently
3. Large Codebase Test:

Analyze all JavaScript/TypeScript files in this project

3. Expected: Significant performance improvement, no UI blocking
4. Error Handling:

Read a file that doesn't exist: /path/to/nonexistent.txt

4. Expected: Graceful error handling, no crashes

Testing Matrix

	🍏	🪟	🐧
npm run	✅	❓	❓
npx	✅	❓	❓
Docker	❓	❓	❓
Podman	❓	-	-
Seatbelt	✅	-	-

Linked issues / bugs

• Resolves Convert synchronous file binary detection to async to eliminate event loop blocking #3286

[gemini-code-assist]

Summary of Changes

Hello @Han5991, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the application's performance and responsiveness by converting synchronous file system operations, specifically for file type detection and binary checks, to asynchronous implementations. This change prevents blocking the Node.js event loop, leading to a smoother user experience and enabling more efficient concurrent file processing.

Highlights

• Asynchronous File Operations: The isBinaryFile and detectFileType functions have been refactored to use fs.promises for non-blocking file I/O, significantly improving performance and responsiveness.
• Improved Resource Management: The isBinaryFile function now properly manages file handles using try/finally blocks to ensure fileHandle.close() is always called, preventing resource leaks.
• Cascading Async Updates: Functions that depend on isBinaryFile and detectFileType, such as processSingleFileContent in read-many-files.ts, have been updated to correctly await these now-asynchronous calls.
• Test Suite Modernization: All relevant unit tests for fileUtils have been updated to reflect the asynchronous nature of the functions, using async/await patterns to ensure correct testing of the new behavior.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request successfully refactors synchronous file I/O operations to be asynchronous, which is a great improvement for performance and responsiveness. The changes are well-described and the test plan is thorough. I've found one critical issue in the error handling of the new isBinaryFile function that could lead to an unhandled promise rejection. My review includes a suggestion to fix this and improve the robustness of the function.

[Han5991]

/gemini review

[gemini-code-assist]

Code Review

This pull request successfully refactors synchronous file I/O operations (isBinaryFile, detectFileType) to be asynchronous, which is a great improvement for performance and UI responsiveness. The changes are well-implemented, using fs.promises and ensuring proper resource cleanup with try...finally. I have one high-severity suggestion to improve debuggability by capturing errors instead of swallowing them silently.

[Han5991]

/gemini review

[gemini-code-assist]

Code Review

This pull request is an excellent improvement. It successfully refactors synchronous file I/O operations in isBinaryFile and detectFileType to be asynchronous, which is a key performance enhancement for a Node.js application. The use of async/await with fs.promises is implemented correctly, and I'm particularly impressed with the robust resource management using try...catch...finally to ensure file handles are always closed, even in error scenarios. The changes are clean, and the corresponding updates to tests and calling functions are thorough. Great work on this!

[NTaylorMullen]

Thank you for the contribution @Han5991 !