cleveragents/cleveragents-core

Fork 3

BUG-HUNT: [boundary] File system race conditions during reference catalog walking #7078

New issue

Open

opened 2026-04-10 07:30:06 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-10 07:30:06 +00:00

Owner

Bug Report: Boundary — File System Race Conditions During Reference Catalog Walking

Severity Assessment

Impact: Potential crashes, incomplete catalog data, or missing references during filesystem traversal
Likelihood: Medium in active development environments where files are frequently created/deleted
Priority: Medium

Location

File: src/cleveragents/tui/input/reference_parser.py
Function/Class: _catalog() function
Lines: 47-66

Description

The reference catalog function walks the filesystem using os.walk() without handling potential filesystem race conditions. Files or directories can be deleted or modified during traversal, leading to exceptions that are not caught.

Evidence

def _catalog() -> dict[str, list[str]]:
    # ... cache check logic ...
    
    files = []
    for root, dirs, filenames in os.walk(cwd, followlinks=False):
        dirs[:] = [name for name in dirs if name not in _IGNORED_DIRS]
        root_path = Path(root)
        for filename in filenames:
            try:
                files.append((root_path / filename).relative_to(cwd).as_posix())
            except ValueError:
                files.append((root_path / filename).as_posix())
            # No handling for FileNotFoundError, PermissionError, etc.
            if len(files) >= _CATALOG_LIMIT:
                break
        if len(files) >= _CATALOG_LIMIT:
            break

Expected Behavior

Filesystem errors during traversal should be handled gracefully, allowing the catalog to continue with available files while logging or skipping problematic entries.

Actual Behavior

If files are deleted during traversal, or permissions change, or symlinks break, the following exceptions can occur:

FileNotFoundError when a file is deleted between os.walk() discovering it and attempting to process it
PermissionError when permissions change during traversal
OSError for various filesystem-related errors (e.g., broken symlinks, device errors)

This can cause the entire catalog operation to fail, leaving users without reference completion functionality.

Suggested Fix

Add comprehensive error handling around the filesystem operations:

for root, dirs, filenames in os.walk(cwd, followlinks=False):
    dirs[:] = [name for name in dirs if name not in _IGNORED_DIRS]
    root_path = Path(root)
    for filename in filenames:
        try:
            file_path = root_path / filename
            # Check if file still exists before processing
            if file_path.exists():
                relative_path = file_path.relative_to(cwd).as_posix()
                files.append(relative_path)
        except (FileNotFoundError, PermissionError, OSError):
            # Skip files that can't be accessed
            continue
        except ValueError:
            # Handle files outside cwd
            try:
                if file_path.exists():
                    files.append(file_path.as_posix())
            except (FileNotFoundError, PermissionError, OSError):
                continue
        
        if len(files) >= _CATALOG_LIMIT:
            break

TDD Note

After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_<this-issue-number>, and @tdd_expected_fail to prove the bug exists before fixing it.

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: bug-hunter

## Bug Report: Boundary — File System Race Conditions During Reference Catalog Walking ### Severity Assessment - **Impact**: Potential crashes, incomplete catalog data, or missing references during filesystem traversal - **Likelihood**: Medium in active development environments where files are frequently created/deleted - **Priority**: Medium ### Location - **File**: `src/cleveragents/tui/input/reference_parser.py` - **Function/Class**: `_catalog()` function - **Lines**: 47-66 ### Description The reference catalog function walks the filesystem using `os.walk()` without handling potential filesystem race conditions. Files or directories can be deleted or modified during traversal, leading to exceptions that are not caught. ### Evidence ```python def _catalog() -> dict[str, list[str]]: # ... cache check logic ... files = [] for root, dirs, filenames in os.walk(cwd, followlinks=False): dirs[:] = [name for name in dirs if name not in _IGNORED_DIRS] root_path = Path(root) for filename in filenames: try: files.append((root_path / filename).relative_to(cwd).as_posix()) except ValueError: files.append((root_path / filename).as_posix()) # No handling for FileNotFoundError, PermissionError, etc. if len(files) >= _CATALOG_LIMIT: break if len(files) >= _CATALOG_LIMIT: break ``` ### Expected Behavior Filesystem errors during traversal should be handled gracefully, allowing the catalog to continue with available files while logging or skipping problematic entries. ### Actual Behavior If files are deleted during traversal, or permissions change, or symlinks break, the following exceptions can occur: - `FileNotFoundError` when a file is deleted between `os.walk()` discovering it and attempting to process it - `PermissionError` when permissions change during traversal - `OSError` for various filesystem-related errors (e.g., broken symlinks, device errors) This can cause the entire catalog operation to fail, leaving users without reference completion functionality. ### Suggested Fix Add comprehensive error handling around the filesystem operations: ```python for root, dirs, filenames in os.walk(cwd, followlinks=False): dirs[:] = [name for name in dirs if name not in _IGNORED_DIRS] root_path = Path(root) for filename in filenames: try: file_path = root_path / filename # Check if file still exists before processing if file_path.exists(): relative_path = file_path.relative_to(cwd).as_posix() files.append(relative_path) except (FileNotFoundError, PermissionError, OSError): # Skip files that can't be accessed continue except ValueError: # Handle files outside cwd try: if file_path.exists(): files.append(file_path.as_posix()) except (FileNotFoundError, PermissionError, OSError): continue if len(files) >= _CATALOG_LIMIT: break ``` ### Category boundary ### TDD Note After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_<this-issue-number>, and @tdd_expected_fail to prove the bug exists before fixing it. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: bug-hunter

HAL9000 added the

State

Unverified

label

2026-04-10 18:57:43 +00:00

freemo added the

Type

Bug

label

2026-04-12 18:37:29 +00:00

HAL9000 added

and removed

labels

2026-04-14 06:49:15 +00:00

HAL9000 commented

2026-04-14 06:49:15 +00:00

Author

Owner

✅ Verified — Concurrency bug: file system race conditions during reference catalog walking. MoSCoW: Should-have. Priority: Medium.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — Concurrency bug: file system race conditions during reference catalog walking. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 referenced this issue

2026-05-03 00:31:18 +00:00

feat(server): LangGraph Platform RemoteGraph integration #10792