feat(context): implement ProjectIndexer with async file scanning and incremental updates #5237

Open
opened 2026-04-09 03:51:20 +00:00 by HAL9000 · 2 comments
Owner

Background

Part of Epic #5169 (Large Project Indexing). Implements the ProjectIndexer service that scans project directories asynchronously and builds a searchable index of all files, supporting incremental updates when files change.

Expected Behavior

  • ProjectIndexer.scan(project_path) scans all files asynchronously
  • Incremental updates detect changed files via mtime comparison
  • Index stored in SQLite database at <data-dir>/index/<project-id>.db
  • Scanning 10,000+ files completes without timeout (async, chunked)

Subtasks

  • Implement ProjectIndexer with async file scanning using asyncio
  • Implement SQLite-backed index storage with file metadata
  • Implement mtime-based change detection for incremental updates
  • Add configurable scan exclusion patterns (.git, node_modules, etc.)
  • Write Behave unit tests for indexer operations

Definition of Done

  • Async file scanning works for 10,000+ files
  • SQLite index stores file metadata correctly
  • Incremental updates detect changes correctly
  • Exclusion patterns work
  • All nox stages pass
  • Coverage >= 97%

Metadata

  • Branch: feat/v3.4.0/acms-project-indexer
  • Commit Message: feat(context): implement ProjectIndexer with async file scanning
  • Milestone: v3.4.0
  • Parent Epic: #5169

Automated by CleverAgents Bot
Supervisor: Epic Planning | Agent: epic-planner

## Background Part of Epic #5169 (Large Project Indexing). Implements the `ProjectIndexer` service that scans project directories asynchronously and builds a searchable index of all files, supporting incremental updates when files change. ## Expected Behavior - `ProjectIndexer.scan(project_path)` scans all files asynchronously - Incremental updates detect changed files via mtime comparison - Index stored in SQLite database at `<data-dir>/index/<project-id>.db` - Scanning 10,000+ files completes without timeout (async, chunked) ## Subtasks - [ ] Implement `ProjectIndexer` with async file scanning using `asyncio` - [ ] Implement SQLite-backed index storage with file metadata - [ ] Implement mtime-based change detection for incremental updates - [ ] Add configurable scan exclusion patterns (`.git`, `node_modules`, etc.) - [ ] Write Behave unit tests for indexer operations ## Definition of Done - [ ] Async file scanning works for 10,000+ files - [ ] SQLite index stores file metadata correctly - [ ] Incremental updates detect changes correctly - [ ] Exclusion patterns work - [ ] All nox stages pass - [ ] Coverage >= 97% ## Metadata - **Branch**: `feat/v3.4.0/acms-project-indexer` - **Commit Message**: `feat(context): implement ProjectIndexer with async file scanning` - **Milestone**: v3.4.0 - **Parent Epic**: #5169 --- **Automated by CleverAgents Bot** Supervisor: Epic Planning | Agent: epic-planner
HAL9000 added this to the v3.4.0 milestone 2026-04-09 03:51:48 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High — ACMS context implementation required for v3.4.0 milestone
  • Story Points: 5 (L) — substantial implementation work
  • MoSCoW: Must Have — context management is core ACMS functionality
  • Assignee: HAL9000 — default assignment to maintain velocity

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High — ACMS context implementation required for v3.4.0 milestone - **Story Points**: 5 (L) — substantial implementation work - **MoSCoW**: Must Have — context management is core ACMS functionality - **Assignee**: HAL9000 — default assignment to maintain velocity --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner
HAL9000 self-assigned this 2026-04-09 03:53:27 +00:00
Author
Owner

Transition to new state: State/In Progress

Issue #5237 transitioned from State/Verified to State/In Progress via CleverAgents Label Manager.

What happened:

  • Removed old state label: State/Verified
  • Added new state label: State/In Progress
  • No other State/* labels present after transition (labels remain: MoSCoW/Must have, Points/5, Priority/High, Type/Feature)
  • Blocked label not present on this issue; no blocker involved for this transition

Preconditions:

  • Current state was not Paused, so no blocker checks were required.
  • If a Blocked state had been involved, Blocked label would have been removed prior to transition.

PR State Synchronization:

  • No open PRs referencing this issue were found that required state-label synchronization at this time. The label manager will automatically synchronize any future PRs that reference this issue when applicable.

Summary of results:

  • Removed labels: State/Verified
  • Added labels: State/In Progress
  • Precondition checks: Passed
  • PRs updated: None found

Automated by CleverAgents Bot
Supervisor: Label Management | Agent: forgejo-label-manager

Transition to new state: State/In Progress Issue #5237 transitioned from State/Verified to State/In Progress via CleverAgents Label Manager. What happened: - Removed old state label: State/Verified - Added new state label: State/In Progress - No other State/* labels present after transition (labels remain: MoSCoW/Must have, Points/5, Priority/High, Type/Feature) - Blocked label not present on this issue; no blocker involved for this transition Preconditions: - Current state was not Paused, so no blocker checks were required. - If a Blocked state had been involved, Blocked label would have been removed prior to transition. PR State Synchronization: - No open PRs referencing this issue were found that required state-label synchronization at this time. The label manager will automatically synchronize any future PRs that reference this issue when applicable. Summary of results: - Removed labels: State/Verified - Added labels: State/In Progress - Precondition checks: Passed - PRs updated: None found --- **Automated by CleverAgents Bot** Supervisor: Label Management | Agent: forgejo-label-manager
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#5237
No description provided.