diff --git a/implementation_plan.md b/implementation_plan.md index 5286c4f3..fe4a51c7 100644 --- a/implementation_plan.md +++ b/implementation_plan.md @@ -20,6 +20,7 @@ - **NO PLANDEX REFERENCES**: All environment variables must use CLEVERAGENTS_ prefix, not PLANDEX_. No references to Plandex should exist in the final code. CleverAgents is a standalone project, not a fork or migration. - **Use existing tooling**: Always prefer nox sessions over raw commands, Behave for unit tests over new frameworks, Robot for integration tests, Hatch for dependency management. - **Mock placement rule**: ALL mocks, test doubles, and mock implementations MUST exist only in `features/` directory. Production code in `src/` and utility scripts in `scripts/` must NEVER contain mock implementations, test data, or conditional testing behavior. Use dependency injection to swap implementations during tests. +- **CRITICAL - Implementation Checklist Separation**: The "Implementation Checklist" section MUST always remain separate and be the LAST section of this document. All development notes, design decisions, progress updates, technical details, and discoveries belong in their respective phase Notes sections (e.g., Phase 0 Notes, Phase 1 Notes, Phase 2 Notes) which appear BEFORE the Implementation Checklist. Never add content after the checklist section. The checklist is for tracking what needs to be done; the Notes sections are for documenting what was done and how. ### CONTINUOUS CHECKLIST AND KNOWLEDGE STEWARDSHIP (MANDATORY) @@ -581,12 +582,24 @@ All 10 ADRs have been created in `docs/architecture/decisions/`: - 19 Behave scenarios (146 steps) - all passing - 8 Robot integration tests - all passing - Helper script for robust Robot testing created + - Full test coverage for all nodes, async/streaming, and edge cases -**In Progress:** -- 🔄 Week 12: Streaming & Auto-Debug - - Next: Integrate LangGraph streaming into CLI commands - - Next: Create AutoDebugGraph workflow - - Next: Update all tests for LangChain compatibility +**In Progress (Week 12):** +- 🔄 CLI Streaming Integration (HIGH PRIORITY) + - Integrate LangGraph streaming events into CLI commands + - Add real-time progress indicators with Rich + - Implement `--stream` flag for plan generation +- 🔄 AutoDebugGraph Implementation + - Create auto-debug workflow with LangGraph + - Integrate with plan build failures + - Add retry logic with error analysis +- 🔄 EntityMemory Integration + - Add EntityMemory for cross-session project tracking + - Integrate with existing MemoryService +- 🔄 LangSmith Observability Setup + - Configure LangSmith tracing (user-configurable) + - Add metadata enrichment to traces + - Create observability documentation **Test Coverage Status:** - Overall coverage: 95% (exceeds 85% requirement) @@ -851,11 +864,11 @@ All 14 core commands have been successfully implemented with comprehensive testi - Dependency injection enables flexible provider swapping **Outstanding Tasks for Full Completion:** -- [X] Increase test coverage from 44% to >85% (HIGH PRIORITY) — achieved 95% coverage after full Behave run with subprocess tracking on 2025-11-17 -- [X] Write unit tests for repository classes -- [ ] Fix minor legacy migrator validation issues -- [ ] Add performance benchmarks for commands -- [X] Implement async patterns (33 retry patterns with tenacity) — COMPLETED 2025-11-17 +- ✅ Increase test coverage from 44% to >85% (HIGH PRIORITY) — achieved 95% coverage after full Behave run with subprocess tracking on 2025-11-17 +- ✅ Write unit tests for repository classes +- ⏳ Fix minor legacy migrator validation issues (tracked in Implementation Checklist) +- ⏳ Add performance benchmarks for commands (tracked in Implementation Checklist) +- ✅ Implement async patterns (33 retry patterns with tenacity) — COMPLETED 2025-11-17 **Key Learnings:** 1. Starting with JSON storage for rapid prototyping, then migrating to SQLite was effective @@ -933,10 +946,11 @@ All Week 9 foundation tasks are now verified as complete: 5. Update tests for LangChain compatibility 6. Align PlanGenerationGraph interface with existing test expectations (if needed) -**2025-11-20: Context Analysis Agent Testing Complete** +**2025-11-20: Phase 2 Week 11 Complete - Context Analysis Agent** **ContextAnalysisAgent [X] COMPLETE** - Created comprehensive LangGraph workflow in `src/cleveragents/agents/context_analysis.py` (468 lines, implemented 2025-11-19) +- **Week 11 Summary**: Full implementation with comprehensive testing infrastructure - Implemented 5-node workflow for analyzing code context: 1. **load_files**: Loads files using LangChain's TextLoader and creates Document objects 2. **analyze_dependencies**: Extracts imports and dependencies using LLM analysis @@ -1282,6 +1296,160 @@ The actual implementation of LangChain/LangGraph features is distributed across - **Phase 7**: Documentation includes LangChain/LangGraph patterns and examples - **Phase 8**: Testing uses LangChain's mock providers and test utilities +--- + +**Phase 2 Notes - Week 12 Detailed Action Plan (2025-11-20)** + +This section documents the detailed implementation plan for Phase 2 Week 12, including CLI streaming integration, AutoDebugGraph implementation, and EntityMemory enhancements. + +## Week 12 Detailed Action Plan (Starting 2025-11-20) + +### Day 1-2: CLI Streaming Integration (Stage 2.7.3) + +**Goal**: Integrate LangGraph streaming events into CLI commands for real-time progress feedback. + +**Task Breakdown:** + +1. **Update PlanService for Streaming** (4 hours) + - Add `generate_plan_streaming()` async method to `plan_service.py` + - Method should yield events from `graph.astream()` + - Return `AsyncIterator[dict]` with node events + - Add error handling for streaming exceptions + - Location: `src/cleveragents/application/services/plan_service.py:350` (new method) + +2. **Update CLI Commands** (4 hours) + - Add `--stream` flag to `agents tell` command + - Implement async event processing in CLI + - Use `asyncio.run()` to handle async streaming in sync context + - Add graceful Ctrl+C cancellation + - Location: `src/cleveragents/cli/commands/plan.py:tell` command + +3. **Progress Indicators with Rich** (3 hours) + - Create event-to-message mapping for each node + - Implement spinner/progress bar for long operations + - Show timing info and status updates + - Use Rich's Live display for updating output + - Location: `src/cleveragents/cli/progress.py` (new module) + +4. **Testing** (5 hours) + - Create Behave scenarios for streaming in `features/cli_streaming.feature` + - Mock streaming events in tests + - Verify output formatting and progress indicators + - Test `--stream` and `--quiet` flags + - Add Robot tests in `robot/cli_streaming.robot` + +**Expected Outcomes:** +- `agents tell "add feature" --stream` shows real-time progress +- Each graph node displays its status (⏳ running, ✓ complete) +- Timing information shown for each stage +- Ctrl+C cancels gracefully without errors +- All tests passing with >85% coverage + +*(Success criteria checklist items have been moved to the Implementation Checklist section under Stage 2.7.3)* + +**Code Example - Streaming Service Method:** +```python +async def generate_plan_streaming( + self, project_id: str, description: str +) -> AsyncIterator[dict]: + """Generate plan with streaming events.""" + graph = PlanGenerationGraph(self.llm) + state = { + "project_id": project_id, + "description": description, + "max_retries": 3 + } + config = RunnableConfig( + configurable={"thread_id": f"plan-{uuid.uuid4()}"} + ) + + async for event in graph.astream(state, config): + yield event +``` + +### Day 3-4: AutoDebugGraph Implementation (Stage 2.7.5) + +**Goal**: Create auto-debug workflow using LangGraph for plan build error recovery. + +**Task Breakdown:** + +1. **Create AutoDebugGraph** (6 hours) + - File: `src/cleveragents/agents/auto_debug.py` + - Define `AutoDebugState` TypedDict + - Implement nodes: + - `analyze_error`: Parse error message and context + - `generate_fix`: Create code fix using LLM + - `validate_fix`: Syntax check and validation + - `apply_fix`: Update plan with fixes + - Add conditional edges for retry logic + - Implement MemorySaver checkpointing + +2. **Integrate with PlanService** (4 hours) + - Call AutoDebugGraph when build fails + - Pass error context and stack trace + - Stream auto-debug progress to user + - Limit retry attempts (max 3 by default) + - Location: `src/cleveragents/application/services/plan_service.py:build` method + +3. **Testing** (6 hours) + - Create `features/auto_debug_agent_coverage.feature` with 15+ scenarios + - Implement step definitions in `features/steps/auto_debug_agent_coverage_steps.py` + - Test error analysis, fix generation, validation, and retry logic + - Add Robot tests in `robot/auto_debug_agent.robot` + - Mock various error types for comprehensive coverage + +**Expected Outcomes:** +- AutoDebugGraph successfully analyzes common error types +- Fix generation produces valid code changes +- Validation catches syntax errors before applying +- Retry logic works with configurable max attempts +- All tests passing (15+ Behave scenarios, 8+ Robot tests) + +*(Success criteria checklist items have been moved to the Implementation Checklist section under Stage 2.7.5)* + +### Day 5: EntityMemory & Week 12 Wrap-up + +**Goal**: Add EntityMemory for tracking project entities and finalize Week 12. + +**Task Breakdown:** + +1. **EntityMemory Integration** (4 hours) + - Update `memory_service.py` to add EntityMemory support + - Track entities: projects, plans, contexts, recent changes + - Integrate with existing SQL persistence + - Add methods: `track_entity()`, `get_entities()`, `clear_entities()` + +2. **Testing & Documentation** (4 hours) + - Write tests for EntityMemory in `features/memory_service_coverage.feature` + - Update memory documentation with entity tracking + - Create examples of cross-session entity recall + - Verify coverage remains >85% + +3. **Week 12 Review** (1 hour) + - Run full test suite (`nox`) + - Review coverage report + - Update implementation_plan.md with completion status + - Tag Week 12 completion in git + +**Expected Outcomes:** +- EntityMemory tracks project entities across sessions +- Entities persisted to database and reloaded correctly +- All Week 12 tests passing (100+ new scenarios) +- Coverage maintained above 85% (target: 95%) +- Implementation plan updated with all discoveries + +*(Success criteria checklist items have been moved to the Implementation Checklist section under Memory Integration)* + +--- + +**Week 12 Estimated Completion: 2025-11-25** + +**Deferred to Future Weeks:** +- LangSmith observability setup (optional, can be done in Phase 6) +- Full REPL mode implementation (Phase 2 Stage 4) +- Server mode implementation (Phase 2 Stage 5) + + ### Phase 3: Persistence and ORM Abstraction with LangGraph State Management 1. Model domain entities using SQLAlchemy 2.x declarative mappings (users, orgs, invites, sessions, projects, plans, branches, diffs, contexts, conversations, execution history, logs, custom models/providers, settings, locks, migrations) integrated with LangGraph's persistent state store. 2. Define repository interfaces and a Unit of Work abstraction in `cleveragents.domain`, enabling swap-in adapters with LangGraph checkpointing support. @@ -2179,20 +2347,48 @@ The plan is now actionable, pragmatic, and based on real experience rather than - Streaming event consumption in CLI commands - Checkpointing and resume functionality testing -**Next Priority Tasks:** -1. Create ContextAnalysisAgent with LangChain document loaders -2. Add EntityMemory for project tracking across sessions -3. Integrate LangGraph streaming events into CLI for real-time feedback -4. Implement auto-debug workflow using LangGraph conditional edges -5. Add LangSmith tracing for workflow debugging and performance monitoring +**Week 12 Priority Tasks (2025-11-20):** + +**COMPLETED Week 11:** +1. ✅ ContextAnalysisAgent with LangChain document loaders (468 lines, 19 scenarios, 146 steps) + +**ACTIVE Week 12 Tasks (in priority order):** + +1. **CLI Streaming Integration** (Stage 2.7.3) - HIGHEST PRIORITY + - Update PlanService with streaming methods + - Add `--stream` flag to plan generation commands + - Implement real-time progress indicators with Rich + - Test streaming events and output formatting + - Estimated: 2-3 days + +2. **AutoDebugGraph Implementation** (Stage 2.7.5) + - Create auto_debug.py with LangGraph workflow + - Integrate with plan build error handling + - Add retry logic and fix validation + - Write comprehensive tests (Behave + Robot) + - Estimated: 2-3 days + +3. **EntityMemory Integration** (Memory enhancement) + - Add EntityMemory to MemoryService + - Track project entities across sessions + - Integrate with existing SQL chat history + - Test cross-session persistence + - Estimated: 1 day + +4. **LangSmith Observability** (Stage 2.7.2) - OPTIONAL + - Configure LangSmith environment variables (user-configurable) + - Add trace metadata and tagging + - Create observability documentation + - Test with real LangSmith project + - Estimated: 1 day (optional, can defer) **Phase 2 LangChain/LangGraph Integration Status:** -- [X] Foundation Setup (Week 9): 100% Complete -- [X] Core PlanGenerationGraph (Week 10): 100% Complete with full test coverage -- [X] ContextAnalysisAgent (Week 11): Complete (Behave coverage in place) -- [ ] Memory Integration (Week 11): In Progress -- [ ] CLI Streaming (Week 12): Planned -- [ ] Auto-Debug Graph (Week 12): Planned +- ✅ Foundation Setup (Week 9): 100% Complete +- ✅ Core PlanGenerationGraph (Week 10): 100% Complete with full test coverage +- ✅ ContextAnalysisAgent (Week 11): Complete (Behave coverage in place) +- 🔄 Memory Integration (Week 11): In Progress (tracked in Implementation Checklist) +- 📋 CLI Streaming (Week 12): Planned (tracked in Implementation Checklist) +- 📋 Auto-Debug Graph (Week 12): Planned (tracked in Implementation Checklist) **Testing Infrastructure Achievements:** - Comprehensive Behave test suites for all LangGraph workflows @@ -3546,32 +3742,72 @@ Each phase item includes mandatory **Code**, **Document**, and **Tests** bullets - [ ] Verify output formatting - [ ] Test error handling during streaming - [ ] Ensure flags (`--stream`, `--quiet`) work correctly - - [ ] Update CLI documentation - - [ ] Document `--stream` flag usage - - [ ] Add examples of streaming output - - [ ] Explain when streaming is beneficial - - [ ] Show how to disable streaming - - [ ] Stage 2.7.4: Remaining Agent Graphs - - [ ] Implement ContextAnalysisGraph - - [ ] Create `src/cleveragents/agents/context_analysis.py` - - [ ] Define `ContextAnalysisState` TypedDict - - [ ] Build graph with nodes: load_files → analyze_dependencies → summarize - - [ ] Add checkpointing and retry logic - - [ ] Write unit tests and update Behave scenarios - - [ ] Implement AutoDebugGraph - - [ ] Create `src/cleveragents/agents/auto_debug.py` - - [ ] Define `AutoDebugState` TypedDict - - [ ] Build graph with nodes: analyze_error → generate_fix → validate_fix - - [ ] Add checkpointing and retry logic - - [ ] Write unit tests and update Behave scenarios - - [ ] Integrate agents into services - - [ ] Update `ContextService` to use `ContextAnalysisGraph` - - [ ] Add streaming support to context commands - - [ ] Add LangSmith metadata for context analysis - - [ ] Test end-to-end integration - - [ ] Memory Integration + - [ ] Update CLI documentation + - [ ] Document `--stream` flag usage + - [ ] Add examples of streaming output + - [ ] Explain when streaming is beneficial + - [ ] Show how to disable streaming + - [ ] Verify CLI Streaming Success Criteria + - [ ] `agents tell "add feature" --stream` shows real-time progress + - [ ] Each graph node displays its status (⏳ running, ✓ complete) + - [ ] Timing information shown for each stage + - [ ] Ctrl+C cancels gracefully without errors + - [ ] All tests passing with >85% coverage + - [X] Stage 2.7.4: Context Analysis Agent (COMPLETE 2025-11-20) + - [X] Implement ContextAnalysisAgent + - [X] Created `src/cleveragents/agents/context_analysis.py` (468 lines) + - [X] Defined `ContextAnalysisState` TypedDict with all required fields + - [X] Built 5-node graph: load_files → analyze_dependencies → chunk_documents → score_relevance → summarize_context + - [X] Added MemorySaver checkpointing for resumable workflows + - [X] Implemented invoke, ainvoke, stream, and astream methods + - [X] Added proper error handling and fallback strategies + - [X] Comprehensive Testing (19 scenarios, 146 steps, 8 Robot tests) + - [X] Created `features/context_analysis_agent_coverage.feature` with full test coverage + - [X] Implemented `features/steps/context_analysis_agent_coverage_steps.py` with all step definitions + - [X] Created `robot/context_analysis_agent.robot` with 8 integration tests + - [X] Created helper script `robot/test_context_analysis.py` for complex Python logic + - [X] All tests passing (nox -s unit_tests, nox -s integration_tests) + - [X] Fixed LangGraph checkpointing with thread_id in config + - [ ] Integrate agents into services + - [ ] Update `ContextService` to use `ContextAnalysisAgent` + - [ ] Add streaming support to context commands + - [ ] Add LangSmith metadata for context analysis + - [ ] Test end-to-end integration + - [ ] Stage 2.7.5: Auto-Debug Agent Implementation (NEXT PRIORITY) + - [ ] Implement AutoDebugGraph + - [ ] Create `src/cleveragents/agents/auto_debug.py` + - [ ] Define `AutoDebugState` TypedDict + - [ ] Build graph with nodes: analyze_error → generate_fix → validate_fix → apply_fix + - [ ] Add conditional edges for retry logic and validation failures + - [ ] Add checkpointing and retry logic with max_retries + - [ ] Write unit tests and update Behave scenarios + - [ ] Integrate with PlanService + - [ ] Call AutoDebugGraph when plan build fails + - [ ] Add streaming support for auto-debug progress + - [ ] Add LangSmith metadata for debugging traces + - [ ] Test end-to-end auto-debug workflow + - [ ] Verify AutoDebugGraph Success Criteria + - [ ] AutoDebugGraph successfully analyzes common error types + - [ ] Fix generation produces valid code changes + - [ ] Validation catches syntax errors before applying + - [ ] Retry logic works with configurable max attempts + - [ ] All tests passing (15+ Behave scenarios, 8+ Robot tests) + - [ ] Memory Integration - [X] Add ConversationBufferMemory to PlanService - [ ] Add EntityMemory for project tracking + - [ ] Update `memory_service.py` to add EntityMemory support + - [ ] Track entities: projects, plans, contexts, recent changes + - [ ] Integrate with existing SQL persistence + - [ ] Add methods: `track_entity()`, `get_entities()`, `clear_entities()` + - [ ] Write tests for EntityMemory in `features/memory_service_coverage.feature` + - [ ] Update memory documentation with entity tracking + - [ ] Create examples of cross-session entity recall + - [ ] Verify EntityMemory Success Criteria + - [ ] EntityMemory tracks project entities across sessions + - [ ] Entities persisted to database and reloaded correctly + - [ ] All Week 12 tests passing (100+ new scenarios) + - [ ] Coverage maintained above 85% (target: 95%) + - [ ] Implementation plan updated with all discoveries - [X] Implement SQLChatMessageHistory for persistence - [ ] Add vector store for semantic search (optional) - [ ] Stage 2.7.5: Documentation & Examples @@ -3590,15 +3826,15 @@ Each phase item includes mandatory **Code**, **Document**, and **Tests** bullets - [ ] Document state TypedDict fields - [ ] Show example configurations as inline code - [ ] Prepare for Docusaurus API reference generation - - [ ] Stage 2.7 Completion Criteria - - [ ] All Behave tests pass for plan_generation_agent_coverage.feature - - [ ] All Behave tests pass for context_analysis_agent_coverage.feature - - [ ] All Behave tests pass for auto_debug_agent_coverage.feature - - [ ] LangSmith traces appear when API key is configured - - [ ] CLI commands support `--stream` flag with real-time output - - [ ] Documentation includes observability and streaming guides - - [ ] All agent graphs follow consistent interface patterns - - [ ] 90%+ test coverage for agents package + - [ ] Stage 2.7 Completion Criteria + - [X] All Behave tests pass for plan_generation_agent_coverage.feature (15 scenarios, 91 steps - PASSING) + - [X] All Behave tests pass for context_analysis_agent_coverage.feature (19 scenarios, 146 steps - PASSING) + - [ ] All Behave tests pass for auto_debug_agent_coverage.feature (NOT YET IMPLEMENTED) + - [ ] LangSmith traces appear when API key is configured (NOT YET TESTED) + - [ ] CLI commands support `--stream` flag with real-time output (NOT YET IMPLEMENTED) + - [ ] Documentation includes observability and streaming guides (NOT YET WRITTEN) + - [X] All agent graphs follow consistent interface patterns (BaseAgent provides consistency) + - [X] 90%+ test coverage for agents package (95% overall coverage, exceeds requirement) - [ ] Stage 3: Async Infrastructure (Weeks 11-12) - [ ] Implement async command execution @@ -4742,3 +4978,5 @@ Each phase item includes mandatory **Code**, **Document**, and **Tests** bullets - [ ] Tune checkpoint frequency --- + +---