From 7caac2714e6180e2a098b85b9d1e5fecd3a4d066 Mon Sep 17 00:00:00 2001 From: "Brent E. Edwards" Date: Wed, 11 Feb 2026 00:59:58 +0000 Subject: [PATCH] refactor: changes requested by Rui Removed `run-semgrep.sh` and automatically run setup-dev.sh --- .devcontainer/post-create.sh | 11 + docs/development/quality-automation.md | 87 +- implementation_plan.md | 16300 ++++++++++++----------- scripts/run-semgrep.sh | 7 - 4 files changed, 8492 insertions(+), 7913 deletions(-) create mode 100755 .devcontainer/post-create.sh delete mode 100755 scripts/run-semgrep.sh diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh new file mode 100755 index 000000000..7d1e4d56f --- /dev/null +++ b/.devcontainer/post-create.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +# Post-create hook for the VS Code Dev Container. +# Called automatically by devcontainer.json after the container is created. +# Delegates to the shared developer-setup script so that the logic lives in +# one place. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" + +exec bash "$REPO_ROOT/scripts/setup-dev.sh" diff --git a/docs/development/quality-automation.md b/docs/development/quality-automation.md index fbacb44c4..d7d8f1ea0 100644 --- a/docs/development/quality-automation.md +++ b/docs/development/quality-automation.md @@ -31,24 +31,24 @@ Pre-commit hooks run automatically on every `git commit`. They are configured in ### Installed Hooks -| Hook | Tool | Purpose | Auto-fix | -|------|------|---------|----------| -| `no-commit-to-branch` | pre-commit-hooks | Prevents commits to `main` | No | -| `check-yaml` | pre-commit-hooks | Validates YAML syntax | No | -| `check-toml` | pre-commit-hooks | Validates TOML syntax | No | -| `check-json` | pre-commit-hooks | Validates JSON syntax | No | -| `check-merge-conflict` | pre-commit-hooks | Detects merge conflict markers | No | -| `check-added-large-files` | pre-commit-hooks | Prevents files >500KB | No | -| `end-of-file-fixer` | pre-commit-hooks | Ensures files end with newline | Yes | -| `trailing-whitespace` | pre-commit-hooks | Removes trailing whitespace | Yes | -| `debug-statements` | pre-commit-hooks | Detects `pdb`/`breakpoint` | No | -| `ruff-format` | ruff | Code formatting | Yes | -| `ruff` | ruff | Linting with safe auto-fixes | Yes | -| `bandit` | bandit | Security scanning | No | -| `vulture` | vulture | Dead code detection | No | -| `pyright` | pyright | Type checking (src/ only) | No | -| `semgrep-eval-exec` | semgrep | eval/exec detection (optional) | No | -| `commitizen` | commitizen | Commit message format (commit-msg stage) | No | +| Hook | Tool | Purpose | Auto-fix | +| ------------------------- | ---------------- | ---------------------------------------- | -------- | +| `no-commit-to-branch` | pre-commit-hooks | Prevents commits to `main` | No | +| `check-yaml` | pre-commit-hooks | Validates YAML syntax | No | +| `check-toml` | pre-commit-hooks | Validates TOML syntax | No | +| `check-json` | pre-commit-hooks | Validates JSON syntax | No | +| `check-merge-conflict` | pre-commit-hooks | Detects merge conflict markers | No | +| `check-added-large-files` | pre-commit-hooks | Prevents files >500KB | No | +| `end-of-file-fixer` | pre-commit-hooks | Ensures files end with newline | Yes | +| `trailing-whitespace` | pre-commit-hooks | Removes trailing whitespace | Yes | +| `debug-statements` | pre-commit-hooks | Detects `pdb`/`breakpoint` | No | +| `ruff-format` | ruff | Code formatting | Yes | +| `ruff` | ruff | Linting with safe auto-fixes | Yes | +| `bandit` | bandit | Security scanning | No | +| `vulture` | vulture | Dead code detection | No | +| `pyright` | pyright | Type checking (src/ only) | No | +| `semgrep-eval-exec` | semgrep | eval/exec detection (optional) | No | +| `commitizen` | commitizen | Commit message format (commit-msg stage) | No | ### Running Hooks Manually @@ -71,21 +71,21 @@ The CI pipeline runs on **Forgejo Actions** (`.forgejo/workflows/ci.yml`). ### CI Jobs -| Job | Trigger | Purpose | Failure Impact | -|-----|---------|---------|----------------| -| `lint` | Push/PR | Ruff format + lint check | Blocks merge | -| `typecheck` | Push/PR | Pyright type checking | Blocks merge | -| `security` | Push/PR | Bandit + Vulture | Blocks merge | -| `quality` | Push/PR | Radon complexity check | Blocks merge (grade F) | -| `behave` | Push/PR | BDD tests (Python 3.11-3.13 matrix) | Blocks merge | -| `coverage` | Push/PR | Coverage measurement | Blocks merge (<85%) | -| `build` | Push/PR | Wheel build | Blocks release | -| `docker` | Push/PR | Docker image build + test | Blocks deployment | -| `helm` | Push/PR | Helm chart lint + template | Blocks deployment | +| Job | Trigger | Purpose | Failure Impact | +| ----------- | ------- | ----------------------------------- | ---------------------- | +| `lint` | Push/PR | Ruff format + lint check | Blocks merge | +| `typecheck` | Push/PR | Pyright type checking | Blocks merge | +| `security` | Push/PR | Bandit + Vulture | Blocks merge | +| `quality` | Push/PR | Radon complexity check | Blocks merge (grade F) | +| `behave` | Push/PR | BDD tests (Python 3.11-3.13 matrix) | Blocks merge | +| `coverage` | Push/PR | Coverage measurement | Blocks merge (<85%) | +| `build` | Push/PR | Wheel build | Blocks release | +| `docker` | Push/PR | Docker image build + test | Blocks deployment | +| `helm` | Push/PR | Helm chart lint + template | Blocks deployment | ### Nightly Quality -A nightly workflow (`.forgejo/workflows/nightly-quality.yml`) runs at midnight UTC: +A nightly workflow (`.forgejo/workflows/nightly-quality.yml`) runs on all branches at midnight UTC: - Full lint, type check, security scan - Complete Behave test suite with coverage @@ -98,6 +98,7 @@ A nightly workflow (`.forgejo/workflows/nightly-quality.yml`) runs at midnight U ### Bandit Configuration Configured in `pyproject.toml` under `[tool.bandit]`: + - Targets: `src/cleveragents` only - Severity threshold: MEDIUM (pre-commit), HIGH (CI fail gate) - Excludes: tests, features, build directories @@ -105,6 +106,7 @@ Configured in `pyproject.toml` under `[tool.bandit]`: ### Semgrep Rules Custom rules in `.semgrep.yml`: + - `no-eval`: Blocks `eval()` usage - `no-exec`: Blocks `exec()` usage - `no-compile-exec`: Blocks `compile()` with exec mode @@ -120,6 +122,7 @@ Vulture checks `src/cleveragents/` for unused code with 80% confidence threshold ### False Positive Whitelist Known false positives are listed in `vulture_whitelist.py`: + - `exc_tb`: Required by `__exit__` protocol but not used in body - `build_data`: Required by mapping interface in legacy migrator @@ -129,16 +132,14 @@ Add new entries when vulture flags intentionally unused code. Radon measures cyclomatic complexity: -| Grade | Complexity | Status | -|-------|-----------|--------| -| A | 1-5 | Excellent | -| B | 6-10 | Good | -| C | 11-15 | Moderate - review recommended | -| D | 16-20 | Complex - refactoring recommended | -| E | 21-30 | Very complex - must refactor | -| F | 31+ | Extremely complex - CI fails | - -Current project average: **A (3.56)** +| Grade | Complexity | Status | +| ----- | ---------- | --------------------------------- | +| A | 1-5 | Excellent | +| B | 6-10 | Good | +| C | 11-15 | Moderate - review recommended | +| D | 16-20 | Complex - refactoring recommended | +| E | 21-30 | Very complex - must refactor | +| F | 31+ | Extremely complex - CI fails | ## Quality Gate Script @@ -149,6 +150,7 @@ python scripts/check-quality-gates.py --coverage-min 85 --complexity-max F ``` Gates checked: + 1. Coverage >= 85% 2. Pyright: 0 type errors 3. Bandit: 0 high-severity security issues @@ -158,20 +160,25 @@ Gates checked: ## Troubleshooting ### Pre-commit hooks not running + ```bash pre-commit install pre-commit install --hook-type commit-msg ``` ### Pyright errors on clean code + Check `pyrightconfig.json` for configuration. Pyright uses `strict` mode. ### Bandit false positive + If bandit flags intentionally safe code, consider if there's a safer alternative first. Only as a last resort, add to `[tool.bandit]` `skips` in `pyproject.toml`. ### Vulture false positive + Add the symbol name to `vulture_whitelist.py` with a comment explaining why. ### Coverage below 85% + Run `nox -s coverage_report` to see which lines are uncovered, then add Behave scenarios. diff --git a/implementation_plan.md b/implementation_plan.md index 2fe070d81..26931088a 100644 --- a/implementation_plan.md +++ b/implementation_plan.md @@ -1,6 +1,7 @@ # CleverAgents Implementation Plan ## **CRITICAL**: Execute These Rules Without Exception + - **Strictly adhere to guidelines in `./CONTRIBUTING.md`**: All rules and guidelines outlined in this file must be strictly followed at all times. - **Python implementation scope only**: Every action described here pertains to building an idiomatic Python codebase that implements the CleverAgents architecture. - **NO BACKWARDS COMPATIBILITY**: CleverAgents is a NEW standalone project. Do NOT maintain any backwards compatibility. No migration guides, no compatibility shims, no support for old configurations or data. @@ -38,27 +39,26 @@ When connected to a **CleverAgents server** (developed independently), the clien While CleverAgents leverages LangGraph and LangChain for the underlying LLM runtime primitives (tool calling, graphs, routing), its value lies in what it builds on top: -* **CleverAgents** provides: - - * A **first-class plan lifecycle** (Action/Strategize/Execute/Apply) for breaking down and tracking complex work, - * A **project + resource model** for grounding tasks in real codebases, databases, documents, and infrastructure, - * A consistent **actor abstraction** for defining and composing intelligent agents, - * A consistent **skill abstraction** for anything an agent can execute, - * A **sandbox + checkpoint** safety model for safe, reversible execution, - * A **CLI/TUI/Web UX** for controlling and monitoring large multi-step autonomous work. +- **CleverAgents** provides: + - A **first-class plan lifecycle** (Action/Strategize/Execute/Apply) for breaking down and tracking complex work, + - A **project + resource model** for grounding tasks in real codebases, databases, documents, and infrastructure, + - A consistent **actor abstraction** for defining and composing intelligent agents, + - A consistent **skill abstraction** for anything an agent can execute, + - A **sandbox + checkpoint** safety model for safe, reversible execution, + - A **CLI/TUI/Web UX** for controlling and monitoring large multi-step autonomous work. ### Key Concepts -| Concept | Definition | -|---------|------------| -| **Plan** | A tracked lifecycle for a single unit-of-work (which may spawn subplans). Phases: Action -> Strategize -> Execute -> Apply | -| **Action** | A reusable plan template. Created via CLI commands (NOT YAML files). | -| **Actor** | Anything conversational; may be a single agent/LLM or an entire graph. Defined via YAML configuration files. Always named `/`. | -| **Project** | A collection of resources + configuration. Created via CLI commands (NOT YAML files). | -| **Resource** | Anything that can be read/written/queried. Each resource defines its own sandbox strategy. | -| **Skill** | A callable capability defined inline in actor YAML as tool nodes. | -| **Namespace** | Scoping mechanism: `local/`, `/`, `/`, or provider namespaces (`openai/`, `anthropic/`). | -| **Decision** | A recorded choice point made during Strategize that affects downstream work. Forms a tree enabling correction and replay. | +| Concept | Definition | +| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| **Plan** | A tracked lifecycle for a single unit-of-work (which may spawn subplans). Phases: Action -> Strategize -> Execute -> Apply | +| **Action** | A reusable plan template. Created via CLI commands (NOT YAML files). | +| **Actor** | Anything conversational; may be a single agent/LLM or an entire graph. Defined via YAML configuration files. Always named `/`. | +| **Project** | A collection of resources + configuration. Created via CLI commands (NOT YAML files). | +| **Resource** | Anything that can be read/written/queried. Each resource defines its own sandbox strategy. | +| **Skill** | A callable capability defined inline in actor YAML as tool nodes. | +| **Namespace** | Scoping mechanism: `local/`, `/`, `/`, or provider namespaces (`openai/`, `anthropic/`). | +| **Decision** | A recorded choice point made during Strategize that affects downstream work. Forms a tree enabling correction and replay. | --- @@ -74,18 +74,21 @@ While CleverAgents leverages LangGraph and LangChain for the underlying LLM runt ### Core Architectural Requirements **Scalability**: The system must handle massive codebases (50,000+ files) through: + - Three-tier memory architecture (hot/warm/cold) - Hierarchical task decomposition - Bounded dependency closures - Lazy resource sandboxing **Reliability**: Prevent cascading failures through: + - Complete execution isolation via sandboxes - Multi-layer semantic error prevention - Checkpoint-based rollback capabilities - Invariant enforcement throughout execution **Autonomy with Control**: Progressive automation through: + - Three-level automation system (manual, review-before-apply, full) - Decision correction without full re-execution - Confidence-based escalation @@ -94,6 +97,7 @@ While CleverAgents leverages LangGraph and LangChain for the underlying LLM runt --- ## Continuous Testing and Documentation Policy + - Do not mark any parent checklist item complete until **all** subordinate Code, Document, Tests tasks and any generated `Fix – …` tasks are resolved and the associated Notes section has the latest context. - Every time new information appears, extend the corresponding Notes section immediately with explicit references to code locations and decisions. - Maintain a running catalog of Behave commands, Robot suites, fixtures, and environments in the Notes sections to assist subsequent contributors. @@ -102,6 +106,7 @@ While CleverAgents leverages LangGraph and LangChain for the underlying LLM runt --- ## Completion Criteria + The implementation concludes only when every checklist item and spawned remediation task is checked, all Notes sections contain final decisions and references, and the full Behave and Robot test suites (unit, integration, end-to-end, benchmarking, packaging, documentation) pass without outstanding failures. --- @@ -115,11 +120,11 @@ Action -> Strategize -> Execute -> Apply -> Applied (terminal) ``` | Current Phase | Command Verb | Next Phase | -|---------------|--------------|------------| -| (none) | `create` | Action | -| Action | `use` | Strategize | -| Strategize | `execute` | Execute | -| Execute | `apply` | Applied | +| ------------- | ------------ | ---------- | +| (none) | `create` | Action | +| Action | `use` | Strategize | +| Strategize | `execute` | Execute | +| Execute | `apply` | Applied | ### Plan States (Per Phase) @@ -130,24 +135,28 @@ Action -> Strategize -> Execute -> Apply -> Applied (terminal) ### Key Architectural Components **Multi-tier Memory System**: + - **Hot tier**: Immediate working context in LLM context window - **Warm tier**: Recent decisions and contexts from current plan tree - **Cold tier**: Historical decisions from past plans, queryable but not in active memory - Context snapshots with cryptographic hashes preserve complete decision context **Dependency Closure Computation**: + - Resource-aware analysis during Strategize - Hierarchical scoping with explicit resource lists - Lazy expansion prevents closure explosion - Interface-based boundaries for modular changes **Execution Coordination**: + - Complete isolation via per-plan sandboxes - Resource-specific sandbox strategies (git worktrees, transactions, etc.) - Hierarchical merge resolution - Checkpoint-based coordination for rollback **Semantic Error Prevention**: + - Decision-time validation during Strategize - Execution-time semantic guards in actors - Invariant enforcement throughout @@ -155,12 +164,12 @@ Action -> Strategize -> Execute -> Apply -> Applied (terminal) ### Namespace Rules -| Namespace | Scope | Storage | -|-----------|-------|---------| -| `local/` | Current machine only | Local database | -| `/` | Personal server namespace | Server database | -| `/` | Organization namespace | Server database | -| `openai/`, `anthropic/`, etc. | Built-in LLM actors | N/A (built-in) | +| Namespace | Scope | Storage | +| ----------------------------- | ------------------------- | --------------- | +| `local/` | Current machine only | Local database | +| `/` | Personal server namespace | Server database | +| `/` | Organization namespace | Server database | +| `openai/`, `anthropic/`, etc. | Built-in LLM actors | N/A (built-in) | ### Configuration Philosophy @@ -177,14 +186,14 @@ All environment variables needed during testing are stored in the `.env` file in ### Current Environment Variables -| Variable Name | Service | Usage | -|--------------|---------|-------| -| `OPENROUTER_API_KEY` | OpenRouter | Access to multiple LLM models through OpenRouter API | -| `OPENAI_API_KEY` | OpenAI | Direct access to OpenAI models (GPT-3.5, GPT-4, etc.) | -| `ANTHROPIC_API_KEY` | Anthropic | Access to Claude models | -| `GOOGLE_API_KEY` | Google AI | Access to Google's API for web searches | -| `GEMINI_API_KEY` | Google Gemini | Access to Google's Gemini models | -| `HF_TOKEN` | Hugging Face | Access to Hugging Face models and datasets | +| Variable Name | Service | Usage | +| -------------------- | ------------- | ----------------------------------------------------- | +| `OPENROUTER_API_KEY` | OpenRouter | Access to multiple LLM models through OpenRouter API | +| `OPENAI_API_KEY` | OpenAI | Direct access to OpenAI models (GPT-3.5, GPT-4, etc.) | +| `ANTHROPIC_API_KEY` | Anthropic | Access to Claude models | +| `GOOGLE_API_KEY` | Google AI | Access to Google's API for web searches | +| `GEMINI_API_KEY` | Google Gemini | Access to Google's Gemini models | +| `HF_TOKEN` | Hugging Face | Access to Hugging Face models and datasets | --- @@ -228,18 +237,20 @@ All 10 ADRs have been created and package structure established. See the Phase 1 The following work from the previous implementation has been completed and will be preserved/adapted: #### Completed Infrastructure -- [X] LangChain/LangGraph dependencies and integration (ADR-011) -- [X] PlanGenerationGraph, ContextAnalysisAgent, AutoDebugGraph workflows -- [X] Memory service with EntityMemory -- [X] SQLite persistence with Alembic migrations -- [X] CLI streaming integration -- [X] Provider adapters (OpenAI, Anthropic, Google, OpenRouter) -- [X] Actor configuration system (Stage 7.5) -- [X] Test coverage at 95% + +- [x] LangChain/LangGraph dependencies and integration (ADR-011) +- [x] PlanGenerationGraph, ContextAnalysisAgent, AutoDebugGraph workflows +- [x] Memory service with EntityMemory +- [x] SQLite persistence with Alembic migrations +- [x] CLI streaming integration +- [x] Provider adapters (OpenAI, Anthropic, Google, OpenRouter) +- [x] Actor configuration system (Stage 7.5) +- [x] Test coverage at 95% #### Phase 2 Notes (Preserved from Previous Work) **2025-11-22**: Week 12 Complete, Phase 2 Core Functionality DONE + - CLI Streaming Integration fully implemented - AutoDebugGraph Implementation complete - Mock provider enhancements with configurable failure modes @@ -250,51 +261,54 @@ The following work from the previous implementation has been completed and will **2025-12-17**: Stage 7 performance optimization complete **2026-02-02**: Stage 7.5 Actor Configuration System complete **2026-02-05**: Stage A1 & A2 Complete - Plan and Action Domain Models + - Created `src/cleveragents/domain/models/core/plan.py` with: - - `PlanPhase` enum (ACTION, STRATEGIZE, EXECUTE, APPLY, APPLIED) - - `ActionState` enum (AVAILABLE, DRAFT, ARCHIVED) - - `ProcessingState` enum (QUEUED, PROCESSING, ERRORED, COMPLETE, CANCELLED) - - `NamespacedName` model with parse() and str() methods - - `PlanIdentity` model with ULID validation - - `Plan` model with full lifecycle support - - `can_transition()` function for phase transition validation + - `PlanPhase` enum (ACTION, STRATEGIZE, EXECUTE, APPLY, APPLIED) + - `ActionState` enum (AVAILABLE, DRAFT, ARCHIVED) + - `ProcessingState` enum (QUEUED, PROCESSING, ERRORED, COMPLETE, CANCELLED) + - `NamespacedName` model with parse() and str() methods + - `PlanIdentity` model with ULID validation + - `Plan` model with full lifecycle support + - `can_transition()` function for phase transition validation - Created `src/cleveragents/domain/models/core/action.py` with: - - `ActionArgument` model with parse() method for CLI argument parsing - - `Action` model with strategy/execution actor references - - Argument validation including type checking + - `ActionArgument` model with parse() method for CLI argument parsing + - `Action` model with strategy/execution actor references + - Argument validation including type checking - Added 52 Behave test scenarios across 2 feature files: - - `features/plan_model.feature` (30 scenarios) - - `features/action_model.feature` (22 scenarios) + - `features/plan_model.feature` (30 scenarios) + - `features/action_model.feature` (22 scenarios) - All new tests pass, existing tests unaffected **2026-02-05**: Stage A3 Complete - PlanLifecycleService + - Created `src/cleveragents/application/services/plan_lifecycle_service.py` with: - - Full plan lifecycle management (Action -> Strategize -> Execute -> Apply -> Applied) - - Action CRUD operations (create, get, list, make_available, archive) - - Plan creation via `use_action()` which transitions Action to Strategize - - Phase transition methods: `execute_plan()`, `apply_plan()` - - State management: `start_*()`, `complete_*()`, `fail_*()` for each phase - - `cancel_plan()` for non-terminal plans - - Custom exceptions: `InvalidPhaseTransitionError`, `ActionNotAvailableError`, `PlanNotReadyError` - - In-memory storage (to be replaced with persistence in Stage A5) + - Full plan lifecycle management (Action -> Strategize -> Execute -> Apply -> Applied) + - Action CRUD operations (create, get, list, make_available, archive) + - Plan creation via `use_action()` which transitions Action to Strategize + - Phase transition methods: `execute_plan()`, `apply_plan()` + - State management: `start_*()`, `complete_*()`, `fail_*()` for each phase + - `cancel_plan()` for non-terminal plans + - Custom exceptions: `InvalidPhaseTransitionError`, `ActionNotAvailableError`, `PlanNotReadyError` + - In-memory storage (to be replaced with persistence in Stage A5) - Added python-ulid dependency for ULID generation - Added 29 Behave test scenarios in `features/plan_lifecycle_service.feature` - Total new test scenarios: 81 (30 + 22 + 29) **2026-02-05**: Stage A4 In Progress - Plan CLI Commands + - Created `src/cleveragents/cli/commands/action.py` with: - - `agents [--data-dir PATH] [--config-path PATH] action create` - Create new action with strategy/execution actors, definition of done, arguments - - `agents [--data-dir PATH] [--config-path PATH] action list` - List actions with filtering by namespace, state - - `agents [--data-dir PATH] [--config-path PATH] action show` - Show action details by ID or name - - `agents [--data-dir PATH] [--config-path PATH] action available` - Make draft action available for use - - `agents [--data-dir PATH] [--config-path PATH] action archive` - Archive an action (soft delete) + - `agents [--data-dir PATH] [--config-path PATH] action create` - Create new action with strategy/execution actors, definition of done, arguments + - `agents [--data-dir PATH] [--config-path PATH] action list` - List actions with filtering by namespace, state + - `agents [--data-dir PATH] [--config-path PATH] action show` - Show action details by ID or name + - `agents [--data-dir PATH] [--config-path PATH] action available` - Make draft action available for use + - `agents [--data-dir PATH] [--config-path PATH] action archive` - Archive an action (soft delete) - Extended `src/cleveragents/cli/commands/plan.py` with v3 lifecycle commands: - - `agents [--data-dir PATH] [--config-path PATH] plan use --project ` - Use action to create plan in Strategize phase - - `agents [--data-dir PATH] [--config-path PATH] plan execute [plan_id]` - Transition plan from Strategize to Execute - - `agents [--data-dir PATH] [--config-path PATH] plan apply [plan_id]` - Transition plan from Execute to Apply - - `agents [--data-dir PATH] [--config-path PATH] plan status [plan_id]` - Show v3 plan status and details - - `agents [--data-dir PATH] [--config-path PATH] plan list` - List v3 lifecycle plans with filtering - - `agents [--data-dir PATH] [--config-path PATH] plan cancel ` - Cancel a non-terminal plan + - `agents [--data-dir PATH] [--config-path PATH] plan use --project ` - Use action to create plan in Strategize phase + - `agents [--data-dir PATH] [--config-path PATH] plan execute [plan_id]` - Transition plan from Strategize to Execute + - `agents [--data-dir PATH] [--config-path PATH] plan apply [plan_id]` - Transition plan from Execute to Apply + - `agents [--data-dir PATH] [--config-path PATH] plan status [plan_id]` - Show v3 plan status and details + - `agents [--data-dir PATH] [--config-path PATH] plan list` - List v3 lifecycle plans with filtering + - `agents [--data-dir PATH] [--config-path PATH] plan cancel ` - Cancel a non-terminal plan - Registered action commands in CLI main.py - Added 15 Behave test scenarios in `features/action_cli.feature` - Total test scenarios: 96 (81 + 15) @@ -317,18 +331,18 @@ The following work from the previous implementation has been completed and will - Fixed all 200 ruff lint findings in `features/` directory -> **0 findings** - **Config-level suppressions** (168 findings): - - Added `per-file-ignores` in `pyproject.toml` for Behave-specific patterns: - - `features/steps/*.py`: F811 (65 redefined `step_impl` — Behave idiom), E501 (long step decorator strings) - - `features/mocks/*.py`, `features/environment.py`: E501 + - Added `per-file-ignores` in `pyproject.toml` for Behave-specific patterns: + - `features/steps/*.py`: F811 (65 redefined `step_impl` — Behave idiom), E501 (long step decorator strings) + - `features/mocks/*.py`, `features/environment.py`: E501 - **Manual fixes** (31 findings across 18 files): - - 11x SIM115: `NamedTemporaryFile` refactored to use `with` context manager (`actor_cli_steps.py`, `actor_cli_run_steps.py`) - - 4x UP028: `for/yield` -> `yield from` (google, openai, openrouter, langchain provider steps) - - 3x SIM117: Nested `with` -> single `with` with parenthesized contexts (`plan_full_coverage_steps.py`, `plan_service_steps.py`) - - 3x RUF005: `list + [item]` -> `[*list, item]` unpacking - - 2x B904: Added `from exc` to `raise` inside `except` (enums, retry patterns) - - 2x RUF012: Added `ClassVar` annotations (`vector_store_service_steps.py`) - - 2x SIM105: `try/except/pass` -> `contextlib.suppress(Exception)` - - 1x each: B007 (unused loop var), B018 (noqa suppression), F821 (missing `Any` import), SIM102 (collapsible if), I001 (auto-fixed unsorted import) + - 11x SIM115: `NamedTemporaryFile` refactored to use `with` context manager (`actor_cli_steps.py`, `actor_cli_run_steps.py`) + - 4x UP028: `for/yield` -> `yield from` (google, openai, openrouter, langchain provider steps) + - 3x SIM117: Nested `with` -> single `with` with parenthesized contexts (`plan_full_coverage_steps.py`, `plan_service_steps.py`) + - 3x RUF005: `list + [item]` -> `[*list, item]` unpacking + - 2x B904: Added `from exc` to `raise` inside `except` (enums, retry patterns) + - 2x RUF012: Added `ClassVar` annotations (`vector_store_service_steps.py`) + - 2x SIM105: `try/except/pass` -> `contextlib.suppress(Exception)` + - 1x each: B007 (unused loop var), B018 (noqa suppression), F821 (missing `Any` import), SIM102 (collapsible if), I001 (auto-fixed unsorted import) - **Verification**: All affected behave tests pass (155 scenarios, 0 failures) - **Files modified**: `pyproject.toml` (config), `environment.py`, and 17 step files in `features/steps/` @@ -336,36 +350,36 @@ The following work from the previous implementation has been completed and will - Fixed all 16 pre-existing bandit findings (2 HIGH, 3 MEDIUM, 11 LOW) -> **0 findings** - **Security hardening** (HIGH+MEDIUM): - - Replaced `jinja2.Environment` with `jinja2.sandbox.SandboxedEnvironment` in `yaml_template_engine.py` and `stream_router.py` — prevents template injection - - Added `_validate_code_ast()` helper: AST-based pre-validation for `exec()` in `SimpleToolAgent` — rejects imports, `exec()`/`eval()`/`compile()`/`__import__()`/`getattr()`/`setattr()` calls, global/nonlocal statements - - Added `_validate_lambda_ast()` helper: restricts transform `eval()` to lambda-only expressions via AST parsing - - Suppressed `0.0.0.0` bind default (`# nosec B104`) — intentional, configurable via `CLEVERAGENTS_SERVER_HOST` + - Replaced `jinja2.Environment` with `jinja2.sandbox.SandboxedEnvironment` in `yaml_template_engine.py` and `stream_router.py` — prevents template injection + - Added `_validate_code_ast()` helper: AST-based pre-validation for `exec()` in `SimpleToolAgent` — rejects imports, `exec()`/`eval()`/`compile()`/`__import__()`/`getattr()`/`setattr()` calls, global/nonlocal statements + - Added `_validate_lambda_ast()` helper: restricts transform `eval()` to lambda-only expressions via AST parsing + - Suppressed `0.0.0.0` bind default (`# nosec B104`) — intentional, configurable via `CLEVERAGENTS_SERVER_HOST` - **Code quality** (LOW): - - Replaced 6 `assert` statements with proper `if`/`raise` (TypeError, RuntimeError, typer.BadParameter) — asserts stripped in optimized bytecode - - Replaced `try/except/pass` with `contextlib.suppress(Exception)` (2 locations in dispose()) - - Added logging to previously-silent exception handlers (migration_runner, nodes retry loop) - - Suppressed false positive `"token_count": 0` flagged as hardcoded password (`# nosec B105`) + - Replaced 6 `assert` statements with proper `if`/`raise` (TypeError, RuntimeError, typer.BadParameter) — asserts stripped in optimized bytecode + - Replaced `try/except/pass` with `contextlib.suppress(Exception)` (2 locations in dispose()) + - Added logging to previously-silent exception handlers (migration_runner, nodes retry loop) + - Suppressed false positive `"token_count": 0` flagged as hardcoded password (`# nosec B105`) - **Files modified**: `stream_router.py`, `yaml_template_engine.py`, `settings.py`, `context_service.py`, `memory_service.py`, `retry_patterns.py`, `context.py` (CLI), `plan_service.py`, `migration_runner.py`, `nodes.py` - **Verification**: `bandit -r src/ -c pyproject.toml` → 0 findings; targeted behave tests pass; smoke tests for AST validation pass **2026-02-09**: Stages Q0, Q1, Q2 Complete - Full Quality Automation Setup [Brent] **Stage Q0 - Pre-commit Hooks:** + - Created `.pre-commit-config.yaml` with 12 hooks across 5 categories: - - Branch protection: `no-commit-to-branch` (prevents commits to main) - - General checks: `check-yaml`, `check-toml`, `check-json`, `check-merge-conflict`, `check-added-large-files`, `end-of-file-fixer`, `trailing-whitespace`, `debug-statements` - - Ruff: `ruff-format` (auto-fix), `ruff` (lint with safe auto-fix) - - Pyright: local system hook running type checking on `src/` only - - Bandit: security scanning with `pyproject.toml` configuration on `src/` only - - Vulture: dead code detection with whitelist at `vulture_whitelist.py` - - Semgrep: custom rules in `.semgrep.yml` for eval/exec/os.system/pickle detection (graceful skip when not installed) - - Commitizen: conventional commit message validation at commit-msg stage + - Branch protection: `no-commit-to-branch` (prevents commits to main) + - General checks: `check-yaml`, `check-toml`, `check-json`, `check-merge-conflict`, `check-added-large-files`, `end-of-file-fixer`, `trailing-whitespace`, `debug-statements` + - Ruff: `ruff-format` (auto-fix), `ruff` (lint with safe auto-fix) + - Pyright: local system hook running type checking on `src/` only + - Bandit: security scanning with `pyproject.toml` configuration on `src/` only + - Vulture: dead code detection with whitelist at `vulture_whitelist.py` + - Semgrep: custom rules in `.semgrep.yml` for eval/exec/os.system/pickle detection (graceful skip when not installed) + - Commitizen: conventional commit message validation at commit-msg stage - Added dev dependencies to `pyproject.toml`: `pre-commit>=3.6.0`, `bandit[toml]>=1.7.5`, `vulture>=2.10`, `radon>=6.0.1` - Added `[tool.bandit]` and `[tool.vulture]` sections to `pyproject.toml` - Created `vulture_whitelist.py` for false positive suppression (exc_tb, build_data) - Created `.semgrep.yml` with 5 custom security rules - Created `scripts/setup-dev.sh` for developer environment setup -- Created `scripts/run-semgrep.sh` wrapper for graceful semgrep execution - Added 4 new nox sessions: `pre_commit`, `security_scan`, `dead_code`, `complexity` - **Discovery**: CI platform is Forgejo (`.forgejo/`), NOT GitHub. Stage Q1 must use Forgejo Actions, not GitHub Actions. - **Discovery**: Pre-existing security findings in production code need remediation (see Q0.8 spawned task) @@ -373,33 +387,35 @@ The following work from the previous implementation has been completed and will - **Discovery**: `features/steps/actor_cli_steps.py` has many F811 (redefined step_impl) violations - Behave pattern - **Discovery**: Average code complexity is A (3.56) across 979 blocks - good baseline - **Discovery**: High complexity methods identified: `LegacyDataMigrator.migrate_project_data` E(37), `ProviderRegistry._create_provider_llm` C(20), `ProviderRegistry.create_ai_provider` C(18) -- Key files: `.pre-commit-config.yaml`, `.semgrep.yml`, `vulture_whitelist.py`, `scripts/setup-dev.sh`, `scripts/run-semgrep.sh` +- Key files: `.pre-commit-config.yaml`, `.semgrep.yml`, `vulture_whitelist.py`, `scripts/setup-dev.sh` **Stage Q1 - CI/CD Pipeline:** + - Extended `.forgejo/workflows/ci.yml` with 3 new jobs: - - `security`: bandit scan (JSON report + high-severity gate) + vulture dead code detection - - `quality`: radon complexity check (grade F fails build) + JSON report - - `coverage`: behave tests with coverage measurement, fail-under=85%, XML artifact + - `security`: bandit scan (JSON report + high-severity gate) + vulture dead code detection + - `quality`: radon complexity check (grade F fails build) + JSON report + - `coverage`: behave tests with coverage measurement, fail-under=85%, XML artifact - Updated `docker` and `helm` jobs to depend on `security` (fail-fast on security issues) - Created `scripts/check-quality-gates.py` aggregating: coverage, typecheck, security, dead code, complexity - All reports uploaded as artifacts for downstream consumption **Stage Q2 - Advanced Automation:** + - Created `.forgejo/workflows/nightly-quality.yml` for nightly quality monitoring: - - Runs at midnight UTC (cron: "0 0 * * *") + manual trigger support - - Full lint, typecheck, security scan (all severities), dead code, complexity analysis - - Behave tests with coverage measurement - - Quality trend JSON generation with timestamp + metrics - - 90-day artifact retention for trend analysis + - Runs at midnight UTC (cron: "0 0 \* \* \*") + manual trigger support + - Full lint, typecheck, security scan (all severities), dead code, complexity analysis + - Behave tests with coverage measurement + - Quality trend JSON generation with timestamp + metrics + - 90-day artifact retention for trend analysis - Created `scripts/check-adr-compliance.py` with AST-based checks for: - - ADR-002: No threading imports in application layer - - ADR-003: Services use constructor dependency injection - - ADR-007: No direct SQLAlchemy usage in service layer + - ADR-002: No threading imports in application layer + - ADR-003: Services use constructor dependency injection + - ADR-007: No direct SQLAlchemy usage in service layer - Added `nox -s adr_compliance` session - Created `.forgejo/pull_request_template.md` with quality checklist - Created `docs/development/quality-automation.md` with full documentation: - - Quick start, pre-commit hooks reference, CI jobs table, security scanning guide - - Complexity monitoring grades, quality gates, troubleshooting + - Quick start, pre-commit hooks reference, CI jobs table, security scanning guide + - Complexity monitoring grades, quality gates, troubleshooting **New nox sessions added:** `pre_commit`, `security_scan`, `dead_code`, `complexity`, `adr_compliance` **Total files created:** 9 new files @@ -408,26 +424,26 @@ The following work from the previous implementation has been completed and will **2026-02-10**: Task 10B.4 - Quality Metrics Baseline Established [Brent] - Ran full quality suite via nox to establish current baseline: - - **Unit Tests**: 105 features, 1613 scenarios, 7555 steps - ALL PASS - - **Lint (ruff)**: 0 findings - - **Typecheck (pyright)**: 0 errors, 0 warnings - - **Security (bandit)**: 0 findings (0 HIGH, 0 MEDIUM, 0 LOW) - - **Dead Code (vulture)**: 0 findings - - **Complexity (radon)**: Average A (3.56), 981 blocks analyzed, no grade-F methods - - High complexity methods to monitor: `LegacyDataMigrator.migrate_project_data` E(37), `Action.validate_arguments` C(20), `ProviderRegistry._create_provider_llm` C(20), `ProviderRegistry.create_ai_provider` C(18), `Settings.resolve_provider_defaults` C(18) - - **Coverage**: 96% (9860 statements, 269 missing, 2852 branches, 213 branch-miss) + - **Unit Tests**: 105 features, 1613 scenarios, 7555 steps - ALL PASS + - **Lint (ruff)**: 0 findings + - **Typecheck (pyright)**: 0 errors, 0 warnings + - **Security (bandit)**: 0 findings (0 HIGH, 0 MEDIUM, 0 LOW) + - **Dead Code (vulture)**: 0 findings + - **Complexity (radon)**: Average A (3.56), 981 blocks analyzed, no grade-F methods + - High complexity methods to monitor: `LegacyDataMigrator.migrate_project_data` E(37), `Action.validate_arguments` C(20), `ProviderRegistry._create_provider_llm` C(20), `ProviderRegistry.create_ai_provider` C(18), `Settings.resolve_provider_defaults` C(18) + - **Coverage**: 96% (9860 statements, 269 missing, 2852 branches, 213 branch-miss) - Fixed pre-existing test failure: `plan_lifecycle_cli_coverage.feature` scenario "Plan lifecycle list shows project summaries" - Rich table column wrapping at narrow terminal widths caused `+1 more` text to be split across rows. Fixed by patching console width to 200 in test setup. - Fixed missing dependency: added `langchain-anthropic>=0.2.0` to `pyproject.toml` (was imported in `src/cleveragents/providers/llm/anthropic_provider.py` but not declared) **2026-02-10**: Task Q1.5 Complete - Branch Protection Rules Documentation [Brent] - Created `docs/development/ci-cd.md` (224 lines) documenting: - - Branch protection rules for `master` (required status checks, review requirements, push/force-push/deletion blocks) - - Step-by-step Forgejo branch protection setup instructions - - Review priority matrix (P0: architecture/security, P1: algorithms, P2: features, P3: tests/docs) - - CI job dependency graph and quality gates summary table - - Nightly quality monitoring reference - - Local development workflow quick-reference + - Branch protection rules for `master` (required status checks, review requirements, push/force-push/deletion blocks) + - Step-by-step Forgejo branch protection setup instructions + - Review priority matrix (P0: architecture/security, P1: algorithms, P2: features, P3: tests/docs) + - CI job dependency graph and quality gates summary table + - Nightly quality monitoring reference + - Local development workflow quick-reference - Cross-references existing `docs/development/quality-automation.md` and `.forgejo/pull_request_template.md` - Required CI checks documented: `lint`, `typecheck`, `security`, `quality`, `behave`, `coverage`, `build` - Review requirement: 1 approving review, selective depth by priority matrix @@ -435,10 +451,10 @@ The following work from the previous implementation has been completed and will **2026-02-10**: Task 10C.1 Complete - Edge Case Test Scenarios [Brent] - Created `features/edge_case_plan_scenarios.feature` (26 scenarios, 141 steps) covering: - - **Concurrent plan execution** (6 scenarios): Duplicate strategize/execute/apply start attempts, concurrent complete+fail on same phase, two plans from same action, concurrent transitions on different plans - - **Resource conflict scenarios** (7 scenarios): Read-only file MODIFY failure, overlapping CREATE+MODIFY on same path, CREATE with None content, MOVE with missing source, DELETE of already-deleted file, file paths with spaces, deeply nested directory creation - - **Validation failure chains** (6 scenarios): Multiple simultaneous argument validation failures (missing + wrong type), unknown + missing arguments combined, empty plan description rejection, ACTION phase with processing state, invalid namespace characters, invalid name characters - - **Rollback edge cases** (7 scenarios): Partial apply failure (first change persists on disk, second unchanged), errored plan cannot restart, fail preserves error message, errored plan not terminal, cancel preserves phase, errored strategize rejects execute, failed strategize rejects complete + - **Concurrent plan execution** (6 scenarios): Duplicate strategize/execute/apply start attempts, concurrent complete+fail on same phase, two plans from same action, concurrent transitions on different plans + - **Resource conflict scenarios** (7 scenarios): Read-only file MODIFY failure, overlapping CREATE+MODIFY on same path, CREATE with None content, MOVE with missing source, DELETE of already-deleted file, file paths with spaces, deeply nested directory creation + - **Validation failure chains** (6 scenarios): Multiple simultaneous argument validation failures (missing + wrong type), unknown + missing arguments combined, empty plan description rejection, ACTION phase with processing state, invalid namespace characters, invalid name characters + - **Rollback edge cases** (7 scenarios): Partial apply failure (first change persists on disk, second unchanged), errored plan cannot restart, fail preserves error message, errored plan not terminal, cancel preserves phase, errored strategize rejects execute, failed strategize rejects complete - Created `features/steps/edge_case_plan_steps.py` with step definitions for all 26 scenarios - Verified no step name collisions with existing 104 feature files - All quality gates pass: lint 0 findings, typecheck 0 errors, full suite 106 features / 1639 scenarios / 7696 steps ALL PASS @@ -446,12 +462,12 @@ The following work from the previous implementation has been completed and will **2026-02-10**: Task 10C.4 Complete - Validation Test Fixtures [Brent] - Created `features/validation_test_fixtures.feature` (34 scenarios, 81 steps) covering 6 validation domains: - - **AST security validation** (`_validate_code_ast`): 12 scenarios testing import/global/nonlocal/exec/eval/compile/__import__/getattr/setattr rejection, syntax errors, and safe code acceptance - - **Lambda AST validation** (`_validate_lambda_ast`): 4 scenarios testing valid lambda, non-lambda rejection, syntax errors, function call rejection - - **Python content sanitization** (`_sanitize_python_content`): 4 scenarios testing passthrough, code fence stripping, docstring wrapping, irrecoverable syntax (null byte) - - **Project model validation**: 6 scenarios testing invalid chars, slashes, exclamation, valid names, relative path resolution, empty name - - **Change list coercion** (`_coerce_change_list`): 4 scenarios testing empty list, mixed entries, non-list, non-change entry - - **ActionArgument parsing**: 4 scenarios testing too few parts, invalid type, invalid requirement, reserved keyword name + - **AST security validation** (`_validate_code_ast`): 12 scenarios testing import/global/nonlocal/exec/eval/compile/**import**/getattr/setattr rejection, syntax errors, and safe code acceptance + - **Lambda AST validation** (`_validate_lambda_ast`): 4 scenarios testing valid lambda, non-lambda rejection, syntax errors, function call rejection + - **Python content sanitization** (`_sanitize_python_content`): 4 scenarios testing passthrough, code fence stripping, docstring wrapping, irrecoverable syntax (null byte) + - **Project model validation**: 6 scenarios testing invalid chars, slashes, exclamation, valid names, relative path resolution, empty name + - **Change list coercion** (`_coerce_change_list`): 4 scenarios testing empty list, mixed entries, non-list, non-change entry + - **ActionArgument parsing**: 4 scenarios testing too few parts, invalid type, invalid requirement, reserved keyword name - Created `features/steps/validation_test_fixture_steps.py` with complete step definitions for all 34 scenarios - Fixed step name collisions: renamed `I create a project with name` -> `I create a project fixture with name` and `the project path should be absolute` -> `the project fixture path should be absolute` to avoid conflicts with `database_integration_steps.py` and `domain_models_steps.py` - Moved inline import (`ArgumentRequirement`, `ArgumentType`) to file-level per CONTRIBUTING.md rules @@ -459,26 +475,27 @@ The following work from the previous implementation has been completed and will - All quality gates pass: lint 0 findings, typecheck 0 errors, full suite 107 features / 1673 scenarios / 7777 steps ALL PASS **2026-02-06**: CRITICAL ARCHITECTURAL DECISION - Tool-Based Resource Modification + - **REPLACED**: OutputParser/code fence parsing approach - **WITH**: Tool-based change tracking (modern approach used by Claude Code, Cursor, Aider) - **Key changes**: - - LLMs call skills/tools directly (edit_file, write_file, delete_file, etc.) - - Skills operate on sandbox state directly - - ChangeSet is built from skill invocation history, NOT by parsing LLM text output - - Added built-in resource skills (C3.6): file ops, dir ops, search, git ops - - Added MCP skill adapter (C3.7): connect to external MCP servers - - Replaced C4 "Multi-File ChangeSet Generation" with "Tool-Based Change Tracking" - - Added SkillInvocationTracker and ToolCallRouter components + - LLMs call skills/tools directly (edit_file, write_file, delete_file, etc.) + - Skills operate on sandbox state directly + - ChangeSet is built from skill invocation history, NOT by parsing LLM text output + - Added built-in resource skills (C3.6): file ops, dir ops, search, git ops + - Added MCP skill adapter (C3.7): connect to external MCP servers + - Replaced C4 "Multi-File ChangeSet Generation" with "Tool-Based Change Tracking" + - Added SkillInvocationTracker and ToolCallRouter components - **Rationale**: - - No parsing ambiguity (is this code or explanation?) - - Each operation is explicit, typed, and trackable - - Supports rollback (replay inverse of recorded changes) - - Resource-agnostic (works for files, databases, APIs, any resource type) - - Compatible with MCP standard for external tools + - No parsing ambiguity (is this code or explanation?) + - Each operation is explicit, typed, and trackable + - Supports rollback (replay inverse of recorded changes) + - Resource-agnostic (works for files, databases, APIs, any resource type) + - Compatible with MCP standard for external tools - See `docs/specification.md` sections: - - "Tool-Based Resource Modification (Modern Architecture)" - - "Unified Resource Abstraction Layer" - - "MCP Integration Architecture" + - "Tool-Based Resource Modification (Modern Architecture)" + - "Unified Resource Abstraction Layer" + - "MCP Integration Architecture" --- @@ -486,21 +503,22 @@ The following work from the previous implementation has been completed and will ### Milestone Overview -| Milestone | Target Date | Description | -|-----------|-------------|-------------| -| **M0: Foundation** | Day 0 (Current) | Existing LangGraph infrastructure preserved | -| **M1: Minimal Plan Lifecycle** | +7 days | Basic Action -> Strategize -> Execute -> Apply working for source code | -| **M2: Projects & Resources** | +10 days | Project/Resource CLI commands, local filesystem sandbox | -| **M3: Actors & Skills** | +14 days | YAML actor loading, skill execution, multi-file generation | -| **M4: Decision Tree** | +21 days | Decision recording during Strategize, basic correction | -| **M5: Multi-Project & Subplans** | +25 days | Subplan spawning, parallel execution | -| **M6: Large Project Autonomy** | +30 days | Handle 10K+ file projects, decision correction, deep subplan hierarchies (LOCAL MODE ONLY) | -| **M7: Server Connectivity** | +35+ days | Client-server communication for remote project support (server developed independently) | -| **M8: Full Feature Set** | +40 days | All spec features complete | +| Milestone | Target Date | Description | +| -------------------------------- | --------------- | ------------------------------------------------------------------------------------------ | +| **M0: Foundation** | Day 0 (Current) | Existing LangGraph infrastructure preserved | +| **M1: Minimal Plan Lifecycle** | +7 days | Basic Action -> Strategize -> Execute -> Apply working for source code | +| **M2: Projects & Resources** | +10 days | Project/Resource CLI commands, local filesystem sandbox | +| **M3: Actors & Skills** | +14 days | YAML actor loading, skill execution, multi-file generation | +| **M4: Decision Tree** | +21 days | Decision recording during Strategize, basic correction | +| **M5: Multi-Project & Subplans** | +25 days | Subplan spawning, parallel execution | +| **M6: Large Project Autonomy** | +30 days | Handle 10K+ file projects, decision correction, deep subplan hierarchies (LOCAL MODE ONLY) | +| **M7: Server Connectivity** | +35+ days | Client-server communication for remote project support (server developed independently) | +| **M8: Full Feature Set** | +40 days | All spec features complete | ### Critical Path to 7-Day MVP (Source Code Only) **WEEK 1 GOAL**: A minimally usable application that can: + 1. Create an action from CLI 2. Use the action on a source code project 3. Execute with sandbox isolation @@ -578,6 +596,7 @@ MERGE POINT 3: After Day 30 (M6 - Large Project Autonomy) ## Quick Reference for Development ### Tool Commands + ```bash # Development setup pip install -e .[dev,tests,docs] # Install with all extras @@ -600,6 +619,7 @@ nox -s docs # Build documentation ``` ### Key Files and Their Purpose + - `pyproject.toml` - All project configuration (no setup.py, no requirements.txt) - `noxfile.py` - All task automation (no Makefile, no scripts/) - `features/` - Behave unit tests (no tests/ directory) @@ -607,7 +627,8 @@ nox -s docs # Build documentation - `docs/reference/` - Discovery artifacts from Phase 0 - `docs/architecture/decisions/` - ADRs from Phase 1 -### Environment Variables (CLEVERAGENTS_* only) +### Environment Variables (CLEVERAGENTS\_\* only) + ```bash # Core configuration CLEVERAGENTS_HOME=~/.cleveragents @@ -631,19 +652,20 @@ Execute all required tests through the appropriate `nox` sessions—never call ` ### Updated Team Assignments (by Expertise) -| Developer | Strengths | Assignment Focus | Availability | -|-----------|-----------|------------------|--------------| -| **Jeff** | CTO, fastest developer, expert in everything | Critical path blockers, architecture, complex integrations, decision correction, unblocking others | PRIMARY - Available for all critical work | -| **Aditya** | Domain expert (agents/LLMs), understands hierarchical configs | Actor YAML configurations, hierarchical actor graphs, strategy/execution actors, MCP integration | HIGH - Primary on actor/skill work | -| **Rui** | Fastest developer, new to Python | Testing (Behave/Robot), simple implementations, CLI scaffolding, test fixtures | HIGH - Parallel testing track | -| **Brent** | Slow but detail-oriented | Days 1-3: Automated quality gates setup; Days 4-8: Selective review; After Day 8: Validation testing | CONTINUOUS - Independent QA track | -| **Hamza** | RDF expert, Python proficient, no LLM experience | Projects, resources, sandbox, database infrastructure, decision models, context indexing | HIGH - Infrastructure lead | -| **Luis** | Best Python architect, pedantic | Algorithms, state machines, service layer architecture, change tracking, validation pipelines | HIGH - Architecture/service layer | -| **Mike/Brian** | Sysadmins | Deployment only (minimal coding tasks) | LOW - Only deployment tasks | +| Developer | Strengths | Assignment Focus | Availability | +| -------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ----------------------------------------- | +| **Jeff** | CTO, fastest developer, expert in everything | Critical path blockers, architecture, complex integrations, decision correction, unblocking others | PRIMARY - Available for all critical work | +| **Aditya** | Domain expert (agents/LLMs), understands hierarchical configs | Actor YAML configurations, hierarchical actor graphs, strategy/execution actors, MCP integration | HIGH - Primary on actor/skill work | +| **Rui** | Fastest developer, new to Python | Testing (Behave/Robot), simple implementations, CLI scaffolding, test fixtures | HIGH - Parallel testing track | +| **Brent** | Slow but detail-oriented | Days 1-3: Automated quality gates setup; Days 4-8: Selective review; After Day 8: Validation testing | CONTINUOUS - Independent QA track | +| **Hamza** | RDF expert, Python proficient, no LLM experience | Projects, resources, sandbox, database infrastructure, decision models, context indexing | HIGH - Infrastructure lead | +| **Luis** | Best Python architect, pedantic | Algorithms, state machines, service layer architecture, change tracking, validation pipelines | HIGH - Architecture/service layer | +| **Mike/Brian** | Sysadmins | Deployment only (minimal coding tasks) | LOW - Only deployment tasks | ### Work Assignment Philosophy **Jeff** should be assigned to: + - Any task that is blocking other developers - Complex integrations requiring deep architectural understanding - Decision correction mechanism (critical for 30-day goal) @@ -653,31 +675,33 @@ Execute all required tests through the appropriate `nox` sessions—never call ` ### Jeff's Critical Path Task Summary (Day-by-Day) -| Day | Task IDs | Description | Blocks | -|-----|----------|-------------|--------| -| **Day 1** | A5.1, A5.2, A5.5 | Plan/Action DB schema + Plan Repository | Luis (A5.3-A5.4), All downstream | -| **Day 2** | A5.7, A5.8 | Service integration + DI wiring | CLI integration (A4 tests) | -| **Day 3** | C3.1, C3.2 | Skill Protocol + Metadata | All skill implementations | -| **Day 4** | C3.3 | SkillContext (read/write/spawn) | Aditya (C3.4), Luis (C4) | -| **Day 5** | C3.6a-b | WriteFileSkill, EditFileSkill | Change tracking tests | -| **Day 6** | C3.6f-g | Skill error handling + registry | Actor compilation | -| **Day 7** | C7 | Plan-Actor integration | MVP verification | -| **Day 8** | M1.1-M1.10 | MVP merge point coordination | Release v0.1.0-rc1 | -| **Day 15-16** | D4.1 | Correction Service core algorithm | Decision correction | -| **Day 17** | D4.2 | Sandbox checkpointing | Re-execution | -| **Day 18** | D4.3 | Re-execution from correction point | M4 milestone | -| **Day 19** | D4 integration | Unblock E3 parallelism | Luis (E3) | -| **Day 20-21** | E3 | Parallel execution fine-tuning | Subplan merging | -| **Day 22-25** | Deep subplan hierarchies | 5+ level subplan testing | M6 | -| **Day 26-28** | Performance optimization | 10K+ file codebase support | Large project autonomy | -| **Day 30** | M6.1-M6.10 | Large project merge point | Release v0.3.0 | +| Day | Task IDs | Description | Blocks | +| ------------- | ------------------------ | --------------------------------------- | -------------------------------- | +| **Day 1** | A5.1, A5.2, A5.5 | Plan/Action DB schema + Plan Repository | Luis (A5.3-A5.4), All downstream | +| **Day 2** | A5.7, A5.8 | Service integration + DI wiring | CLI integration (A4 tests) | +| **Day 3** | C3.1, C3.2 | Skill Protocol + Metadata | All skill implementations | +| **Day 4** | C3.3 | SkillContext (read/write/spawn) | Aditya (C3.4), Luis (C4) | +| **Day 5** | C3.6a-b | WriteFileSkill, EditFileSkill | Change tracking tests | +| **Day 6** | C3.6f-g | Skill error handling + registry | Actor compilation | +| **Day 7** | C7 | Plan-Actor integration | MVP verification | +| **Day 8** | M1.1-M1.10 | MVP merge point coordination | Release v0.1.0-rc1 | +| **Day 15-16** | D4.1 | Correction Service core algorithm | Decision correction | +| **Day 17** | D4.2 | Sandbox checkpointing | Re-execution | +| **Day 18** | D4.3 | Re-execution from correction point | M4 milestone | +| **Day 19** | D4 integration | Unblock E3 parallelism | Luis (E3) | +| **Day 20-21** | E3 | Parallel execution fine-tuning | Subplan merging | +| **Day 22-25** | Deep subplan hierarchies | 5+ level subplan testing | M6 | +| **Day 26-28** | Performance optimization | 10K+ file codebase support | Large project autonomy | +| **Day 30** | M6.1-M6.10 | Large project merge point | Release v0.3.0 | **BLOCKING CHAIN**: If Jeff is unavailable, the following chains stall: + - A5.1 → A5.5 → A5.7 → Plan persistence (Day 1-2) - C3.1 → C3.3 → C3.6 → Skill execution (Day 3-6) - D4.1 → D4.2 → D4.3 → Decision correction (Day 15-18) **Aditya** should be assigned to: + - ALL actor configuration YAML files and examples - Hierarchical actor graph compositions - Strategy and execution actor templates @@ -686,6 +710,7 @@ Execute all required tests through the appropriate `nox` sessions—never call ` - Anything requiring understanding of LLM tool calling patterns **Luis** should be assigned to: + - State machine implementations (plan lifecycle, phase transitions) - Repository pattern implementations - Service layer architecture @@ -694,6 +719,7 @@ Execute all required tests through the appropriate `nox` sessions—never call ` - Should NOT be assigned to simple CRUD or UI work **Hamza** should be assigned to: + - Database schema design and Alembic migrations - Resource model and sandbox infrastructure - Context indexing and RDF graph store integration @@ -702,6 +728,7 @@ Execute all required tests through the appropriate `nox` sessions—never call ` - Should NOT be assigned to LLM/agent-specific logic **Rui** should be assigned to: + - ALL Behave test scenarios (write BEFORE implementation) - Robot integration tests - Simple CLI scaffolding @@ -709,6 +736,7 @@ Execute all required tests through the appropriate `nox` sessions—never call ` - Should ALWAYS work in parallel with feature developers **Brent** should be assigned to: + - Days 1-3: Setting up comprehensive automated quality gates - Days 4-8: Selective manual review of high-priority items only - After Day 8: High-impact validation and edge case testing with Luis @@ -717,15 +745,15 @@ Execute all required tests through the appropriate `nox` sessions—never call ` ### Updated Timeline Targets -| Milestone | Day | Goal | Success Criteria | -|-----------|-----|------|------------------| -| **M1: MVP** | Day 7 | Create action → Use on project → Execute with sandbox → Apply changes (source code only) | `agents [--data-dir PATH] [--config-path PATH] action create` + `agents [--data-dir PATH] [--config-path PATH] plan use` + `agents [--data-dir PATH] [--config-path PATH] plan execute` + `agents [--data-dir PATH] [--config-path PATH] plan apply` working end-to-end on a git repository with sandboxed execution | -| **M2: Projects** | Day 10 | Project/Resource CLI working with git worktree sandbox | `agents [--data-dir PATH] [--config-path PATH] project create` + `agents [--data-dir PATH] [--config-path PATH] project add-resource` + git worktree isolation verified | -| **M3: Actors** | Day 14 | Full plan lifecycle with actors, skills, multi-file generation | Actor YAML parsed → LangGraph compiled → Skills executed → Multi-file ChangeSet produced → Validation passing → Applied | -| **M4: Decisions** | Day 21 | Decision recording, tree viewing, correction mechanism | `agents [--data-dir PATH] [--config-path PATH] plan tree` shows decisions → `agents [--data-dir PATH] [--config-path PATH] plan explain` works → `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=revert` re-executes from correction point | -| **M5: Subplans** | Day 25 | Hierarchical subplans with parallel execution and merging | Parent plan spawns 5+ subplans → Parallel execution → Three-way merge → Validation passes | -| **M6: Large Projects** | Day 30 | Handle 10,000+ file projects, autonomous language porting | Can port a 500-file Python module to TypeScript using hierarchical decomposition with decision correction | -| **Server Connectivity** | Beyond Day 30 | Client interfaces for server communication; server developed independently | Client-to-server API abstractions defined, but local execution only; no server implementation in this project | +| Milestone | Day | Goal | Success Criteria | +| ----------------------- | ------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **M1: MVP** | Day 7 | Create action → Use on project → Execute with sandbox → Apply changes (source code only) | `agents [--data-dir PATH] [--config-path PATH] action create` + `agents [--data-dir PATH] [--config-path PATH] plan use` + `agents [--data-dir PATH] [--config-path PATH] plan execute` + `agents [--data-dir PATH] [--config-path PATH] plan apply` working end-to-end on a git repository with sandboxed execution | +| **M2: Projects** | Day 10 | Project/Resource CLI working with git worktree sandbox | `agents [--data-dir PATH] [--config-path PATH] project create` + `agents [--data-dir PATH] [--config-path PATH] project add-resource` + git worktree isolation verified | +| **M3: Actors** | Day 14 | Full plan lifecycle with actors, skills, multi-file generation | Actor YAML parsed → LangGraph compiled → Skills executed → Multi-file ChangeSet produced → Validation passing → Applied | +| **M4: Decisions** | Day 21 | Decision recording, tree viewing, correction mechanism | `agents [--data-dir PATH] [--config-path PATH] plan tree` shows decisions → `agents [--data-dir PATH] [--config-path PATH] plan explain` works → `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=revert` re-executes from correction point | +| **M5: Subplans** | Day 25 | Hierarchical subplans with parallel execution and merging | Parent plan spawns 5+ subplans → Parallel execution → Three-way merge → Validation passes | +| **M6: Large Projects** | Day 30 | Handle 10,000+ file projects, autonomous language porting | Can port a 500-file Python module to TypeScript using hierarchical decomposition with decision correction | +| **Server Connectivity** | Beyond Day 30 | Client interfaces for server communication; server developed independently | Client-to-server API abstractions defined, but local execution only; no server implementation in this project | ### CRITICAL: Server Connectivity Policy @@ -737,6 +765,7 @@ Execute all required tests through the appropriate `nox` sessions—never call ` 4. **Focus on Core Functionality**: All effort goes to plan lifecycle, actors, skills, sandboxing, decisions, and subplans The following Section 8 (Server Connectivity) items are explicitly **OUT OF SCOPE** for the 30-day deadline: + - Client-to-server API integration - WebSocket streaming to server - Remote project execution via server @@ -750,36 +779,43 @@ The following Section 8 (Server Connectivity) items are explicitly **OUT OF SCOP The following chains represent sequential dependencies where each item MUST complete before the next can start: **CHAIN 1: Data Layer (Days 1-3)** + ``` A5.1 (Plan DB Schema) → A5.3 (Plan SQLAlchemy Model) → A5.5 (Plan Repository) → A5.7 (Update Service) ``` **CHAIN 2: Resource Layer (Days 2-4)** + ``` B1.1 (Project Model) → B1.2 (Resource Model) → B2.1 (Project CLI) → B5.1 (Project Persistence) ``` **CHAIN 3: Sandbox Layer (Days 3-5)** + ``` B3.1 (Sandbox Protocol) → B3.3 (Git Worktree) → B3.6 (Sandbox Factory) → B3.7 (Sandbox Manager) ``` **CHAIN 4: Actor Layer (Days 4-7)** + ``` C1.1 (Actor Schema) → C2.1 (Actor Parser) → C2.2 (Actor Compiler) → C2.4 (Reference Resolution) ``` **CHAIN 5: Skill Layer (Days 5-8)** + ``` C3.1 (Skill Protocol) → C3.2 (Skill Metadata) → C3.4 (Inline Executor) → C3.6 (Built-in Skills) ``` **CHAIN 6: Change Tracking (Days 6-9)** + ``` C4.1 (Change Model) → C4.2 (Invocation Tracker) → C4.3 (Tool Router) → C4.5 (Diff Generator) ``` **CHAIN 7: Plan-Actor Integration (Days 8-14)** + ``` C7.1 (Execute Strategize) → C7.3 (Execute Execution) → C7.4 (Apply Plan) → C7.7 (Diff Review) ``` @@ -1003,12 +1039,14 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target - **Workstream T**: Testing [Rui] - CONTINUOUS **Merge Points**: + - **Day 7 (M1)**: All workstreams coordinate for MVP verification - **Day 14 (M3)**: Full plan lifecycle integration - **Day 21 (M4)**: Decision tree and correction mechanism - **Day 30 (M6)**: Large project autonomy target **MERGE POINT DAY 14 (M3) - Full Plan Lifecycle Integration**: + ``` ├── [Jeff - 9:00 AM] M3.1: Verify full plan lifecycle end-to-end │ └── Test: action create → plan use → plan execute → plan apply @@ -1028,6 +1066,7 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target ``` **MERGE POINT DAY 21 (M4) - Decision Tree & Correction**: + ``` ├── [Hamza - 9:00 AM] M4.1: Verify decision recording captures context │ └── Test: Execute strategy phase, verify decisions recorded with snapshots @@ -1054,166 +1093,168 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target **CRITICAL**: This MUST be completed before other workstreams begin to ensure all code meets quality standards from the start. -- [X] **Stage Q0: Pre-commit Hooks Setup** (Day 1) **[Brent - CRITICAL]** - COMPLETED 2026-02-09 - - [X] Code: Create automated pre-commit quality checks - - [X] **Q0.1** [Brent] Install and configure pre-commit framework: - - [X] **Q0.1a** Add `pre-commit>=3.6.0` to `pyproject.toml` dev dependencies - - [X] **Q0.1b** Create `.pre-commit-config.yaml` in project root - - [X] **Q0.1c** Add branch protection hook to prevent commits to main - - [X] Commit: "feat(qa): add pre-commit framework" - - [X] **Q0.2** [Brent] Configure Ruff for formatting and linting: - - [X] **Q0.2a** Add Ruff formatting hook with auto-fix - - [X] **Q0.2b** Add Ruff linting hook with auto-fix for safe fixes - - [X] **Q0.2c** Test with intentionally bad code - - [X] Commit: "feat(qa): add ruff formatting and linting hooks" - - [X] **Q0.3** [Brent] Add pyright type checking: - - [X] **Q0.3a** Configure pyright hook for changed Python files - - [X] **Q0.3b** Set to run serially (slow but thorough) - - [X] **Q0.3c** Test with code containing type errors - - [X] Commit: "feat(qa): add pyright type checking hook" - - [X] **Q0.4** [Brent] Add security scanning: - - [X] **Q0.4a** Add `bandit[toml]>=1.7.5` to dev dependencies - - [X] **Q0.4b** Configure bandit in `pyproject.toml` - - [X] **Q0.4c** Add bandit pre-commit hook - - [X] **Q0.4d** Add semgrep with custom rules for eval/exec detection - - [X] Commit: "feat(qa): add security scanning hooks" - - [X] **Q0.5** [Brent] Add code quality checks: - - [X] **Q0.5a** Add `vulture>=2.10` for dead code detection - - [X] **Q0.5b** Configure vulture whitelist for false positives - - [X] **Q0.5c** Add commit message linting for conventional commits - - [X] Commit: "feat(qa): add code quality hooks" - - [X] **Q0.6** [Brent] Create developer setup automation: - - [X] **Q0.6a** Create `scripts/setup-dev.sh` to install pre-commit - - [X] **Q0.6b** Update README.md with setup instructions - COMPLETED 2026-02-09 - - [X] **Q0.6c** Test on fresh checkout - - [X] Commit: "feat(qa): add developer setup script" - - [X] Tests: Verify all hooks work correctly - - [X] **Q0.7** [Brent] Verified all pre-commit hooks individually: - - [X] Test formatting fixes (ruff-format auto-fixed 37 files) - - [X] Test linting catches issues (ruff lint found pre-existing issues in features/) - - [X] Test type checking blocks bad types (pyright passes on src/) - - [X] Test security scanning catches eval() (bandit found 5 issues including exec/eval in stream_router.py) - - [X] Test vulture detects dead code (passed with whitelist) - - [X] Test branch protection prevents commits to main (passed) - - [X] Commit: "test(qa): add pre-commit hook tests" — No additional test files needed; verification is covered by `nox -s pre_commit` (runs `pre-commit run --all-files`), the CI `security`/`quality`/`coverage` jobs, and the nightly quality workflow. - - [X] **Q0.8** [Brent] Fix pre-existing bandit security findings (spawned task) - COMPLETED 2026-02-09: - - [X] `stream_router.py` B102 exec(): Added `_validate_code_ast()` AST pre-validation rejecting imports, dangerous calls (exec/eval/compile/__import__/getattr/setattr), global/nonlocal + `# nosec B102` - - [X] `stream_router.py` B307 eval(): Added `_validate_lambda_ast()` restricting to lambda-only expressions, compile+eval with `# nosec B307` - - [X] `yaml_template_engine.py` B701: `Environment` -> `SandboxedEnvironment` from `jinja2.sandbox` - - [X] `stream_router.py` B701: `Environment()` -> `SandboxedEnvironment()` from `jinja2.sandbox` - - [X] `settings.py` B104: Added `# nosec B104` (intentional bind-all, configurable via env var) - - [X] `context_service.py` B101: `assert service is not None` -> `if service is None: return []` - - [X] `memory_service.py` B101 (x2): `assert isinstance(value, Sequence)` -> `raise TypeError` - - [X] `retry_patterns.py` B101 (x2): `assert result is not None` -> `raise RuntimeError` - - [X] `context.py` (CLI) B101: `assert name is not None` -> `raise typer.BadParameter` - - [X] `plan_service.py` B105: `# nosec B105` (false positive on `"token_count": 0`) - - [X] `stream_router.py` B110 (x2): `try/except/pass` -> `contextlib.suppress(Exception)` - - [X] `migration_runner.py` B110: Added `logging.debug()` before pass (auto-approve fallback) - - [X] `nodes.py` B112: Added `self.logger.warning()` before continue (retry loop) +- [x] **Stage Q0: Pre-commit Hooks Setup** (Day 1) **[Brent - CRITICAL]** - COMPLETED 2026-02-09 + - [x] Code: Create automated pre-commit quality checks + - [x] **Q0.1** [Brent] Install and configure pre-commit framework: + - [x] **Q0.1a** Add `pre-commit>=3.6.0` to `pyproject.toml` dev dependencies + - [x] **Q0.1b** Create `.pre-commit-config.yaml` in project root + - [x] **Q0.1c** Add branch protection hook to prevent commits to main + - [x] Commit: "feat(qa): add pre-commit framework" + - [x] **Q0.2** [Brent] Configure Ruff for formatting and linting: + - [x] **Q0.2a** Add Ruff formatting hook with auto-fix + - [x] **Q0.2b** Add Ruff linting hook with auto-fix for safe fixes + - [x] **Q0.2c** Test with intentionally bad code + - [x] Commit: "feat(qa): add ruff formatting and linting hooks" + - [x] **Q0.3** [Brent] Add pyright type checking: + - [x] **Q0.3a** Configure pyright hook for changed Python files + - [x] **Q0.3b** Set to run serially (slow but thorough) + - [x] **Q0.3c** Test with code containing type errors + - [x] Commit: "feat(qa): add pyright type checking hook" + - [x] **Q0.4** [Brent] Add security scanning: + - [x] **Q0.4a** Add `bandit[toml]>=1.7.5` to dev dependencies + - [x] **Q0.4b** Configure bandit in `pyproject.toml` + - [x] **Q0.4c** Add bandit pre-commit hook + - [x] **Q0.4d** Add semgrep with custom rules for eval/exec detection + - [x] Commit: "feat(qa): add security scanning hooks" + - [x] **Q0.5** [Brent] Add code quality checks: + - [x] **Q0.5a** Add `vulture>=2.10` for dead code detection + - [x] **Q0.5b** Configure vulture whitelist for false positives + - [x] **Q0.5c** Add commit message linting for conventional commits + - [x] Commit: "feat(qa): add code quality hooks" + - [x] **Q0.6** [Brent] Create developer setup automation: + - [x] **Q0.6a** Create `scripts/setup-dev.sh` to install pre-commit + - [x] **Q0.6b** Update README.md with setup instructions - COMPLETED 2026-02-09 + - [x] **Q0.6c** Test on fresh checkout + - [x] Commit: "feat(qa): add developer setup script" + - [x] Tests: Verify all hooks work correctly + - [x] **Q0.7** [Brent] Verified all pre-commit hooks individually: + - [x] Test formatting fixes (ruff-format auto-fixed 37 files) + - [x] Test linting catches issues (ruff lint found pre-existing issues in features/) + - [x] Test type checking blocks bad types (pyright passes on src/) + - [x] Test security scanning catches eval() (bandit found 5 issues including exec/eval in stream_router.py) + - [x] Test vulture detects dead code (passed with whitelist) + - [x] Test branch protection prevents commits to main (passed) + - [x] Commit: "test(qa): add pre-commit hook tests" — No additional test files needed; verification is covered by `nox -s pre_commit` (runs `pre-commit run --all-files`), the CI `security`/`quality`/`coverage` jobs, and the nightly quality workflow. + - [x] **Q0.8** [Brent] Fix pre-existing bandit security findings (spawned task) - COMPLETED 2026-02-09: + - [x] `stream_router.py` B102 exec(): Added `_validate_code_ast()` AST pre-validation rejecting imports, dangerous calls (exec/eval/compile/**import**/getattr/setattr), global/nonlocal + `# nosec B102` + - [x] `stream_router.py` B307 eval(): Added `_validate_lambda_ast()` restricting to lambda-only expressions, compile+eval with `# nosec B307` + - [x] `yaml_template_engine.py` B701: `Environment` -> `SandboxedEnvironment` from `jinja2.sandbox` + - [x] `stream_router.py` B701: `Environment()` -> `SandboxedEnvironment()` from `jinja2.sandbox` + - [x] `settings.py` B104: Added `# nosec B104` (intentional bind-all, configurable via env var) + - [x] `context_service.py` B101: `assert service is not None` -> `if service is None: return []` + - [x] `memory_service.py` B101 (x2): `assert isinstance(value, Sequence)` -> `raise TypeError` + - [x] `retry_patterns.py` B101 (x2): `assert result is not None` -> `raise RuntimeError` + - [x] `context.py` (CLI) B101: `assert name is not None` -> `raise typer.BadParameter` + - [x] `plan_service.py` B105: `# nosec B105` (false positive on `"token_count": 0`) + - [x] `stream_router.py` B110 (x2): `try/except/pass` -> `contextlib.suppress(Exception)` + - [x] `migration_runner.py` B110: Added `logging.debug()` before pass (auto-approve fallback) + - [x] `nodes.py` B112: Added `self.logger.warning()` before continue (retry loop) - Result: **0 bandit findings** (was 16: 2 HIGH, 3 MEDIUM, 11 LOW). 4 nosec suppressions. - - [X] **Q0.9** [Brent] Fix pre-existing ruff lint issues in features/ - COMPLETED 2026-02-09: - - [X] Config: Added per-file-ignores in `pyproject.toml` for `features/steps/*.py` (F811, E501) and `features/mocks/*.py`, `features/environment.py` (E501) - - F811 (65 findings): Behave pattern — `step_impl` is intentionally redefined per step - - E501 (103 findings): Long behave step decorator strings that cannot be reasonably split - - [X] Auto-fixed I001 (1 unsorted import) - - [X] Fixed 31 code quality findings manually: - - 11x SIM115: `tempfile.NamedTemporaryFile` refactored to use `with` context manager - - 4x UP028: `for/yield` loops replaced with `yield from` - - 3x SIM117: Nested `with` statements combined into single `with` using parenthesized syntax - - 3x RUF005: List concatenation replaced with `[*a, b]` unpacking - - 2x B904: Added `from exc`/`from e` to `raise` inside `except` clauses - - 2x RUF012: Mutable class attributes annotated with `ClassVar` - - 2x SIM105: `try/except/pass` replaced with `contextlib.suppress(Exception)` - - 1x B007: Unused loop variable renamed to `_node_name` - - 1x B018: Added `# noqa: B018` for intentional import-verification expression - - 1x F821: Added missing `from typing import Any` import - - 1x SIM102: Collapsed nested `if`/`elif` into single condition - - Result: **0 ruff findings** (was 200). All behave tests pass (155 scenarios across affected files). + - [x] **Q0.9** [Brent] Fix pre-existing ruff lint issues in features/ - COMPLETED 2026-02-09: + - [x] Config: Added per-file-ignores in `pyproject.toml` for `features/steps/*.py` (F811, E501) and `features/mocks/*.py`, `features/environment.py` (E501) + - F811 (65 findings): Behave pattern — `step_impl` is intentionally redefined per step + - E501 (103 findings): Long behave step decorator strings that cannot be reasonably split + - [x] Auto-fixed I001 (1 unsorted import) + - [x] Fixed 31 code quality findings manually: + - 11x SIM115: `tempfile.NamedTemporaryFile` refactored to use `with` context manager + - 4x UP028: `for/yield` loops replaced with `yield from` + - 3x SIM117: Nested `with` statements combined into single `with` using parenthesized syntax + - 3x RUF005: List concatenation replaced with `[*a, b]` unpacking + - 2x B904: Added `from exc`/`from e` to `raise` inside `except` clauses + - 2x RUF012: Mutable class attributes annotated with `ClassVar` + - 2x SIM105: `try/except/pass` replaced with `contextlib.suppress(Exception)` + - 1x B007: Unused loop variable renamed to `_node_name` + - 1x B018: Added `# noqa: B018` for intentional import-verification expression + - 1x F821: Added missing `from typing import Any` import + - 1x SIM102: Collapsed nested `if`/`elif` into single condition + - Result: **0 ruff findings** (was 200). All behave tests pass (155 scenarios across affected files). -- [X] **Stage Q1: CI/CD Pipeline** (Day 2) **[Brent]** - COMPLETED 2026-02-09 - - [X] Code: Extend Forgejo Actions workflow for comprehensive PR validation - - NOTE: CI platform is Forgejo (`.forgejo/workflows/ci.yml`), NOT GitHub. Existing CI already has lint, typecheck, behave, build, docker, helm jobs. - - [X] **Q1.1** [Brent] Extend `.forgejo/workflows/ci.yml` with security and quality jobs: - - [X] **Q1.1a** Added `security` job: bandit scan + vulture dead code detection + artifact upload - - [X] **Q1.1b** Added `quality` job: radon complexity check (fail on grade F) + artifact upload - - [X] **Q1.1c** Added `coverage` job: behave tests with coverage, fail-under=85%, artifact upload - - [X] **Q1.1d** Updated `docker` and `helm` jobs to depend on `security` job - - [X] Commit: "feat(ci): add security, quality, and coverage jobs to Forgejo CI" - - [X] **Q1.2** [Brent] Add security and quality checks to CI: - - [X] **Q1.2a** Bandit security scanning job with JSON report output - - [X] **Q1.2b** Vulture dead code detection in security job - - [X] **Q1.2c** Radon complexity check in quality job (grade F fails build) - - [X] Commit: "feat(ci): add security and quality checks" - - [X] **Q1.3** [Brent] Add test execution with coverage: - - [X] **Q1.3a** Added `coverage` CI job that runs behave with coverage measurement - - [X] **Q1.3b** Coverage report fails if below 85% (`coverage report --fail-under=85`) - - [X] **Q1.3c** Coverage XML artifact uploaded for downstream consumption - - [X] **Q1.3d** Upload test artifacts via actions/upload-artifact - - [X] Commit: "feat(ci): add test execution with coverage" - - [X] **Q1.4** [Brent] Add quality gates: - - [X] **Q1.4a** Created `scripts/check-quality-gates.py` - aggregates all quality metrics - - [X] **Q1.4b** Fail if coverage <85% (via coverage report in CI) - - [X] **Q1.4c** Fail if any type errors (pyright in existing typecheck job) - - [X] **Q1.4d** Fail if any high-severity security issues (bandit in security job) - - [X] **Q1.4e** Quality gate script generates summary report - - [X] Commit: "feat(ci): add quality gate enforcement" - - [X] **Q1.5** [Brent] Document branch protection rules - COMPLETED 2026-02-10: - - [X] **Q1.5a** Require PR validation to pass - - [X] **Q1.5b** Require 1 review (selective by Brent) - - [X] **Q1.5c** Document in `docs/development/ci-cd.md` - - [X] Commit: "docs(ci): add branch protection guide" - - [ ] Tests: Verify CI pipeline works (Q1.5 docs complete; Q1.6 pending Rui) - - [ ] **Q1.6** [Rui] Test CI pipeline with sample PRs: - - [ ] PR with perfect code (should pass) - - [ ] PR with type errors (should fail with annotations) - - [ ] PR with low coverage (should fail with comment) - - [ ] PR with security issues (should fail) +- [ ] **Stage Q1: CI/CD Pipeline** (Day 2) **[Brent]** - COMPLETED 2026-02-09 + - [ ] Code: Extend Forgejo Actions workflow for comprehensive PR validation + - NOTE: CI platform is Forgejo (`.forgejo/workflows/ci.yml`), NOT GitHub. Existing CI already has lint, typecheck, behave, build, docker, helm jobs. + - [x] **Q1.1** [Brent] Extend `.forgejo/workflows/ci.yml` with security and quality jobs: + - [x] **Q1.1a** Added `security` job: bandit scan + vulture dead code detection + artifact upload + - [x] **Q1.1b** Added `quality` job: radon complexity check (fail on grade F) + artifact upload + - [x] **Q1.1c** Added `coverage` job: behave tests with coverage, fail-under=85%, artifact upload + - [x] **Q1.1d** Updated `docker` and `helm` jobs to depend on `security` job + - [x] Commit: "feat(ci): add security, quality, and coverage jobs to Forgejo CI" + - [x] **Q1.2** [Brent] Add security and quality checks to CI: + - [x] **Q1.2a** Bandit security scanning job with JSON report output + - [x] **Q1.2b** Vulture dead code detection in security job + - [x] **Q1.2c** Radon complexity check in quality job (grade F fails build) + - [x] Commit: "feat(ci): add security and quality checks" + - [x] **Q1.3** [Brent] Add test execution with coverage: + - [x] **Q1.3a** Added `coverage` CI job that runs behave with coverage measurement + - [x] **Q1.3b** Coverage report fails if below 85% (`coverage report --fail-under=85`) + - [x] **Q1.3c** Coverage XML artifact uploaded for downstream consumption + - [x] **Q1.3d** Upload test artifacts via actions/upload-artifact + - [x] Commit: "feat(ci): add test execution with coverage" + - [x] **Q1.4** [Brent] Add quality gates: + - [x] **Q1.4a** Created `scripts/check-quality-gates.py` - aggregates all quality metrics + - [x] **Q1.4b** Fail if coverage <85% (via coverage report in CI) + - [x] **Q1.4c** Fail if any type errors (pyright in existing typecheck job) + - [x] **Q1.4d** Fail if any high-severity security issues (bandit in security job) + - [x] **Q1.4e** Quality gate script generates summary report + - [x] Commit: "feat(ci): add quality gate enforcement" + - [x] **Q1.5** [Brent] Document branch protection rules - COMPLETED 2026-02-10: + - [x] **Q1.5a** Require PR validation to pass + - [x] **Q1.5b** Require 1 review (selective by Brent) + - [x] **Q1.5c** Document in `docs/development/ci-cd.md` + - [x] Commit: "docs(ci): add branch protection guide" + - [ ] Tests: Verify CI pipeline works (Q1.5 docs complete; Q1.6 pending Rui) + - [ ] **Q1.6** [Rui] Test CI pipeline with sample PRs: + - [ ] PR with perfect code (should pass) + - [ ] PR with type errors (should fail with annotations) + - [ ] PR with low coverage (should fail with comment) + - [ ] PR with security issues (should fail) -- [X] **Stage Q2: Advanced Automation** (Day 3) **[Brent]** - COMPLETED 2026-02-09 - - [X] Code: Set up advanced quality monitoring - - [X] **Q2.1** [Brent] Create nightly quality workflow: - - [X] **Q2.1a** Schedule for midnight UTC (`.forgejo/workflows/nightly-quality.yml`) - - [X] **Q2.1b** Run full test suite including coverage, security, complexity - - [X] **Q2.1c** Generate quality trend reports (JSON artifacts with 90-day retention) - - [X] Commit: "feat(ci): add nightly quality checks" - - [X] **Q2.2** [Brent] Add complexity monitoring: - - [X] **Q2.2a** Install `radon>=6.0.1` for complexity metrics (added to pyproject.toml dev deps) - - [X] **Q2.2b** Added `nox -s complexity` session for radon analysis - - [X] **Q2.2c** CI fails if any function has grade F complexity (31+) - - [X] Commit: "feat(qa): add complexity monitoring" - - [X] **Q2.3** [Brent] Create quality dashboard: - - [X] **Q2.3a** `scripts/check-quality-gates.py` aggregates all quality metrics - - [X] **Q2.3b** Coverage tracked via nightly workflow artifacts - - [X] **Q2.3c** Type coverage tracked via pyright in nightly workflow - - [X] **Q2.3d** Nightly workflow generates quality-trend.json with timestamp + metrics - - [X] Commit: "feat(qa): add quality dashboard" - - [X] **Q2.4** [Brent] Add ADR compliance checking: - - [X] **Q2.4a** Created `scripts/check-adr-compliance.py` with AST-based analysis - - [X] **Q2.4b** Check async usage per ADR-002 (no threading in application layer) - - [X] **Q2.4c** Check DI usage per ADR-003 (services have injected dependencies) - - [X] **Q2.4d** Check repository pattern per ADR-007 (no direct SQLAlchemy in services) - - [X] **Q2.4e** Added `nox -s adr_compliance` session - - [X] Commit: "feat(qa): add ADR compliance checks" - - [X] **Q2.5** [Brent] Create PR template: - - [X] **Q2.5a** Added `.forgejo/pull_request_template.md` (Forgejo, not GitHub) - - [X] **Q2.5b** Include quality checklist (type check, lint, coverage, security, dead code) - - [X] **Q2.5c** Require testing description and test commands - - [X] Commit: "feat(qa): add PR template" - - [X] Documentation: Quality automation guide - - [X] **Q2.6** [Brent] Document quality automation: - - [X] **Q2.6a** Created `docs/development/quality-automation.md` - - [X] **Q2.6b** Documented all hooks, CI jobs, nox sessions, and tools - - [X] **Q2.6c** Added troubleshooting guide for common issues - - [X] **Q2.6d** Included quick start section for developer onboarding - - [X] Commit: "docs(qa): comprehensive quality guide" +- [x] **Stage Q2: Advanced Automation** (Day 3) **[Brent]** - COMPLETED 2026-02-09 + - [x] Code: Set up advanced quality monitoring + - [x] **Q2.1** [Brent] Create nightly quality workflow: + - [x] **Q2.1a** Schedule for midnight UTC (`.forgejo/workflows/nightly-quality.yml`) + - [x] **Q2.1b** Run full test suite including coverage, security, complexity + - [x] **Q2.1c** Generate quality trend reports (JSON artifacts with 90-day retention) + - [x] Commit: "feat(ci): add nightly quality checks" + - [x] **Q2.2** [Brent] Add complexity monitoring: + - [x] **Q2.2a** Install `radon>=6.0.1` for complexity metrics (added to pyproject.toml dev deps) + - [x] **Q2.2b** Added `nox -s complexity` session for radon analysis + - [x] **Q2.2c** CI fails if any function has grade F complexity (31+) + - [x] Commit: "feat(qa): add complexity monitoring" + - [x] **Q2.3** [Brent] Create quality dashboard: + - [x] **Q2.3a** `scripts/check-quality-gates.py` aggregates all quality metrics + - [x] **Q2.3b** Coverage tracked via nightly workflow artifacts + - [x] **Q2.3c** Type coverage tracked via pyright in nightly workflow + - [x] **Q2.3d** Nightly workflow generates quality-trend.json with timestamp + metrics + - [x] Commit: "feat(qa): add quality dashboard" + - [x] **Q2.4** [Brent] Add ADR compliance checking: + - [x] **Q2.4a** Created `scripts/check-adr-compliance.py` with AST-based analysis + - [x] **Q2.4b** Check async usage per ADR-002 (no threading in application layer) + - [x] **Q2.4c** Check DI usage per ADR-003 (services have injected dependencies) + - [x] **Q2.4d** Check repository pattern per ADR-007 (no direct SQLAlchemy in services) + - [x] **Q2.4e** Added `nox -s adr_compliance` session + - [x] Commit: "feat(qa): add ADR compliance checks" + - [x] **Q2.5** [Brent] Create PR template: + - [x] **Q2.5a** Added `.forgejo/pull_request_template.md` (Forgejo, not GitHub) + - [x] **Q2.5b** Include quality checklist (type check, lint, coverage, security, dead code) + - [x] **Q2.5c** Require testing description and test commands + - [x] Commit: "feat(qa): add PR template" + - [x] Documentation: Quality automation guide + - [x] **Q2.6** [Brent] Document quality automation: + - [x] **Q2.6a** Created `docs/development/quality-automation.md` + - [x] **Q2.6b** Documented all hooks, CI jobs, nox sessions, and tools + - [x] **Q2.6c** Added troubleshooting guide for common issues + - [x] **Q2.6d** Included quick start section for developer onboarding + - [x] Commit: "docs(qa): comprehensive quality guide" **After Day 3**: Brent transitions to selective manual review (Days 4-8) focusing only on: + - Architectural decisions and design patterns - Complex algorithms and business logic - API contracts and interfaces - Security-sensitive code paths **After Day 8**: Brent transitions to validation testing support, working with Luis on: + - Edge case identification and testing - Semantic validation implementation - Performance testing for large codebases @@ -1223,55 +1264,55 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target ### Section 1: Completed Foundation (Phases 0-1) [PRESERVED] -- [X] Phase 0: Discovery and Requirements Elaboration - - [X] Code: Implement Python discovery tooling for CLI, server, data contracts, supporting assets, environment variables, implicit behaviors, and parity matrix generation. - - [X] Document: Append findings, scripts, and open questions to **Phase 0 Notes** with `file_path:line_number` references. - - [X] Tests: Run Behave discovery scenarios and Robot smoke suites covering the generated artifacts. +- [x] Phase 0: Discovery and Requirements Elaboration + - [x] Code: Implement Python discovery tooling for CLI, server, data contracts, supporting assets, environment variables, implicit behaviors, and parity matrix generation. + - [x] Document: Append findings, scripts, and open questions to **Phase 0 Notes** with `file_path:line_number` references. + - [x] Tests: Run Behave discovery scenarios and Robot smoke suites covering the generated artifacts. -- [X] Phase 1: Architecture Definition - - [X] Code: Produce ADR stubs, module scaffolding, coding standard configurations, and packaging setup. - - [X] ADR-001: Python Package Layering and Module Boundaries - - [X] ADR-002: Asyncio Concurrency Model - - [X] ADR-003: Dependency Injection Framework - - [X] ADR-004: Pydantic for Data Validation - - [X] ADR-005: Error Handling Hierarchy - - [X] ADR-006: CLEVERAGENTS Environment Variables - - [X] ADR-007: Repository Pattern for Persistence - - [X] ADR-008: Provider Plugin Architecture - - [X] ADR-009: CLI Framework Selection - - [X] ADR-010: Logging and Observability - - [X] Document: Update **Phase 1 Notes** with ADR locations, dependency policies, and style guides. - - [X] Tests: Execute Ruff, pyright, Behave architecture scenarios, and Robot lint pipelines. +- [x] Phase 1: Architecture Definition + - [x] Code: Produce ADR stubs, module scaffolding, coding standard configurations, and packaging setup. + - [x] ADR-001: Python Package Layering and Module Boundaries + - [x] ADR-002: Asyncio Concurrency Model + - [x] ADR-003: Dependency Injection Framework + - [x] ADR-004: Pydantic for Data Validation + - [x] ADR-005: Error Handling Hierarchy + - [x] ADR-006: CLEVERAGENTS Environment Variables + - [x] ADR-007: Repository Pattern for Persistence + - [x] ADR-008: Provider Plugin Architecture + - [x] ADR-009: CLI Framework Selection + - [x] ADR-010: Logging and Observability + - [x] Document: Update **Phase 1 Notes** with ADR locations, dependency policies, and style guides. + - [x] Tests: Execute Ruff, pyright, Behave architecture scenarios, and Robot lint pipelines. --- ### Section 2: Preserved Work from Previous Implementation [PRESERVED] -- [X] LangChain/LangGraph Foundation - - [X] Dependencies installed (langchain, langgraph, langsmith, etc.) - - [X] ADR-011: LangChain/LangGraph Integration Patterns - - [X] Base StateGraph classes (BaseAgent, BaseStateGraph) - - [X] LangChain mock provider (FakeListLLM) +- [x] LangChain/LangGraph Foundation + - [x] Dependencies installed (langchain, langgraph, langsmith, etc.) + - [x] ADR-011: LangChain/LangGraph Integration Patterns + - [x] Base StateGraph classes (BaseAgent, BaseStateGraph) + - [x] LangChain mock provider (FakeListLLM) -- [X] Core LangGraph Workflows - - [X] PlanGenerationGraph (load_context -> analyze_requirements -> generate_plan -> validate) - - [X] ContextAnalysisAgent (5-node workflow for context analysis) - - [X] AutoDebugGraph (analyze_error -> generate_fix -> validate_fix -> apply_fix) - - [X] Memory service with EntityMemory - - [X] CLI streaming integration +- [x] Core LangGraph Workflows + - [x] PlanGenerationGraph (load_context -> analyze_requirements -> generate_plan -> validate) + - [x] ContextAnalysisAgent (5-node workflow for context analysis) + - [x] AutoDebugGraph (analyze_error -> generate_fix -> validate_fix -> apply_fix) + - [x] Memory service with EntityMemory + - [x] CLI streaming integration -- [X] Provider Integration - - [X] Provider registry - - [X] OpenAI, Anthropic, Google, OpenRouter adapters - - [X] LangSmith observability +- [x] Provider Integration + - [x] Provider registry + - [x] OpenAI, Anthropic, Google, OpenRouter adapters + - [x] LangSmith observability -- [X] Actor System (Stage 7.5) - - [X] Actor domain model with config hashing - - [X] Actor persistence (database, repository) - - [X] Actor registry (built-ins from provider registry) - - [X] Actor CLI commands (add, update, remove, list, show) - - [X] Actor-first plan/chat commands (--actor flag) - - [X] v2 format compatibility for actor configs +- [x] Actor System (Stage 7.5) + - [x] Actor domain model with config hashing + - [x] Actor persistence (database, repository) + - [x] Actor registry (built-ins from provider registry) + - [x] Actor CLI commands (add, update, remove, list, show) + - [x] Actor-first plan/chat commands (--actor flag) + - [x] v2 format compatibility for actor configs --- @@ -1281,597 +1322,597 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target **WEEK 1 - CRITICAL PATH** -- [X] **Stage A1: Plan Data Model** (Day 1) - COMPLETED 2026-02-05 - - [X] Code: Create Plan domain model - - [X] Define `Plan` Pydantic model with fields (plan_id ULID, parent_plan_id, root_plan_id, attempt counter, phase, state, timestamps) - - [X] Define `PlanPhase` enum (Action, Strategize, Execute, Apply, Applied) - - [X] Define `PlanState` enum per phase (available, draft, archived, queued, processing, errored, complete, cancelled) - - [X] Add namespace support to plan naming (`[server:][namespace/]`) - - [X] Location: `src/cleveragents/domain/models/core/plan.py` - - [X] Tests: Behave scenarios for plan model validation, phase/state transitions (30 scenarios in `features/plan_model.feature`) +- [x] **Stage A1: Plan Data Model** (Day 1) - COMPLETED 2026-02-05 + - [x] Code: Create Plan domain model + - [x] Define `Plan` Pydantic model with fields (plan_id ULID, parent_plan_id, root_plan_id, attempt counter, phase, state, timestamps) + - [x] Define `PlanPhase` enum (Action, Strategize, Execute, Apply, Applied) + - [x] Define `PlanState` enum per phase (available, draft, archived, queued, processing, errored, complete, cancelled) + - [x] Add namespace support to plan naming (`[server:][namespace/]`) + - [x] Location: `src/cleveragents/domain/models/core/plan.py` + - [x] Tests: Behave scenarios for plan model validation, phase/state transitions (30 scenarios in `features/plan_model.feature`) -- [X] **Stage A2: Action Model** (Day 1) - COMPLETED 2026-02-05 - - [X] Code: Create Action domain model - - [X] Define `Action` Pydantic model (name, description, definition_of_done, strategy_actor, execution_actor, inputs_schema, reusable, read_only) - - [X] Add argument parsing for action parameters (`--arg name:type:required|optional:description`) - - [X] Location: `src/cleveragents/domain/models/core/action.py` - - [X] Tests: Behave scenarios for action model validation (22 scenarios in `features/action_model.feature`) - - [ ] **A2.1** [Luis] Extend Action model with additional fields (follow-up): - - [ ] Field `estimation_actor: str | None` - optional actor for cost/risk estimation - - [ ] Field `review_actor: str | None` - optional actor for code review - - [ ] Field `safety_profile: SafetyProfile | None` - safety constraints (DEFERRED to post-30; see Stage POST1) - - [ ] **A2.2** [Luis] Define `SafetyProfile` model (DEFERRED to post-30; see Stage POST1): - - [ ] Field `allowed_skill_categories: list[str] | None` - whitelist of skill types - - [ ] Field `require_checkpoints: bool` - require checkpointable skills - - [ ] Field `require_sandbox: bool` - require sandbox for all resources - - [ ] Field `require_human_approval: bool` - require approval at Apply - - [ ] Field `max_cost_usd: float | None` - budget cap - - [ ] Field `max_retries: int` - maximum retry attempts - - [ ] **A2.3** [Rui] Write tests for extended action model (DEFERRED to post-30; see Stage POST1): - - [ ] Scenario: Action with estimation_actor validates correctly - - [ ] Scenario: Safety profile enforced during execution +- [ ] **Stage A2: Action Model** (Day 1) - COMPLETED 2026-02-05 + - [x] Code: Create Action domain model + - [x] Define `Action` Pydantic model (name, description, definition_of_done, strategy_actor, execution_actor, inputs_schema, reusable, read_only) + - [x] Add argument parsing for action parameters (`--arg name:type:required|optional:description`) + - [x] Location: `src/cleveragents/domain/models/core/action.py` + - [x] Tests: Behave scenarios for action model validation (22 scenarios in `features/action_model.feature`) + - [ ] **A2.1** [Luis] Extend Action model with additional fields (follow-up): + - [ ] Field `estimation_actor: str | None` - optional actor for cost/risk estimation + - [ ] Field `review_actor: str | None` - optional actor for code review + - [ ] Field `safety_profile: SafetyProfile | None` - safety constraints (DEFERRED to post-30; see Stage POST1) + - [ ] **A2.2** [Luis] Define `SafetyProfile` model (DEFERRED to post-30; see Stage POST1): + - [ ] Field `allowed_skill_categories: list[str] | None` - whitelist of skill types + - [ ] Field `require_checkpoints: bool` - require checkpointable skills + - [ ] Field `require_sandbox: bool` - require sandbox for all resources + - [ ] Field `require_human_approval: bool` - require approval at Apply + - [ ] Field `max_cost_usd: float | None` - budget cap + - [ ] Field `max_retries: int` - maximum retry attempts + - [ ] **A2.3** [Rui] Write tests for extended action model (DEFERRED to post-30; see Stage POST1): + - [ ] Scenario: Action with estimation_actor validates correctly + - [ ] Scenario: Safety profile enforced during execution -- [X] **Stage A3: Plan State Machine** (Day 1-2) - COMPLETED 2026-02-05 - - [X] Code: Implement plan lifecycle state machine - - [X] Create `PlanLifecycleService` with phase transition methods - - [X] Implement `create_action()` - creates plan in Action phase - - [X] Implement `use_action(action, projects, args)` - transitions to Strategize - - [X] Implement `execute_plan()` - transitions to Execute - - [X] Implement `apply_plan()` - transitions to Applied - - [X] Add validation for phase transitions (only valid transitions allowed) - - [X] Location: `src/cleveragents/application/services/plan_lifecycle_service.py` - - [X] Tests: Behave scenarios for all phase transitions, invalid transition errors (29 scenarios in `features/plan_lifecycle_service.feature`) +- [x] **Stage A3: Plan State Machine** (Day 1-2) - COMPLETED 2026-02-05 + - [x] Code: Implement plan lifecycle state machine + - [x] Create `PlanLifecycleService` with phase transition methods + - [x] Implement `create_action()` - creates plan in Action phase + - [x] Implement `use_action(action, projects, args)` - transitions to Strategize + - [x] Implement `execute_plan()` - transitions to Execute + - [x] Implement `apply_plan()` - transitions to Applied + - [x] Add validation for phase transitions (only valid transitions allowed) + - [x] Location: `src/cleveragents/application/services/plan_lifecycle_service.py` + - [x] Tests: Behave scenarios for all phase transitions, invalid transition errors (29 scenarios in `features/plan_lifecycle_service.feature`) -- [X] **Stage A4: Plan CLI Commands** (Day 2-3) - IN PROGRESS 2026-02-05 - - [X] Code: Implement plan lifecycle CLI - - [X] `agents [--data-dir PATH] [--config-path PATH] action create --name --strategy-actor --execution-actor --definition-of-done "" [--arg ...]` - - [X] `agents [--data-dir PATH] [--config-path PATH] action list` - list available actions - - [X] `agents [--data-dir PATH] [--config-path PATH] action show ` - show action details - - [X] `agents [--data-dir PATH] [--config-path PATH] action available ` - make action available - - [X] `agents [--data-dir PATH] [--config-path PATH] action archive ` - archive action - - [X] `agents [--data-dir PATH] [--config-path PATH] plan use --project [--arg name=value ...]` - create plan from action - - [X] `agents [--data-dir PATH] [--config-path PATH] plan execute [plan_id]` - execute current or specified plan - - [X] `agents [--data-dir PATH] [--config-path PATH] plan apply [plan_id]` - apply executed plan (v3 lifecycle) - - [X] `agents [--data-dir PATH] [--config-path PATH] plan status [plan_id]` - show plan phase/state - - [X] `agents [--data-dir PATH] [--config-path PATH] plan list` - list plans with phases/states - - [X] `agents [--data-dir PATH] [--config-path PATH] plan cancel ` - cancel non-terminal plan - - [X] Location: `src/cleveragents/cli/commands/action.py`, `src/cleveragents/cli/commands/plan.py` - - [X] Tests: Behave tests for action CLI (15 scenarios in `features/action_cli.feature`) - - [ ] Tests: Behave tests for plan lifecycle CLI commands (pending) - - **[Rui]** Write 20 Behave scenarios in `features/plan_lifecycle_cli.feature` covering: - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with valid action and project - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with missing project error - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with invalid action error - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with argument validation - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on strategize-complete plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on non-strategize plan (error case) - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on execute-complete plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on non-execute plan (error case) - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan status` output format verification - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by phase - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by state - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on active plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on already-applied plan (error case) - - [ ] Tests: Robot integration tests for CLI commands (pending) - - **[Rui]** Write Robot test suite `robot/plan_lifecycle_cli.robot` for end-to-end CLI testing +- [ ] **Stage A4: Plan CLI Commands** (Day 2-3) - IN PROGRESS 2026-02-05 + - [x] Code: Implement plan lifecycle CLI + - [x] `agents [--data-dir PATH] [--config-path PATH] action create --name --strategy-actor --execution-actor --definition-of-done "" [--arg ...]` + - [x] `agents [--data-dir PATH] [--config-path PATH] action list` - list available actions + - [x] `agents [--data-dir PATH] [--config-path PATH] action show ` - show action details + - [x] `agents [--data-dir PATH] [--config-path PATH] action available ` - make action available + - [x] `agents [--data-dir PATH] [--config-path PATH] action archive ` - archive action + - [x] `agents [--data-dir PATH] [--config-path PATH] plan use --project [--arg name=value ...]` - create plan from action + - [x] `agents [--data-dir PATH] [--config-path PATH] plan execute [plan_id]` - execute current or specified plan + - [x] `agents [--data-dir PATH] [--config-path PATH] plan apply [plan_id]` - apply executed plan (v3 lifecycle) + - [x] `agents [--data-dir PATH] [--config-path PATH] plan status [plan_id]` - show plan phase/state + - [x] `agents [--data-dir PATH] [--config-path PATH] plan list` - list plans with phases/states + - [x] `agents [--data-dir PATH] [--config-path PATH] plan cancel ` - cancel non-terminal plan + - [x] Location: `src/cleveragents/cli/commands/action.py`, `src/cleveragents/cli/commands/plan.py` + - [x] Tests: Behave tests for action CLI (15 scenarios in `features/action_cli.feature`) + - [ ] Tests: Behave tests for plan lifecycle CLI commands (pending) + - **[Rui]** Write 20 Behave scenarios in `features/plan_lifecycle_cli.feature` covering: + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with valid action and project + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with missing project error + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with invalid action error + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with argument validation + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on strategize-complete plan + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on non-strategize plan (error case) + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on execute-complete plan + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on non-execute plan (error case) + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan status` output format verification + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by phase + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by state + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on active plan + - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on already-applied plan (error case) + - [ ] Tests: Robot integration tests for CLI commands (pending) + - **[Rui]** Write Robot test suite `robot/plan_lifecycle_cli.robot` for end-to-end CLI testing - [ ] **Stage A5: Plan Persistence** (Day 1-2) **[Jeff + Luis - Critical Path]** - **PARALLEL SUBTRACK A5.alpha [Jeff - Day 1 AM]**: Database Schema (A5.1, A5.2) - **PARALLEL SUBTRACK A5.beta [Luis - Day 1 AM]**: SQLAlchemy Models (A5.3, A5.4) - can start with schema design doc - **SEQUENTIAL AFTER alpha+beta [Jeff - Day 1 PM]**: Repository Implementation (A5.5, A5.6) - **SEQUENTIAL AFTER repos [Jeff - Day 2 AM]**: Service Integration (A5.7, A5.8) - **PARALLEL CONTINUOUS [Rui - Day 1-2]**: Test Writing (A5.9, A5.10, A5.11) - - - [ ] Code: Plan database schema and repository - - [ ] **A5.1** [Jeff] Create Alembic migration for `lifecycle_plans` table in `alembic/versions/xxx_add_lifecycle_plans.py`: - - [ ] **A5.1a** [Jeff] Create migration file with `revision` and `down_revision` links: - - [ ] Run `alembic revision -m "add_lifecycle_plans_table"` to generate file - - [ ] Verify revision ID is unique - - [ ] Set `down_revision` to point to previous migration (likely actions table) - - [ ] Commit: "feat(db): add lifecycle_plans migration scaffold" - - [ ] **A5.1b** [Jeff] Define `lifecycle_plans` table schema in upgrade() function: - - [ ] Column `plan_id` TEXT PRIMARY KEY (ULID format, validated at application layer) - - [ ] Column `parent_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - for subplan hierarchy - - [ ] Column `root_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - always points to topmost plan - - [ ] Column `action_id` TEXT NOT NULL FK references actions(action_id) - the action template used - - [ ] Column `phase` TEXT NOT NULL CHECK(phase IN ('ACTION','STRATEGIZE','EXECUTE','APPLY','APPLIED')) - lifecycle phase - - [ ] Column `state` TEXT NOT NULL - processing state within phase (available/draft/archived for ACTION; queued/processing/errored/complete/cancelled for others) - - [ ] Column `attempt` INTEGER NOT NULL DEFAULT 1 - increments on re-execution after correction - - [ ] Column `automation_level` TEXT NOT NULL DEFAULT 'manual' CHECK(automation_level IN ('manual','review_before_apply','full_automation')) - - [ ] Column `project_ids` TEXT NOT NULL - JSON array of project ULIDs this plan operates on - - [ ] Column `arguments` TEXT NULLABLE - JSON object mapping argument name to provided value - - [ ] Column `strategy_context` TEXT NULLABLE - JSON blob storing Strategize phase outputs (strategy, execution blueprint, resource queries) - - [ ] Column `execution_log` TEXT NULLABLE - JSON array of execution events [{timestamp, event_type, details}] - - [ ] Column `changeset_id` TEXT NULLABLE FK references changesets(changeset_id) - link to generated changes - - [ ] Column `sandbox_refs` TEXT NULLABLE - JSON object mapping resource_id to sandbox_path - - [ ] Column `error_message` TEXT NULLABLE - last error message if state is errored - - [ ] Column `created_at` TEXT NOT NULL - ISO8601 timestamp of plan creation - - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 timestamp of last modification - - [ ] Column `completed_at` TEXT NULLABLE - ISO8601 timestamp when plan reached terminal state - - [ ] Column `created_by` TEXT NULLABLE - user/session identifier who created the plan - - [ ] Commit: "feat(db): define lifecycle_plans table columns" - - [ ] **A5.1c** [Jeff] Create indices for common queries: - - [ ] Index `ix_lifecycle_plans_phase` on `phase` - for phase-based filtering - - [ ] Index `ix_lifecycle_plans_state` on `state` - for state-based filtering - - [ ] Index `ix_lifecycle_plans_parent` on `parent_plan_id` - for subplan lookups - - [ ] Index `ix_lifecycle_plans_root` on `root_plan_id` - for full tree queries - - [ ] Index `ix_lifecycle_plans_created` on `created_at` - for recent plans - - [ ] Index `ix_lifecycle_plans_action` on `action_id` - for action usage lookups - - [ ] Index `ix_lifecycle_plans_project` on `project_ids` - for project-based queries (use json_extract if needed) - - [ ] Commit: "feat(db): add lifecycle_plans indices" - - [ ] **A5.1d** [Jeff] Define foreign key ON DELETE behaviors: - - [ ] parent_plan_id: ON DELETE SET NULL - orphan subplans if parent deleted (preserve for debugging) - - [ ] root_plan_id: ON DELETE SET NULL - same reasoning - - [ ] action_id: ON DELETE RESTRICT - cannot delete action if plans exist using it - - [ ] changeset_id: ON DELETE SET NULL - preserve plan record even if changeset cleaned up - - [ ] Commit: "feat(db): define lifecycle_plans FK constraints" - - [ ] **A5.1e** [Jeff] Write `downgrade()` function to drop table: - - [ ] Drop all indices first - - [ ] Drop the lifecycle_plans table - - [ ] Verify downgrade works with `alembic downgrade -1` - - [ ] Commit: "feat(db): add lifecycle_plans downgrade function" - - [ ] **A5.2** [Jeff] Create Alembic migration for `actions` table in `alembic/versions/xxx_add_actions.py`: - - [ ] **A5.2a** [Jeff] Create migration file: - - [ ] Run `alembic revision -m "add_actions_table"` - - [ ] This migration MUST run BEFORE lifecycle_plans (set down_revision appropriately) - - [ ] Commit: "feat(db): add actions migration scaffold" - - [ ] **A5.2b** [Jeff] Define `actions` table schema: - - [ ] Column `action_id` TEXT PRIMARY KEY - ULID format - - [ ] Column `name` TEXT NOT NULL - full namespaced name (e.g., "local/code-coverage", "myorg/deploy-action") - - [ ] Column `namespace` TEXT NOT NULL - extracted namespace portion for filtering (e.g., "local", "myorg") - - [ ] Column `short_name` TEXT NOT NULL - extracted name portion after namespace (e.g., "code-coverage") - - [ ] Column `description` TEXT NULLABLE - human-readable description - - [ ] Column `definition_of_done` TEXT NOT NULL - explicit testable completion criteria (must/should/may format) - - [ ] Column `strategy_actor` TEXT NOT NULL - namespaced actor reference for Strategize phase (e.g., "local/coverage-strategist") - - [ ] Column `execution_actor` TEXT NOT NULL - namespaced actor reference for Execute phase - - [ ] Column `estimation_actor` TEXT NULLABLE - optional actor for cost/risk estimation (runs after Strategize) - - [ ] Column `review_actor` TEXT NULLABLE - optional actor for code review - - [ ] Column `inputs_schema` TEXT NOT NULL DEFAULT '[]' - JSON array of ActionArgument definitions - - [ ] Column `state` TEXT NOT NULL DEFAULT 'draft' CHECK(state IN ('available','draft','archived')) - - [ ] Column `reusable` BOOLEAN NOT NULL DEFAULT TRUE - if false, action self-deletes after first use - - [ ] Column `read_only` BOOLEAN NOT NULL DEFAULT FALSE - if true, only read-only skills allowed - - [ ] Column `safety_profile` TEXT NULLABLE - JSON object for SafetyProfile constraints (DEFERRED to post-30; see POST1) - - [ ] Column `created_at` TEXT NOT NULL - ISO8601 creation timestamp - - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 last modification timestamp - - [ ] Commit: "feat(db): define actions table columns" - - [ ] **A5.2c** [Jeff] Create indices: - - [ ] UNIQUE index on `name` - enforce unique namespaced names - - [ ] Index `ix_actions_namespace` on `namespace` - for namespace filtering - - [ ] Index `ix_actions_state` on `state` - for state filtering - - [ ] Index `ix_actions_short_name` on `short_name` - for partial name searches - - [ ] Commit: "feat(db): add actions indices" - - [ ] **A5.2d** [Jeff] Write `downgrade()` function: - - [ ] Drop indices and table - - [ ] Verify with `alembic downgrade -1` - - [ ] Commit: "feat(db): add actions downgrade function" - - [ ] **A5.3** [Luis] Create `LifecyclePlanModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **A5.3a** [Luis] Define class structure: - - [ ] Create class `LifecyclePlanModel(Base)` with `__tablename__ = 'lifecycle_plans'` - - [ ] Import necessary SQLAlchemy types: `Column, String, Integer, Boolean, Text, ForeignKey, DateTime` - - [ ] Import relationship types: `relationship, backref` - - [ ] Commit: "feat(models): add LifecyclePlanModel class scaffold" - - [ ] **A5.3b** [Luis] Define all columns matching migration schema: - - [ ] `plan_id = Column(String(26), primary_key=True)` - ULID is 26 chars - - [ ] `parent_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` - - [ ] `root_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` - - [ ] `action_id = Column(String(26), ForeignKey('actions.action_id', ondelete='RESTRICT'), nullable=False)` - - [ ] `phase = Column(String(20), nullable=False)` - enum handled at domain layer - - [ ] `state = Column(String(20), nullable=False)` - - [ ] `attempt = Column(Integer, nullable=False, default=1)` - - [ ] `automation_level = Column(String(30), nullable=False, default='manual')` - - [ ] `project_ids = Column(Text, nullable=False)` - JSON string - - [ ] `arguments = Column(Text, nullable=True)` - JSON string - - [ ] `strategy_context = Column(Text, nullable=True)` - large JSON blob - - [ ] `execution_log = Column(Text, nullable=True)` - JSON array - - [ ] `changeset_id = Column(String(26), nullable=True)` - - [ ] `sandbox_refs = Column(Text, nullable=True)` - JSON object - - [ ] `error_message = Column(Text, nullable=True)` - - [ ] `created_at = Column(String(30), nullable=False)` - ISO8601 - - [ ] `updated_at = Column(String(30), nullable=False)` - - [ ] `completed_at = Column(String(30), nullable=True)` - - [ ] `created_by = Column(String(255), nullable=True)` - - [ ] Commit: "feat(models): define LifecyclePlanModel columns" - - [ ] **A5.3c** [Luis] Define relationships: - - [ ] `parent_plan = relationship('LifecyclePlanModel', remote_side=[plan_id], backref='children', foreign_keys=[parent_plan_id])` - - [ ] `action = relationship('ActionModel', backref='plans')` - - [ ] NOTE: root_plan relationship not needed as query pattern is different - - [ ] Commit: "feat(models): define LifecyclePlanModel relationships" - - [ ] **A5.3d** [Luis] Implement `to_domain() -> Plan` method: - - [ ] Import `Plan, PlanPhase, ProcessingState, AutomationLevel` from domain - - [ ] Convert `phase` string to `PlanPhase` enum: `PlanPhase[self.phase]` - - [ ] Convert `state` string to appropriate state enum based on phase - - [ ] Parse `project_ids` JSON: `json.loads(self.project_ids)` with error handling - - [ ] Parse `arguments` JSON if not None: `json.loads(self.arguments) if self.arguments else None` - - [ ] Parse `strategy_context` JSON if not None - - [ ] Parse `execution_log` JSON if not None - - [ ] Parse `sandbox_refs` JSON if not None - - [ ] Convert timestamp strings to `datetime.fromisoformat()` objects - - [ ] Construct and return `Plan(plan_id=self.plan_id, ...)` - - [ ] Add comprehensive docstring explaining the conversion - - [ ] Commit: "feat(models): implement LifecyclePlanModel.to_domain()" - - [ ] **A5.3e** [Luis] Implement classmethod `from_domain(plan: Plan) -> LifecyclePlanModel`: - - [ ] Add `@classmethod` decorator - - [ ] Convert `plan.phase.name` to string for phase column - - [ ] Convert state enum `.name` to string - - [ ] Serialize `project_ids` to JSON: `json.dumps(plan.project_ids)` - - [ ] Serialize `arguments` to JSON if not None - - [ ] Serialize `strategy_context` to JSON if not None (handle nested objects) - - [ ] Serialize `execution_log` to JSON if not None - - [ ] Serialize `sandbox_refs` to JSON if not None - - [ ] Convert datetime objects to `.isoformat()` strings - - [ ] Return constructed `LifecyclePlanModel` instance - - [ ] Commit: "feat(models): implement LifecyclePlanModel.from_domain()" - - [ ] **A5.4** [Luis] Create `ActionModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **A5.4a** [Luis] Define class structure and columns: - - [ ] Create class `ActionModel(Base)` with `__tablename__ = 'actions'` - - [ ] `action_id = Column(String(26), primary_key=True)` - - [ ] `name = Column(String(255), nullable=False, unique=True)` - - [ ] `namespace = Column(String(100), nullable=False)` - - [ ] `short_name = Column(String(150), nullable=False)` - - [ ] `description = Column(Text, nullable=True)` - - [ ] `definition_of_done = Column(Text, nullable=False)` - - [ ] `strategy_actor = Column(String(255), nullable=False)` - - [ ] `execution_actor = Column(String(255), nullable=False)` - - [ ] `estimation_actor = Column(String(255), nullable=True)` - - [ ] `review_actor = Column(String(255), nullable=True)` - - [ ] `inputs_schema = Column(Text, nullable=False, default='[]')` - - [ ] `state = Column(String(20), nullable=False, default='draft')` - - [ ] `reusable = Column(Boolean, nullable=False, default=True)` - - [ ] `read_only = Column(Boolean, nullable=False, default=False)` - - [ ] `safety_profile = Column(Text, nullable=True)` (DEFERRED to post-30; see POST1) - - [ ] `created_at = Column(String(30), nullable=False)` - - [ ] `updated_at = Column(String(30), nullable=False)` - - [ ] Commit: "feat(models): define ActionModel columns" - - [ ] **A5.4b** [Luis] Implement `to_domain() -> Action` method: - - [ ] Import `Action, ActionState, ActionArgument` from domain - - [ ] Convert `state` string to `ActionState` enum - - [ ] Parse `inputs_schema` JSON and convert to `list[ActionArgument]` - - [ ] Parse `safety_profile` JSON if present to `SafetyProfile` or None (DEFERRED to post-30; see POST1) - - [ ] Convert timestamps to datetime objects - - [ ] Construct and return `Action` instance - - [ ] Commit: "feat(models): implement ActionModel.to_domain()" - - [ ] **A5.4c** [Luis] Implement classmethod `from_domain(action: Action) -> ActionModel`: - - [ ] Extract namespace and short_name from action.name using `NamespacedName.parse()` - - [ ] Serialize `inputs_schema` to JSON from list of ActionArgument (call `.model_dump()` on each) - - [ ] Serialize `safety_profile` to JSON if present (DEFERRED to post-30; see POST1) - - [ ] Convert timestamps to ISO8601 strings - - [ ] Return constructed `ActionModel` instance - - [ ] Commit: "feat(models): implement ActionModel.from_domain()" - - [ ] **A5.5** [Jeff] Implement `LifecyclePlanRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **A5.5a** [Jeff] Define class structure: - - [ ] Create class `LifecyclePlanRepository` with proper typing - - [ ] Add `__init__(self, session_factory: Callable[[], Session])` - session factory injection - - [ ] Store `self._session_factory = session_factory` - - [ ] Add class docstring explaining repository pattern usage - - [ ] Commit: "feat(repo): add LifecyclePlanRepository scaffold" - - [ ] **A5.5b** [Jeff] Implement `create(plan: Plan) -> Plan`: - - [ ] Open session using context manager: `with self._session_factory() as session:` - - [ ] Convert domain model: `model = LifecyclePlanModel.from_domain(plan)` - - [ ] Add to session: `session.add(model)` - - [ ] Commit transaction: `session.commit()` - - [ ] Refresh to get any database-generated values: `session.refresh(model)` - - [ ] Convert back and return: `return model.to_domain()` - - [ ] Wrap in try/except for `IntegrityError` - raise custom `DuplicatePlanError` if duplicate ID - - [ ] Add type hints and docstring - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.create()" - - [ ] **A5.5c** [Jeff] Implement `get_by_id(plan_id: str) -> Plan | None`: - - [ ] Query by primary key: `session.query(LifecyclePlanModel).filter_by(plan_id=plan_id).first()` - - [ ] Return `None` if not found - - [ ] Convert to domain model if found - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_id()" - - [ ] **A5.5d** [Jeff] Implement `get_by_phase(phase: PlanPhase, limit: int = 100) -> list[Plan]`: - - [ ] Filter by phase column: `.filter_by(phase=phase.name)` - - [ ] Order by created_at DESC: `.order_by(LifecyclePlanModel.created_at.desc())` - - [ ] Apply limit: `.limit(limit)` - - [ ] Convert all results to domain models using list comprehension - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_phase()" - - [ ] **A5.5e** [Jeff] Implement `get_by_state(state: ProcessingState, limit: int = 100) -> list[Plan]`: - - [ ] Similar pattern to get_by_phase - - [ ] Filter by state column - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_state()" - - [ ] **A5.5f** [Jeff] Implement `get_children(parent_plan_id: str) -> list[Plan]`: - - [ ] Filter by parent_plan_id: `.filter_by(parent_plan_id=parent_plan_id)` - - [ ] Order by created_at ASC (oldest first for processing order) - - [ ] Convert all to domain models - - [ ] Used for listing direct subplans - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_children()" - - [ ] **A5.5g** [Jeff] Implement `get_tree(root_plan_id: str) -> list[Plan]`: - - [ ] Use recursive CTE query for all descendants: - ```python - from sqlalchemy import text - cte = text(''' - WITH RECURSIVE plan_tree AS ( - SELECT * FROM lifecycle_plans WHERE plan_id = :root_id - UNION ALL - SELECT lp.* FROM lifecycle_plans lp - INNER JOIN plan_tree pt ON lp.parent_plan_id = pt.plan_id - ) - SELECT * FROM plan_tree ORDER BY created_at ASC - ''') - ``` - - [ ] Execute and map results to domain models - - [ ] Return in tree order (parent before children by creation time) - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_tree()" - - [ ] **A5.5h** [Jeff] Implement `update(plan: Plan) -> Plan`: - - [ ] Fetch existing record by plan_id - - [ ] Raise `PlanNotFoundError` if not exists - - [ ] Update all fields from domain model (use a helper to copy attributes) - - [ ] Auto-update `updated_at` timestamp to now - - [ ] Commit transaction - - [ ] Return updated plan (re-query to ensure consistency) - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.update()" - - [ ] **A5.5i** [Jeff] Implement `list_all(limit: int = 100, offset: int = 0) -> list[Plan]`: - - [ ] Query all with pagination: `.offset(offset).limit(limit)` - - [ ] Order by created_at DESC - - [ ] Convert to domain models - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.list_all()" - - [ ] **A5.5j** [Jeff] Implement `count(phase: PlanPhase | None = None, state: ProcessingState | None = None) -> int`: - - [ ] Use `session.query(func.count(LifecyclePlanModel.plan_id))` - - [ ] Apply optional phase filter - - [ ] Apply optional state filter - - [ ] Return `.scalar()` result - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.count()" - - [ ] **A5.5k** [Jeff] Add `@retry_database` decorator to all methods: - - [ ] Import from `src/cleveragents/core/retry_patterns.py` - - [ ] Configure: 3 retries, exponential backoff (1s, 2s, 4s) - - [ ] Only retry on `OperationalError` (database locked, connection timeout) - - [ ] Do NOT retry on `IntegrityError` (these are application logic errors) - - [ ] Commit: "feat(repo): add retry decorator to LifecyclePlanRepository" - - [ ] **A5.6** [Luis] Implement `ActionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **A5.6a** [Luis] Define class with session factory injection: - - [ ] Create class `ActionRepository` - - [ ] Add `__init__(self, session_factory: Callable[[], Session])` - - [ ] Commit: "feat(repo): add ActionRepository scaffold" - - [ ] **A5.6b** [Luis] Implement `create(action: Action) -> Action`: - - [ ] Same pattern as LifecyclePlanRepository - - [ ] Handle duplicate name error specifically - - [ ] Commit: "feat(repo): implement ActionRepository.create()" - - [ ] **A5.6c** [Luis] Implement `get_by_id(action_id: str) -> Action | None`: - - [ ] Query by primary key - - [ ] Convert to domain or return None - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_id()" - - [ ] **A5.6d** [Luis] Implement `get_by_name(name: str) -> Action | None`: - - [ ] Query by exact namespaced name match: `.filter_by(name=name).first()` - - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action show local/my-action` - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_name()" - - [ ] **A5.6e** [Luis] Implement `get_by_namespace(namespace: str, state: ActionState | None = None) -> list[Action]`: - - [ ] Filter by namespace column - - [ ] Optionally filter by state - - [ ] Order by short_name ASC for consistent display - - [ ] Convert all to domain models - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_namespace()" - - [ ] **A5.6f** [Luis] Implement `get_by_state(state: ActionState) -> list[Action]`: - - [ ] Filter by state column - - [ ] Order by updated_at DESC (most recently modified first) - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_state()" - - [ ] **A5.6g** [Luis] Implement `update(action: Action) -> Action`: - - [ ] Fetch by action_id - - [ ] Update all fields - - [ ] Auto-update updated_at - - [ ] Commit and return - - [ ] Commit: "feat(repo): implement ActionRepository.update()" - - [ ] **A5.6h** [Luis] Implement `list_available(namespace: str | None = None) -> list[Action]`: - - [ ] Filter by state='available' - - [ ] Optionally filter by namespace - - [ ] Order by namespace ASC, short_name ASC - - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action list` - - [ ] Commit: "feat(repo): implement ActionRepository.list_available()" - - [ ] **A5.6i** [Luis] Implement `delete(action_id: str) -> bool`: - - [ ] First check if any plans reference this action: `plan_repo.count(action_id=action_id)` - - [ ] If plans exist, raise `ActionInUseError` with count of plans - - [ ] Otherwise delete the action - - [ ] Return True if deleted - - [ ] Commit: "feat(repo): implement ActionRepository.delete()" - - [ ] **A5.6j** [Luis] Add retry decorator to all methods: - - [ ] Same pattern as LifecyclePlanRepository - - [ ] Commit: "feat(repo): add retry decorator to ActionRepository" - - [ ] **A5.7** [Jeff] Update `PlanLifecycleService` to use repositories: - - [ ] **A5.7a** [Jeff] Modify `__init__()` to accept repository dependencies: - - [ ] Change signature: `def __init__(self, plan_repository: LifecyclePlanRepository, action_repository: ActionRepository):` - - [ ] Store as instance variables: `self._plan_repo = plan_repository`, `self._action_repo = action_repository` - - [ ] REMOVE the in-memory storage: Delete `self._plans: dict` and `self._actions: dict` - - [ ] Update docstring to reflect dependency injection - - [ ] Commit: "refactor(service): update PlanLifecycleService to inject repositories" - - [ ] **A5.7b** [Jeff] Update `create_action()` to use ActionRepository: - - [ ] Replace `self._actions[action.action_id] = action` with `self._action_repo.create(action)` - - [ ] Handle `DuplicateActionError` by converting to user-friendly error message - - [ ] Return the created action from repository (may have database-modified fields) - - [ ] Commit: "refactor(service): update create_action() to use repository" - - [ ] **A5.7c** [Jeff] Update `get_action()` to use repository: - - [ ] Replace dict lookup with `self._action_repo.get_by_id()` or `get_by_name()` - - [ ] Handle both ID and namespaced name lookups - - [ ] Commit: "refactor(service): update get_action() to use repository" - - [ ] **A5.7d** [Jeff] Update `list_actions()` to use repository: - - [ ] Replace dict.values() iteration with `self._action_repo.list_available()` - - [ ] Add namespace filter parameter - - [ ] Add state filter parameter - - [ ] Commit: "refactor(service): update list_actions() to use repository" - - [ ] **A5.7e** [Jeff] Update `use_action()` to use both repositories: - - [ ] Fetch action from ActionRepository by name - - [ ] Raise `ActionNotFoundError` if not exists - - [ ] Raise `ActionNotAvailableError` if action.state != AVAILABLE - - [ ] Create new Plan domain object with ULID, set action_id reference - - [ ] Persist plan via LifecyclePlanRepository.create() - - [ ] Return the created plan - - [ ] Commit: "refactor(service): update use_action() to use repositories" - - [ ] **A5.7f** [Jeff] Update all plan state transition methods: - - [ ] `start_strategize()`: fetch plan → verify phase → update state → save - - [ ] `complete_strategize()`: fetch → verify → update phase+state → save - - [ ] `fail_strategize()`: fetch → update state to ERRORED → set error_message → save - - [ ] `start_execute()`: same pattern - - [ ] `complete_execute()`: same pattern, store changeset_id - - [ ] `fail_execute()`: same pattern - - [ ] `apply_plan()`: verify Execute phase complete → update to APPLIED → set completed_at → save - - [ ] All methods must re-fetch after save to return current state - - [ ] Commit: "refactor(service): update phase transition methods to use repository" - - [ ] **A5.7g** [Jeff] Update `cancel_plan()` to use repository: - - [ ] Fetch plan - - [ ] Verify not in terminal state (APPLIED or CANCELLED) - - [ ] Set state to CANCELLED, set completed_at - - [ ] Persist via repository - - [ ] Commit: "refactor(service): update cancel_plan() to use repository" - - [ ] **A5.7h** [Jeff] Add transaction handling for multi-step operations: - - [ ] For operations that modify multiple entities (e.g., use_action creates plan + may update action): - - [ ] Use UnitOfWork pattern: start transaction, do all operations, commit atomically - - [ ] If any step fails, rollback all changes - - [ ] Create `UnitOfWork` class if not exists: manages session lifecycle - - [ ] Commit: "feat(service): add transaction handling for multi-step operations" - - [ ] **A5.8** [Luis] Update DI container in `src/cleveragents/application/container.py`: - - [ ] **A5.8a** [Luis] Add `LifecyclePlanRepository` provider: - - [ ] Create factory function that instantiates repository with session factory - - [ ] Register with container - - [ ] Ensure proper scoping (singleton or per-request based on usage pattern) - - [ ] Commit: "feat(di): add LifecyclePlanRepository provider" - - [ ] **A5.8b** [Luis] Add `ActionRepository` provider: - - [ ] Same pattern as plan repository - - [ ] Commit: "feat(di): add ActionRepository provider" - - [ ] **A5.8c** [Luis] Update `PlanLifecycleService` provider to inject repositories: - - [ ] Modify service factory to resolve both repositories - - [ ] Pass to PlanLifecycleService constructor - - [ ] Verify dependency chain is correct - - [ ] Commit: "feat(di): update PlanLifecycleService provider with repositories" - - [ ] Tests: Integration tests for plan/action persistence - - [ ] **A5.9** [Rui] Write Behave scenarios in `features/plan_persistence.feature`: - - [ ] **A5.9a** [Rui] Scenario: Create plan stores record in database - - [ ] Given: An action "local/test-action" exists in database with state=AVAILABLE - - [ ] And: A project "local/test-project" exists - - [ ] When: I call `plan_service.use_action("local/test-action", project_ids=["proj-123"])` - - [ ] Then: A plan record exists in the lifecycle_plans table - - [ ] And: The plan_id is a valid 26-character ULID - - [ ] And: The plan.phase is STRATEGIZE - - [ ] And: The plan.state is QUEUED - - [ ] And: The plan.action_id matches the action - - [ ] Commit: "test(behave): add plan creation persistence scenario" - - [ ] **A5.9b** [Rui] Scenario: Update plan phase persists correctly - - [ ] Given: A plan exists in database with phase=STRATEGIZE, state=QUEUED - - [ ] When: I call `plan_service.complete_strategize(plan_id, strategy_context={...})` - - [ ] And: I call `plan_service.start_execute(plan_id)` - - [ ] Then: The database record shows phase='EXECUTE' - - [ ] And: The database record shows state='PROCESSING' - - [ ] And: The updated_at timestamp has changed - - [ ] Commit: "test(behave): add plan phase update persistence scenario" - - [ ] **A5.9c** [Rui] Scenario: Query plans by phase returns filtered results - - [ ] Given: 3 plans exist: 1 in STRATEGIZE, 1 in EXECUTE, 1 in APPLIED - - [ ] When: I query `plan_repo.get_by_phase(PlanPhase.STRATEGIZE)` - - [ ] Then: Only 1 plan is returned - - [ ] And: Its phase is STRATEGIZE - - [ ] Commit: "test(behave): add plan phase query scenario" - - [ ] **A5.9d** [Rui] Scenario: Query plans by state returns filtered results - - [ ] Given: 3 plans exist: 1 QUEUED, 1 PROCESSING, 1 ERRORED - - [ ] When: I query `plan_repo.get_by_state(ProcessingState.ERRORED)` - - [ ] Then: Only 1 plan is returned - - [ ] And: Its state is ERRORED - - [ ] Commit: "test(behave): add plan state query scenario" - - [ ] **A5.9e** [Rui] Scenario: Get plan tree returns parent and all children - - [ ] Given: A root plan exists with plan_id="root-123" - - [ ] And: A child plan exists with parent_plan_id="root-123" - - [ ] And: A grandchild plan exists with parent_plan_id=child_plan_id - - [ ] When: I query `plan_repo.get_tree("root-123")` - - [ ] Then: 3 plans are returned in order - - [ ] And: First plan is the root - - [ ] And: Second plan is the child - - [ ] And: Third plan is the grandchild - - [ ] Commit: "test(behave): add plan tree query scenario" - - [ ] **A5.9f** [Rui] Scenario: Concurrent plan creation is thread-safe - - [ ] Given: An action exists - - [ ] When: 10 threads simultaneously call `plan_service.use_action()` - - [ ] Then: All 10 plans are created successfully - - [ ] And: All 10 plan_ids are unique - - [ ] And: No database integrity errors occurred - - [ ] Commit: "test(behave): add concurrent plan creation scenario" - - [ ] **A5.10** [Rui] Write Behave scenarios in `features/action_persistence.feature`: - - [ ] **A5.10a** [Rui] Scenario: Create action stores record in database - - [ ] Given: No action named "local/test-action" exists - - [ ] When: I call `action_service.create_action()` with valid parameters - - [ ] Then: An action record exists in the actions table - - [ ] And: The action_id is a valid 26-character ULID - - [ ] And: The namespace column is "local" - - [ ] And: The short_name column is "test-action" - - [ ] Commit: "test(behave): add action creation persistence scenario" - - [ ] **A5.10b** [Rui] Scenario: Get action by namespaced name works - - [ ] Given: An action "local/my-action" exists in database - - [ ] When: I call `action_repo.get_by_name("local/my-action")` - - [ ] Then: The action is returned - - [ ] And: Its name matches "local/my-action" - - [ ] Commit: "test(behave): add action name lookup scenario" - - [ ] **A5.10c** [Rui] Scenario: List available excludes archived actions - - [ ] Given: 3 actions exist: 2 with state=AVAILABLE, 1 with state=ARCHIVED - - [ ] When: I call `action_repo.list_available()` - - [ ] Then: Only 2 actions are returned - - [ ] And: Neither has state=ARCHIVED - - [ ] Commit: "test(behave): add action list available scenario" - - [ ] **A5.10d** [Rui] Scenario: Update action state persists - - [ ] Given: An action exists with state=DRAFT - - [ ] When: I call `action_service.make_available(action_id)` - - [ ] Then: The database record shows state='AVAILABLE' - - [ ] Commit: "test(behave): add action state update scenario" - - [ ] **A5.10e** [Rui] Scenario: Delete action with existing plans fails - - [ ] Given: An action "local/used-action" exists - - [ ] And: A plan exists that references this action - - [ ] When: I call `action_repo.delete(action_id)` - - [ ] Then: An ActionInUseError is raised - - [ ] And: The action still exists in the database - - [ ] Commit: "test(behave): add action delete protection scenario" - - [ ] **A5.11** [Rui] Write Robot test `robot/plan_persistence_e2e.robot`: - - [ ] **A5.11a** [Rui] Test: Full lifecycle persists all transitions - - [ ] Create action via CLI: `agents [--data-dir PATH] [--config-path PATH] action create --name local/e2e-test ...` - - [ ] Make action available: `agents [--data-dir PATH] [--config-path PATH] action available ` - - [ ] Create project: `agents [--data-dir PATH] [--config-path PATH] project create --name local/e2e-project` - - [ ] Use action on project: `agents [--data-dir PATH] [--config-path PATH] plan use local/e2e-test --project local/e2e-project` - - [ ] Execute plan: `agents [--data-dir PATH] [--config-path PATH] plan execute ` - - [ ] Apply plan: `agents [--data-dir PATH] [--config-path PATH] plan apply ` - - [ ] Verify via `agents [--data-dir PATH] [--config-path PATH] plan status ` shows APPLIED phase - - [ ] Query database directly to verify all state transitions recorded - - [ ] Commit: "test(robot): add full lifecycle persistence e2e test" - - [ ] **A5.11b** [Rui] Test: Restart persistence - - [ ] Create plan via CLI - - [ ] Get plan_id from output - - [ ] Simulate process crash (kill the process or restart CLI) - - [ ] Run new CLI command: `agents [--data-dir PATH] [--config-path PATH] plan status ` - - [ ] Verify plan still exists and shows correct state - - [ ] Commit: "test(robot): add restart persistence e2e test" - - [ ] **A5.11c** [Rui] Test: Concurrent CLI access - - [ ] Start two CLI processes accessing same plan - - [ ] One process starts execute, other queries status - - [ ] Verify no data corruption or deadlocks - - [ ] Both processes complete successfully - - [ ] Commit: "test(robot): add concurrent CLI access e2e test" + **PARALLEL SUBTRACK A5.alpha [Jeff - Day 1 AM]**: Database Schema (A5.1, A5.2) + **PARALLEL SUBTRACK A5.beta [Luis - Day 1 AM]**: SQLAlchemy Models (A5.3, A5.4) - can start with schema design doc + **SEQUENTIAL AFTER alpha+beta [Jeff - Day 1 PM]**: Repository Implementation (A5.5, A5.6) + **SEQUENTIAL AFTER repos [Jeff - Day 2 AM]**: Service Integration (A5.7, A5.8) + **PARALLEL CONTINUOUS [Rui - Day 1-2]**: Test Writing (A5.9, A5.10, A5.11) + - [ ] Code: Plan database schema and repository + - [ ] **A5.1** [Jeff] Create Alembic migration for `lifecycle_plans` table in `alembic/versions/xxx_add_lifecycle_plans.py`: + - [ ] **A5.1a** [Jeff] Create migration file with `revision` and `down_revision` links: + - [ ] Run `alembic revision -m "add_lifecycle_plans_table"` to generate file + - [ ] Verify revision ID is unique + - [ ] Set `down_revision` to point to previous migration (likely actions table) + - [ ] Commit: "feat(db): add lifecycle_plans migration scaffold" + - [ ] **A5.1b** [Jeff] Define `lifecycle_plans` table schema in upgrade() function: + - [ ] Column `plan_id` TEXT PRIMARY KEY (ULID format, validated at application layer) + - [ ] Column `parent_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - for subplan hierarchy + - [ ] Column `root_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - always points to topmost plan + - [ ] Column `action_id` TEXT NOT NULL FK references actions(action_id) - the action template used + - [ ] Column `phase` TEXT NOT NULL CHECK(phase IN ('ACTION','STRATEGIZE','EXECUTE','APPLY','APPLIED')) - lifecycle phase + - [ ] Column `state` TEXT NOT NULL - processing state within phase (available/draft/archived for ACTION; queued/processing/errored/complete/cancelled for others) + - [ ] Column `attempt` INTEGER NOT NULL DEFAULT 1 - increments on re-execution after correction + - [ ] Column `automation_level` TEXT NOT NULL DEFAULT 'manual' CHECK(automation_level IN ('manual','review_before_apply','full_automation')) + - [ ] Column `project_ids` TEXT NOT NULL - JSON array of project ULIDs this plan operates on + - [ ] Column `arguments` TEXT NULLABLE - JSON object mapping argument name to provided value + - [ ] Column `strategy_context` TEXT NULLABLE - JSON blob storing Strategize phase outputs (strategy, execution blueprint, resource queries) + - [ ] Column `execution_log` TEXT NULLABLE - JSON array of execution events [{timestamp, event_type, details}] + - [ ] Column `changeset_id` TEXT NULLABLE FK references changesets(changeset_id) - link to generated changes + - [ ] Column `sandbox_refs` TEXT NULLABLE - JSON object mapping resource_id to sandbox_path + - [ ] Column `error_message` TEXT NULLABLE - last error message if state is errored + - [ ] Column `created_at` TEXT NOT NULL - ISO8601 timestamp of plan creation + - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 timestamp of last modification + - [ ] Column `completed_at` TEXT NULLABLE - ISO8601 timestamp when plan reached terminal state + - [ ] Column `created_by` TEXT NULLABLE - user/session identifier who created the plan + - [ ] Commit: "feat(db): define lifecycle_plans table columns" + - [ ] **A5.1c** [Jeff] Create indices for common queries: + - [ ] Index `ix_lifecycle_plans_phase` on `phase` - for phase-based filtering + - [ ] Index `ix_lifecycle_plans_state` on `state` - for state-based filtering + - [ ] Index `ix_lifecycle_plans_parent` on `parent_plan_id` - for subplan lookups + - [ ] Index `ix_lifecycle_plans_root` on `root_plan_id` - for full tree queries + - [ ] Index `ix_lifecycle_plans_created` on `created_at` - for recent plans + - [ ] Index `ix_lifecycle_plans_action` on `action_id` - for action usage lookups + - [ ] Index `ix_lifecycle_plans_project` on `project_ids` - for project-based queries (use json_extract if needed) + - [ ] Commit: "feat(db): add lifecycle_plans indices" + - [ ] **A5.1d** [Jeff] Define foreign key ON DELETE behaviors: + - [ ] parent_plan_id: ON DELETE SET NULL - orphan subplans if parent deleted (preserve for debugging) + - [ ] root_plan_id: ON DELETE SET NULL - same reasoning + - [ ] action_id: ON DELETE RESTRICT - cannot delete action if plans exist using it + - [ ] changeset_id: ON DELETE SET NULL - preserve plan record even if changeset cleaned up + - [ ] Commit: "feat(db): define lifecycle_plans FK constraints" + - [ ] **A5.1e** [Jeff] Write `downgrade()` function to drop table: + - [ ] Drop all indices first + - [ ] Drop the lifecycle_plans table + - [ ] Verify downgrade works with `alembic downgrade -1` + - [ ] Commit: "feat(db): add lifecycle_plans downgrade function" + - [ ] **A5.2** [Jeff] Create Alembic migration for `actions` table in `alembic/versions/xxx_add_actions.py`: + - [ ] **A5.2a** [Jeff] Create migration file: + - [ ] Run `alembic revision -m "add_actions_table"` + - [ ] This migration MUST run BEFORE lifecycle_plans (set down_revision appropriately) + - [ ] Commit: "feat(db): add actions migration scaffold" + - [ ] **A5.2b** [Jeff] Define `actions` table schema: + - [ ] Column `action_id` TEXT PRIMARY KEY - ULID format + - [ ] Column `name` TEXT NOT NULL - full namespaced name (e.g., "local/code-coverage", "myorg/deploy-action") + - [ ] Column `namespace` TEXT NOT NULL - extracted namespace portion for filtering (e.g., "local", "myorg") + - [ ] Column `short_name` TEXT NOT NULL - extracted name portion after namespace (e.g., "code-coverage") + - [ ] Column `description` TEXT NULLABLE - human-readable description + - [ ] Column `definition_of_done` TEXT NOT NULL - explicit testable completion criteria (must/should/may format) + - [ ] Column `strategy_actor` TEXT NOT NULL - namespaced actor reference for Strategize phase (e.g., "local/coverage-strategist") + - [ ] Column `execution_actor` TEXT NOT NULL - namespaced actor reference for Execute phase + - [ ] Column `estimation_actor` TEXT NULLABLE - optional actor for cost/risk estimation (runs after Strategize) + - [ ] Column `review_actor` TEXT NULLABLE - optional actor for code review + - [ ] Column `inputs_schema` TEXT NOT NULL DEFAULT '[]' - JSON array of ActionArgument definitions + - [ ] Column `state` TEXT NOT NULL DEFAULT 'draft' CHECK(state IN ('available','draft','archived')) + - [ ] Column `reusable` BOOLEAN NOT NULL DEFAULT TRUE - if false, action self-deletes after first use + - [ ] Column `read_only` BOOLEAN NOT NULL DEFAULT FALSE - if true, only read-only skills allowed + - [ ] Column `safety_profile` TEXT NULLABLE - JSON object for SafetyProfile constraints (DEFERRED to post-30; see POST1) + - [ ] Column `created_at` TEXT NOT NULL - ISO8601 creation timestamp + - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 last modification timestamp + - [ ] Commit: "feat(db): define actions table columns" + - [ ] **A5.2c** [Jeff] Create indices: + - [ ] UNIQUE index on `name` - enforce unique namespaced names + - [ ] Index `ix_actions_namespace` on `namespace` - for namespace filtering + - [ ] Index `ix_actions_state` on `state` - for state filtering + - [ ] Index `ix_actions_short_name` on `short_name` - for partial name searches + - [ ] Commit: "feat(db): add actions indices" + - [ ] **A5.2d** [Jeff] Write `downgrade()` function: + - [ ] Drop indices and table + - [ ] Verify with `alembic downgrade -1` + - [ ] Commit: "feat(db): add actions downgrade function" + - [ ] **A5.3** [Luis] Create `LifecyclePlanModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: + - [ ] **A5.3a** [Luis] Define class structure: + - [ ] Create class `LifecyclePlanModel(Base)` with `__tablename__ = 'lifecycle_plans'` + - [ ] Import necessary SQLAlchemy types: `Column, String, Integer, Boolean, Text, ForeignKey, DateTime` + - [ ] Import relationship types: `relationship, backref` + - [ ] Commit: "feat(models): add LifecyclePlanModel class scaffold" + - [ ] **A5.3b** [Luis] Define all columns matching migration schema: + - [ ] `plan_id = Column(String(26), primary_key=True)` - ULID is 26 chars + - [ ] `parent_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` + - [ ] `root_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` + - [ ] `action_id = Column(String(26), ForeignKey('actions.action_id', ondelete='RESTRICT'), nullable=False)` + - [ ] `phase = Column(String(20), nullable=False)` - enum handled at domain layer + - [ ] `state = Column(String(20), nullable=False)` + - [ ] `attempt = Column(Integer, nullable=False, default=1)` + - [ ] `automation_level = Column(String(30), nullable=False, default='manual')` + - [ ] `project_ids = Column(Text, nullable=False)` - JSON string + - [ ] `arguments = Column(Text, nullable=True)` - JSON string + - [ ] `strategy_context = Column(Text, nullable=True)` - large JSON blob + - [ ] `execution_log = Column(Text, nullable=True)` - JSON array + - [ ] `changeset_id = Column(String(26), nullable=True)` + - [ ] `sandbox_refs = Column(Text, nullable=True)` - JSON object + - [ ] `error_message = Column(Text, nullable=True)` + - [ ] `created_at = Column(String(30), nullable=False)` - ISO8601 + - [ ] `updated_at = Column(String(30), nullable=False)` + - [ ] `completed_at = Column(String(30), nullable=True)` + - [ ] `created_by = Column(String(255), nullable=True)` + - [ ] Commit: "feat(models): define LifecyclePlanModel columns" + - [ ] **A5.3c** [Luis] Define relationships: + - [ ] `parent_plan = relationship('LifecyclePlanModel', remote_side=[plan_id], backref='children', foreign_keys=[parent_plan_id])` + - [ ] `action = relationship('ActionModel', backref='plans')` + - [ ] NOTE: root_plan relationship not needed as query pattern is different + - [ ] Commit: "feat(models): define LifecyclePlanModel relationships" + - [ ] **A5.3d** [Luis] Implement `to_domain() -> Plan` method: + - [ ] Import `Plan, PlanPhase, ProcessingState, AutomationLevel` from domain + - [ ] Convert `phase` string to `PlanPhase` enum: `PlanPhase[self.phase]` + - [ ] Convert `state` string to appropriate state enum based on phase + - [ ] Parse `project_ids` JSON: `json.loads(self.project_ids)` with error handling + - [ ] Parse `arguments` JSON if not None: `json.loads(self.arguments) if self.arguments else None` + - [ ] Parse `strategy_context` JSON if not None + - [ ] Parse `execution_log` JSON if not None + - [ ] Parse `sandbox_refs` JSON if not None + - [ ] Convert timestamp strings to `datetime.fromisoformat()` objects + - [ ] Construct and return `Plan(plan_id=self.plan_id, ...)` + - [ ] Add comprehensive docstring explaining the conversion + - [ ] Commit: "feat(models): implement LifecyclePlanModel.to_domain()" + - [ ] **A5.3e** [Luis] Implement classmethod `from_domain(plan: Plan) -> LifecyclePlanModel`: + - [ ] Add `@classmethod` decorator + - [ ] Convert `plan.phase.name` to string for phase column + - [ ] Convert state enum `.name` to string + - [ ] Serialize `project_ids` to JSON: `json.dumps(plan.project_ids)` + - [ ] Serialize `arguments` to JSON if not None + - [ ] Serialize `strategy_context` to JSON if not None (handle nested objects) + - [ ] Serialize `execution_log` to JSON if not None + - [ ] Serialize `sandbox_refs` to JSON if not None + - [ ] Convert datetime objects to `.isoformat()` strings + - [ ] Return constructed `LifecyclePlanModel` instance + - [ ] Commit: "feat(models): implement LifecyclePlanModel.from_domain()" + - [ ] **A5.4** [Luis] Create `ActionModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: + - [ ] **A5.4a** [Luis] Define class structure and columns: + - [ ] Create class `ActionModel(Base)` with `__tablename__ = 'actions'` + - [ ] `action_id = Column(String(26), primary_key=True)` + - [ ] `name = Column(String(255), nullable=False, unique=True)` + - [ ] `namespace = Column(String(100), nullable=False)` + - [ ] `short_name = Column(String(150), nullable=False)` + - [ ] `description = Column(Text, nullable=True)` + - [ ] `definition_of_done = Column(Text, nullable=False)` + - [ ] `strategy_actor = Column(String(255), nullable=False)` + - [ ] `execution_actor = Column(String(255), nullable=False)` + - [ ] `estimation_actor = Column(String(255), nullable=True)` + - [ ] `review_actor = Column(String(255), nullable=True)` + - [ ] `inputs_schema = Column(Text, nullable=False, default='[]')` + - [ ] `state = Column(String(20), nullable=False, default='draft')` + - [ ] `reusable = Column(Boolean, nullable=False, default=True)` + - [ ] `read_only = Column(Boolean, nullable=False, default=False)` + - [ ] `safety_profile = Column(Text, nullable=True)` (DEFERRED to post-30; see POST1) + - [ ] `created_at = Column(String(30), nullable=False)` + - [ ] `updated_at = Column(String(30), nullable=False)` + - [ ] Commit: "feat(models): define ActionModel columns" + - [ ] **A5.4b** [Luis] Implement `to_domain() -> Action` method: + - [ ] Import `Action, ActionState, ActionArgument` from domain + - [ ] Convert `state` string to `ActionState` enum + - [ ] Parse `inputs_schema` JSON and convert to `list[ActionArgument]` + - [ ] Parse `safety_profile` JSON if present to `SafetyProfile` or None (DEFERRED to post-30; see POST1) + - [ ] Convert timestamps to datetime objects + - [ ] Construct and return `Action` instance + - [ ] Commit: "feat(models): implement ActionModel.to_domain()" + - [ ] **A5.4c** [Luis] Implement classmethod `from_domain(action: Action) -> ActionModel`: + - [ ] Extract namespace and short_name from action.name using `NamespacedName.parse()` + - [ ] Serialize `inputs_schema` to JSON from list of ActionArgument (call `.model_dump()` on each) + - [ ] Serialize `safety_profile` to JSON if present (DEFERRED to post-30; see POST1) + - [ ] Convert timestamps to ISO8601 strings + - [ ] Return constructed `ActionModel` instance + - [ ] Commit: "feat(models): implement ActionModel.from_domain()" + - [ ] **A5.5** [Jeff] Implement `LifecyclePlanRepository` in `src/cleveragents/infrastructure/database/repositories.py`: + - [ ] **A5.5a** [Jeff] Define class structure: + - [ ] Create class `LifecyclePlanRepository` with proper typing + - [ ] Add `__init__(self, session_factory: Callable[[], Session])` - session factory injection + - [ ] Store `self._session_factory = session_factory` + - [ ] Add class docstring explaining repository pattern usage + - [ ] Commit: "feat(repo): add LifecyclePlanRepository scaffold" + - [ ] **A5.5b** [Jeff] Implement `create(plan: Plan) -> Plan`: + - [ ] Open session using context manager: `with self._session_factory() as session:` + - [ ] Convert domain model: `model = LifecyclePlanModel.from_domain(plan)` + - [ ] Add to session: `session.add(model)` + - [ ] Commit transaction: `session.commit()` + - [ ] Refresh to get any database-generated values: `session.refresh(model)` + - [ ] Convert back and return: `return model.to_domain()` + - [ ] Wrap in try/except for `IntegrityError` - raise custom `DuplicatePlanError` if duplicate ID + - [ ] Add type hints and docstring + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.create()" + - [ ] **A5.5c** [Jeff] Implement `get_by_id(plan_id: str) -> Plan | None`: + - [ ] Query by primary key: `session.query(LifecyclePlanModel).filter_by(plan_id=plan_id).first()` + - [ ] Return `None` if not found + - [ ] Convert to domain model if found + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_id()" + - [ ] **A5.5d** [Jeff] Implement `get_by_phase(phase: PlanPhase, limit: int = 100) -> list[Plan]`: + - [ ] Filter by phase column: `.filter_by(phase=phase.name)` + - [ ] Order by created_at DESC: `.order_by(LifecyclePlanModel.created_at.desc())` + - [ ] Apply limit: `.limit(limit)` + - [ ] Convert all results to domain models using list comprehension + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_phase()" + - [ ] **A5.5e** [Jeff] Implement `get_by_state(state: ProcessingState, limit: int = 100) -> list[Plan]`: + - [ ] Similar pattern to get_by_phase + - [ ] Filter by state column + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_state()" + - [ ] **A5.5f** [Jeff] Implement `get_children(parent_plan_id: str) -> list[Plan]`: + - [ ] Filter by parent_plan_id: `.filter_by(parent_plan_id=parent_plan_id)` + - [ ] Order by created_at ASC (oldest first for processing order) + - [ ] Convert all to domain models + - [ ] Used for listing direct subplans + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_children()" + - [ ] **A5.5g** [Jeff] Implement `get_tree(root_plan_id: str) -> list[Plan]`: + - [ ] Use recursive CTE query for all descendants: + ```python + from sqlalchemy import text + cte = text(''' + WITH RECURSIVE plan_tree AS ( + SELECT * FROM lifecycle_plans WHERE plan_id = :root_id + UNION ALL + SELECT lp.* FROM lifecycle_plans lp + INNER JOIN plan_tree pt ON lp.parent_plan_id = pt.plan_id + ) + SELECT * FROM plan_tree ORDER BY created_at ASC + ''') + ``` + - [ ] Execute and map results to domain models + - [ ] Return in tree order (parent before children by creation time) + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_tree()" + - [ ] **A5.5h** [Jeff] Implement `update(plan: Plan) -> Plan`: + - [ ] Fetch existing record by plan_id + - [ ] Raise `PlanNotFoundError` if not exists + - [ ] Update all fields from domain model (use a helper to copy attributes) + - [ ] Auto-update `updated_at` timestamp to now + - [ ] Commit transaction + - [ ] Return updated plan (re-query to ensure consistency) + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.update()" + - [ ] **A5.5i** [Jeff] Implement `list_all(limit: int = 100, offset: int = 0) -> list[Plan]`: + - [ ] Query all with pagination: `.offset(offset).limit(limit)` + - [ ] Order by created_at DESC + - [ ] Convert to domain models + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.list_all()" + - [ ] **A5.5j** [Jeff] Implement `count(phase: PlanPhase | None = None, state: ProcessingState | None = None) -> int`: + - [ ] Use `session.query(func.count(LifecyclePlanModel.plan_id))` + - [ ] Apply optional phase filter + - [ ] Apply optional state filter + - [ ] Return `.scalar()` result + - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.count()" + - [ ] **A5.5k** [Jeff] Add `@retry_database` decorator to all methods: + - [ ] Import from `src/cleveragents/core/retry_patterns.py` + - [ ] Configure: 3 retries, exponential backoff (1s, 2s, 4s) + - [ ] Only retry on `OperationalError` (database locked, connection timeout) + - [ ] Do NOT retry on `IntegrityError` (these are application logic errors) + - [ ] Commit: "feat(repo): add retry decorator to LifecyclePlanRepository" + - [ ] **A5.6** [Luis] Implement `ActionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: + - [ ] **A5.6a** [Luis] Define class with session factory injection: + - [ ] Create class `ActionRepository` + - [ ] Add `__init__(self, session_factory: Callable[[], Session])` + - [ ] Commit: "feat(repo): add ActionRepository scaffold" + - [ ] **A5.6b** [Luis] Implement `create(action: Action) -> Action`: + - [ ] Same pattern as LifecyclePlanRepository + - [ ] Handle duplicate name error specifically + - [ ] Commit: "feat(repo): implement ActionRepository.create()" + - [ ] **A5.6c** [Luis] Implement `get_by_id(action_id: str) -> Action | None`: + - [ ] Query by primary key + - [ ] Convert to domain or return None + - [ ] Commit: "feat(repo): implement ActionRepository.get_by_id()" + - [ ] **A5.6d** [Luis] Implement `get_by_name(name: str) -> Action | None`: + - [ ] Query by exact namespaced name match: `.filter_by(name=name).first()` + - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action show local/my-action` + - [ ] Commit: "feat(repo): implement ActionRepository.get_by_name()" + - [ ] **A5.6e** [Luis] Implement `get_by_namespace(namespace: str, state: ActionState | None = None) -> list[Action]`: + - [ ] Filter by namespace column + - [ ] Optionally filter by state + - [ ] Order by short_name ASC for consistent display + - [ ] Convert all to domain models + - [ ] Commit: "feat(repo): implement ActionRepository.get_by_namespace()" + - [ ] **A5.6f** [Luis] Implement `get_by_state(state: ActionState) -> list[Action]`: + - [ ] Filter by state column + - [ ] Order by updated_at DESC (most recently modified first) + - [ ] Commit: "feat(repo): implement ActionRepository.get_by_state()" + - [ ] **A5.6g** [Luis] Implement `update(action: Action) -> Action`: + - [ ] Fetch by action_id + - [ ] Update all fields + - [ ] Auto-update updated_at + - [ ] Commit and return + - [ ] Commit: "feat(repo): implement ActionRepository.update()" + - [ ] **A5.6h** [Luis] Implement `list_available(namespace: str | None = None) -> list[Action]`: + - [ ] Filter by state='available' + - [ ] Optionally filter by namespace + - [ ] Order by namespace ASC, short_name ASC + - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action list` + - [ ] Commit: "feat(repo): implement ActionRepository.list_available()" + - [ ] **A5.6i** [Luis] Implement `delete(action_id: str) -> bool`: + - [ ] First check if any plans reference this action: `plan_repo.count(action_id=action_id)` + - [ ] If plans exist, raise `ActionInUseError` with count of plans + - [ ] Otherwise delete the action + - [ ] Return True if deleted + - [ ] Commit: "feat(repo): implement ActionRepository.delete()" + - [ ] **A5.6j** [Luis] Add retry decorator to all methods: + - [ ] Same pattern as LifecyclePlanRepository + - [ ] Commit: "feat(repo): add retry decorator to ActionRepository" + - [ ] **A5.7** [Jeff] Update `PlanLifecycleService` to use repositories: + - [ ] **A5.7a** [Jeff] Modify `__init__()` to accept repository dependencies: + - [ ] Change signature: `def __init__(self, plan_repository: LifecyclePlanRepository, action_repository: ActionRepository):` + - [ ] Store as instance variables: `self._plan_repo = plan_repository`, `self._action_repo = action_repository` + - [ ] REMOVE the in-memory storage: Delete `self._plans: dict` and `self._actions: dict` + - [ ] Update docstring to reflect dependency injection + - [ ] Commit: "refactor(service): update PlanLifecycleService to inject repositories" + - [ ] **A5.7b** [Jeff] Update `create_action()` to use ActionRepository: + - [ ] Replace `self._actions[action.action_id] = action` with `self._action_repo.create(action)` + - [ ] Handle `DuplicateActionError` by converting to user-friendly error message + - [ ] Return the created action from repository (may have database-modified fields) + - [ ] Commit: "refactor(service): update create_action() to use repository" + - [ ] **A5.7c** [Jeff] Update `get_action()` to use repository: + - [ ] Replace dict lookup with `self._action_repo.get_by_id()` or `get_by_name()` + - [ ] Handle both ID and namespaced name lookups + - [ ] Commit: "refactor(service): update get_action() to use repository" + - [ ] **A5.7d** [Jeff] Update `list_actions()` to use repository: + - [ ] Replace dict.values() iteration with `self._action_repo.list_available()` + - [ ] Add namespace filter parameter + - [ ] Add state filter parameter + - [ ] Commit: "refactor(service): update list_actions() to use repository" + - [ ] **A5.7e** [Jeff] Update `use_action()` to use both repositories: + - [ ] Fetch action from ActionRepository by name + - [ ] Raise `ActionNotFoundError` if not exists + - [ ] Raise `ActionNotAvailableError` if action.state != AVAILABLE + - [ ] Create new Plan domain object with ULID, set action_id reference + - [ ] Persist plan via LifecyclePlanRepository.create() + - [ ] Return the created plan + - [ ] Commit: "refactor(service): update use_action() to use repositories" + - [ ] **A5.7f** [Jeff] Update all plan state transition methods: + - [ ] `start_strategize()`: fetch plan → verify phase → update state → save + - [ ] `complete_strategize()`: fetch → verify → update phase+state → save + - [ ] `fail_strategize()`: fetch → update state to ERRORED → set error_message → save + - [ ] `start_execute()`: same pattern + - [ ] `complete_execute()`: same pattern, store changeset_id + - [ ] `fail_execute()`: same pattern + - [ ] `apply_plan()`: verify Execute phase complete → update to APPLIED → set completed_at → save + - [ ] All methods must re-fetch after save to return current state + - [ ] Commit: "refactor(service): update phase transition methods to use repository" + - [ ] **A5.7g** [Jeff] Update `cancel_plan()` to use repository: + - [ ] Fetch plan + - [ ] Verify not in terminal state (APPLIED or CANCELLED) + - [ ] Set state to CANCELLED, set completed_at + - [ ] Persist via repository + - [ ] Commit: "refactor(service): update cancel_plan() to use repository" + - [ ] **A5.7h** [Jeff] Add transaction handling for multi-step operations: + - [ ] For operations that modify multiple entities (e.g., use_action creates plan + may update action): + - [ ] Use UnitOfWork pattern: start transaction, do all operations, commit atomically + - [ ] If any step fails, rollback all changes + - [ ] Create `UnitOfWork` class if not exists: manages session lifecycle + - [ ] Commit: "feat(service): add transaction handling for multi-step operations" + - [ ] **A5.8** [Luis] Update DI container in `src/cleveragents/application/container.py`: + - [ ] **A5.8a** [Luis] Add `LifecyclePlanRepository` provider: + - [ ] Create factory function that instantiates repository with session factory + - [ ] Register with container + - [ ] Ensure proper scoping (singleton or per-request based on usage pattern) + - [ ] Commit: "feat(di): add LifecyclePlanRepository provider" + - [ ] **A5.8b** [Luis] Add `ActionRepository` provider: + - [ ] Same pattern as plan repository + - [ ] Commit: "feat(di): add ActionRepository provider" + - [ ] **A5.8c** [Luis] Update `PlanLifecycleService` provider to inject repositories: + - [ ] Modify service factory to resolve both repositories + - [ ] Pass to PlanLifecycleService constructor + - [ ] Verify dependency chain is correct + - [ ] Commit: "feat(di): update PlanLifecycleService provider with repositories" + - [ ] Tests: Integration tests for plan/action persistence + - [ ] **A5.9** [Rui] Write Behave scenarios in `features/plan_persistence.feature`: + - [ ] **A5.9a** [Rui] Scenario: Create plan stores record in database + - [ ] Given: An action "local/test-action" exists in database with state=AVAILABLE + - [ ] And: A project "local/test-project" exists + - [ ] When: I call `plan_service.use_action("local/test-action", project_ids=["proj-123"])` + - [ ] Then: A plan record exists in the lifecycle_plans table + - [ ] And: The plan_id is a valid 26-character ULID + - [ ] And: The plan.phase is STRATEGIZE + - [ ] And: The plan.state is QUEUED + - [ ] And: The plan.action_id matches the action + - [ ] Commit: "test(behave): add plan creation persistence scenario" + - [ ] **A5.9b** [Rui] Scenario: Update plan phase persists correctly + - [ ] Given: A plan exists in database with phase=STRATEGIZE, state=QUEUED + - [ ] When: I call `plan_service.complete_strategize(plan_id, strategy_context={...})` + - [ ] And: I call `plan_service.start_execute(plan_id)` + - [ ] Then: The database record shows phase='EXECUTE' + - [ ] And: The database record shows state='PROCESSING' + - [ ] And: The updated_at timestamp has changed + - [ ] Commit: "test(behave): add plan phase update persistence scenario" + - [ ] **A5.9c** [Rui] Scenario: Query plans by phase returns filtered results + - [ ] Given: 3 plans exist: 1 in STRATEGIZE, 1 in EXECUTE, 1 in APPLIED + - [ ] When: I query `plan_repo.get_by_phase(PlanPhase.STRATEGIZE)` + - [ ] Then: Only 1 plan is returned + - [ ] And: Its phase is STRATEGIZE + - [ ] Commit: "test(behave): add plan phase query scenario" + - [ ] **A5.9d** [Rui] Scenario: Query plans by state returns filtered results + - [ ] Given: 3 plans exist: 1 QUEUED, 1 PROCESSING, 1 ERRORED + - [ ] When: I query `plan_repo.get_by_state(ProcessingState.ERRORED)` + - [ ] Then: Only 1 plan is returned + - [ ] And: Its state is ERRORED + - [ ] Commit: "test(behave): add plan state query scenario" + - [ ] **A5.9e** [Rui] Scenario: Get plan tree returns parent and all children + - [ ] Given: A root plan exists with plan_id="root-123" + - [ ] And: A child plan exists with parent_plan_id="root-123" + - [ ] And: A grandchild plan exists with parent_plan_id=child_plan_id + - [ ] When: I query `plan_repo.get_tree("root-123")` + - [ ] Then: 3 plans are returned in order + - [ ] And: First plan is the root + - [ ] And: Second plan is the child + - [ ] And: Third plan is the grandchild + - [ ] Commit: "test(behave): add plan tree query scenario" + - [ ] **A5.9f** [Rui] Scenario: Concurrent plan creation is thread-safe + - [ ] Given: An action exists + - [ ] When: 10 threads simultaneously call `plan_service.use_action()` + - [ ] Then: All 10 plans are created successfully + - [ ] And: All 10 plan_ids are unique + - [ ] And: No database integrity errors occurred + - [ ] Commit: "test(behave): add concurrent plan creation scenario" + - [ ] **A5.10** [Rui] Write Behave scenarios in `features/action_persistence.feature`: + - [ ] **A5.10a** [Rui] Scenario: Create action stores record in database + - [ ] Given: No action named "local/test-action" exists + - [ ] When: I call `action_service.create_action()` with valid parameters + - [ ] Then: An action record exists in the actions table + - [ ] And: The action_id is a valid 26-character ULID + - [ ] And: The namespace column is "local" + - [ ] And: The short_name column is "test-action" + - [ ] Commit: "test(behave): add action creation persistence scenario" + - [ ] **A5.10b** [Rui] Scenario: Get action by namespaced name works + - [ ] Given: An action "local/my-action" exists in database + - [ ] When: I call `action_repo.get_by_name("local/my-action")` + - [ ] Then: The action is returned + - [ ] And: Its name matches "local/my-action" + - [ ] Commit: "test(behave): add action name lookup scenario" + - [ ] **A5.10c** [Rui] Scenario: List available excludes archived actions + - [ ] Given: 3 actions exist: 2 with state=AVAILABLE, 1 with state=ARCHIVED + - [ ] When: I call `action_repo.list_available()` + - [ ] Then: Only 2 actions are returned + - [ ] And: Neither has state=ARCHIVED + - [ ] Commit: "test(behave): add action list available scenario" + - [ ] **A5.10d** [Rui] Scenario: Update action state persists + - [ ] Given: An action exists with state=DRAFT + - [ ] When: I call `action_service.make_available(action_id)` + - [ ] Then: The database record shows state='AVAILABLE' + - [ ] Commit: "test(behave): add action state update scenario" + - [ ] **A5.10e** [Rui] Scenario: Delete action with existing plans fails + - [ ] Given: An action "local/used-action" exists + - [ ] And: A plan exists that references this action + - [ ] When: I call `action_repo.delete(action_id)` + - [ ] Then: An ActionInUseError is raised + - [ ] And: The action still exists in the database + - [ ] Commit: "test(behave): add action delete protection scenario" + - [ ] **A5.11** [Rui] Write Robot test `robot/plan_persistence_e2e.robot`: + - [ ] **A5.11a** [Rui] Test: Full lifecycle persists all transitions + - [ ] Create action via CLI: `agents [--data-dir PATH] [--config-path PATH] action create --name local/e2e-test ...` + - [ ] Make action available: `agents [--data-dir PATH] [--config-path PATH] action available ` + - [ ] Create project: `agents [--data-dir PATH] [--config-path PATH] project create --name local/e2e-project` + - [ ] Use action on project: `agents [--data-dir PATH] [--config-path PATH] plan use local/e2e-test --project local/e2e-project` + - [ ] Execute plan: `agents [--data-dir PATH] [--config-path PATH] plan execute ` + - [ ] Apply plan: `agents [--data-dir PATH] [--config-path PATH] plan apply ` + - [ ] Verify via `agents [--data-dir PATH] [--config-path PATH] plan status ` shows APPLIED phase + - [ ] Query database directly to verify all state transitions recorded + - [ ] Commit: "test(robot): add full lifecycle persistence e2e test" + - [ ] **A5.11b** [Rui] Test: Restart persistence + - [ ] Create plan via CLI + - [ ] Get plan_id from output + - [ ] Simulate process crash (kill the process or restart CLI) + - [ ] Run new CLI command: `agents [--data-dir PATH] [--config-path PATH] plan status ` + - [ ] Verify plan still exists and shows correct state + - [ ] Commit: "test(robot): add restart persistence e2e test" + - [ ] **A5.11c** [Rui] Test: Concurrent CLI access + - [ ] Start two CLI processes accessing same plan + - [ ] One process starts execute, other queries status + - [ ] Verify no data corruption or deadlocks + - [ ] Both processes complete successfully + - [ ] Commit: "test(robot): add concurrent CLI access e2e test" - [ ] **Stage A6: Automation Levels Foundation** (Day 4-5) **[Luis]** - - [ ] Code: Implement basic automation level support - - [ ] **A6.1** [Luis] Add `AutomationLevel` enum to `src/cleveragents/domain/models/core/plan.py`: - - [ ] Value `MANUAL` - user triggers each phase transition - - [ ] Value `REVIEW_BEFORE_APPLY` - auto strategize+execute, pause before apply - - [ ] Value `FULL_AUTOMATION` - all phases automatic - - [ ] **A6.2** [Luis] Add automation level configuration to `src/cleveragents/config/settings.py`: - - [ ] Add `default_automation_level: AutomationLevel` setting - - [ ] Add `CLEVERAGENTS_AUTOMATION_LEVEL` environment variable - - [ ] Implement hierarchy: plan-level > session-level > global-level - - [ ] **A6.3** [Luis] Update `PlanLifecycleService` to respect automation levels: - - [ ] Add `automation_level` parameter to `use_action()` method - - [ ] If automation allows, automatically call `execute_plan()` after strategize completes - - [ ] If full automation, automatically call `apply_plan()` after execute completes - - [ ] Add pause/resume capability for review-before-apply mode - - [ ] **A6.4** [Luis] Update CLI commands to support automation levels: - - [ ] Add `--automation-level` flag to `agents [--data-dir PATH] [--config-path PATH] plan use` command - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] config set automation-level ` command - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] plan set-automation-level ` command: - - [ ] Can change automation level for existing plan - - [ ] Only affects future phase transitions - - [ ] Subplans created after change use new level - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] session set automation-level ` command: - - [ ] Set session-level automation (overrides global) - - [ ] Persists for current session only - - [ ] Tests: Automation level tests - - [ ] **A6.5** [Rui] Write Behave scenarios in `features/automation_levels.feature`: - - [ ] Scenario: Manual mode requires explicit execute command - - [ ] Scenario: Review-before-apply auto-executes but pauses at apply - - [ ] Scenario: Full automation runs all phases without user input - - [ ] Scenario: Plan-level automation overrides global setting - - [ ] Scenario: Change automation level mid-plan works correctly + - [ ] Code: Implement basic automation level support + - [ ] **A6.1** [Luis] Add `AutomationLevel` enum to `src/cleveragents/domain/models/core/plan.py`: + - [ ] Value `MANUAL` - user triggers each phase transition + - [ ] Value `REVIEW_BEFORE_APPLY` - auto strategize+execute, pause before apply + - [ ] Value `FULL_AUTOMATION` - all phases automatic + - [ ] **A6.2** [Luis] Add automation level configuration to `src/cleveragents/config/settings.py`: + - [ ] Add `default_automation_level: AutomationLevel` setting + - [ ] Add `CLEVERAGENTS_AUTOMATION_LEVEL` environment variable + - [ ] Implement hierarchy: plan-level > session-level > global-level + - [ ] **A6.3** [Luis] Update `PlanLifecycleService` to respect automation levels: + - [ ] Add `automation_level` parameter to `use_action()` method + - [ ] If automation allows, automatically call `execute_plan()` after strategize completes + - [ ] If full automation, automatically call `apply_plan()` after execute completes + - [ ] Add pause/resume capability for review-before-apply mode + - [ ] **A6.4** [Luis] Update CLI commands to support automation levels: + - [ ] Add `--automation-level` flag to `agents [--data-dir PATH] [--config-path PATH] plan use` command + - [ ] Add `agents [--data-dir PATH] [--config-path PATH] config set automation-level ` command + - [ ] Add `agents [--data-dir PATH] [--config-path PATH] plan set-automation-level ` command: + - [ ] Can change automation level for existing plan + - [ ] Only affects future phase transitions + - [ ] Subplans created after change use new level + - [ ] Add `agents [--data-dir PATH] [--config-path PATH] session set automation-level ` command: + - [ ] Set session-level automation (overrides global) + - [ ] Persists for current session only + - [ ] Tests: Automation level tests + - [ ] **A6.5** [Rui] Write Behave scenarios in `features/automation_levels.feature`: + - [ ] Scenario: Manual mode requires explicit execute command + - [ ] Scenario: Review-before-apply auto-executes but pauses at apply + - [ ] Scenario: Full automation runs all phases without user input + - [ ] Scenario: Plan-level automation overrides global setting + - [ ] Scenario: Change automation level mid-plan works correctly **M1 SUCCESS CRITERIA**: + - [ ] Can create an action via CLI and it persists to database - [ ] Can use an action on a project to create a plan - [ ] Plan transitions through phases with database persistence @@ -1909,1557 +1950,1633 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation - [ ] **Stage B1: Project Data Model** (Day 1-2) **[Hamza - Python Expert, RDF Background]** - **SEQUENTIAL ORDER**: B1.3 (ResourceType) → B1.4 (SandboxStrategy) → B1.2 (Resource) → B1.5 (ValidationConfig) → B1.6 (ContextConfig) → B1.1 (Project) - The order matters because Project depends on Resource, which depends on enums. - - - [ ] Code: Create Project domain model - - [ ] **B1.3** [Hamza] Define `ResourceType` enum in `src/cleveragents/domain/models/core/resource.py`: - - [ ] **B1.3a** [Hamza] Create the file with proper imports: - - [ ] Import `Enum` from enum module - - [ ] Import `str` for string mixin: `class ResourceType(str, Enum):` - - [ ] Add module docstring explaining resource types - - [ ] Commit: "feat(domain): create resource.py with ResourceType enum scaffold" - - [ ] **B1.3b** [Hamza] Define enum values with descriptive docstrings: - - [ ] `GIT_REPOSITORY = "git_repository"` - Git repo (local path or remote URL), supports worktree sandboxing - - [ ] `FILESYSTEM = "filesystem"` - Local directory or file tree, supports copy-on-write sandboxing - - [ ] `DATABASE = "database"` - SQL/NoSQL database endpoint, supports transaction-based sandboxing - - [ ] `API_ENDPOINT = "api_endpoint"` - REST/GraphQL API, typically cannot be sandboxed - - [ ] `DOCUMENT_CORPUS = "document_corpus"` - Collection of documents (PDFs, markdown, wikis) - - [ ] `CLOUD_INFRASTRUCTURE = "cloud_infrastructure"` - Cloud resources (AWS, GCP, Azure) - - [ ] Commit: "feat(domain): define ResourceType enum values" - - [ ] **B1.4** [Hamza] Define `SandboxStrategy` enum in same file: - - [ ] **B1.4a** [Hamza] Define enum with string values: - - [ ] `GIT_WORKTREE = "git_worktree"` - Use `git worktree add` for isolation, efficient for git repos - - [ ] `COPY_ON_WRITE = "copy_on_write"` - Copy directory to temp location, universal but can be slow for large dirs - - [ ] `OVERLAY = "overlay"` - Use overlayfs (Linux only), efficient copy-on-write for large directories - - [ ] `TRANSACTION_ROLLBACK = "transaction_rollback"` - Database transaction that can be rolled back - - [ ] `VERSIONING = "versioning"` - Use versioning features (e.g., S3 versioning) - - [ ] `NONE = "none"` - No sandboxing possible, modifications are immediate and irreversible - - [ ] Commit: "feat(domain): define SandboxStrategy enum values" - - [ ] **B1.4b** [Hamza] Add helper method to check sandbox capabilities: - - [ ] `@classmethod def supports_rollback(cls, strategy: 'SandboxStrategy') -> bool:` - Returns True for all except NONE - - [ ] `@classmethod def is_copy_based(cls, strategy: 'SandboxStrategy') -> bool:` - Returns True for COPY_ON_WRITE, OVERLAY - - [ ] Commit: "feat(domain): add SandboxStrategy helper methods" - - [ ] **B1.2** [Hamza] Define `Resource` Pydantic model in `src/cleveragents/domain/models/core/resource.py`: - - [ ] **B1.2a** [Hamza] Create basic model structure: - - [ ] Import `BaseModel, Field, field_validator` from pydantic - - [ ] Import `datetime` for timestamps - - [ ] Import `Any` from typing for metadata dict - - [ ] Create `class Resource(BaseModel):` with `model_config = ConfigDict(frozen=True)` - - [ ] Commit: "feat(domain): add Resource model scaffold" - - [ ] **B1.2b** [Hamza] Define all fields with proper types and descriptions: - - [ ] `resource_id: str = Field(..., description="ULID primary identifier")` - Required, no default - - [ ] `name: str = Field(..., min_length=1, max_length=100, description="Human-readable resource name")` - - [ ] `type: ResourceType = Field(..., description="Type of resource determining available operations")` - - [ ] `location: str = Field(..., description="Path, URL, or connection string")` - - [ ] `is_remote: bool = Field(default=False, description="Whether resource is network-accessible")` - - [ ] `sandbox_strategy: SandboxStrategy = Field(..., description="How to sandbox this resource during execution")` - - [ ] `read_only: bool = Field(default=False, description="If True, write operations are blocked")` - - [ ] `metadata: dict[str, Any] = Field(default_factory=dict, description="Additional type-specific metadata")` - - [ ] `created_at: datetime = Field(default_factory=datetime.utcnow, description="Creation timestamp")` - - [ ] Commit: "feat(domain): define Resource model fields" - - [ ] **B1.2c** [Hamza] Add validators: - - [ ] `@field_validator('resource_id')` - Validate ULID format (26 alphanumeric chars) - - [ ] `@field_validator('location')` - Validate based on type (path for filesystem, URL for git remote, etc.) - - [ ] `@field_validator('sandbox_strategy')` - Warn if incompatible with resource type (e.g., GIT_WORKTREE on FILESYSTEM) - - [ ] Add `@model_validator(mode='after')` to check sandbox_strategy is compatible with type - - [ ] Commit: "feat(domain): add Resource model validators" - - [ ] **B1.2d** [Hamza] Add computed properties and helper methods: - - [ ] `@property def supports_sandbox(self) -> bool:` - Returns True if sandbox_strategy != NONE - - [ ] `@property def can_write(self) -> bool:` - Returns True if not read_only - - [ ] `def get_sandbox_path(self, base_dir: str) -> str:` - Generate sandbox path for this resource - - [ ] Commit: "feat(domain): add Resource helper methods" - - [ ] **B1.5** [Hamza] Define `ValidationConfig` Pydantic model in `src/cleveragents/domain/models/core/project.py`: - - [ ] **B1.5a** [Hamza] Create file with ValidationConfig model: - - [ ] Import necessary Pydantic types - - [ ] Create `class ValidationConfig(BaseModel):` - - [ ] Commit: "feat(domain): create project.py with ValidationConfig scaffold" - - [ ] **B1.5b** [Hamza] Define all fields: - - [ ] `test_command: str | None = Field(default=None, description="Shell command to run tests (e.g., 'pytest')")` - - [ ] `lint_command: str | None = Field(default=None, description="Shell command to run linter (e.g., 'ruff check .')")` - - [ ] `type_check_command: str | None = Field(default=None, description="Shell command for type checking (e.g., 'pyright')")` - - [ ] `build_command: str | None = Field(default=None, description="Shell command to build project (e.g., 'npm run build')")` - - [ ] `custom_commands: dict[str, str] = Field(default_factory=dict, description="Named custom validation commands")` - - [ ] `timeout_seconds: int = Field(default=300, description="Maximum time for each validation command")` - - [ ] `fail_on_lint_error: bool = Field(default=True, description="Whether lint errors should block apply")` - - [ ] Commit: "feat(domain): define ValidationConfig fields" - - [ ] **B1.5c** [Hamza] Add helper methods: - - [ ] `def get_all_commands(self) -> dict[str, str]:` - Returns all non-None commands as dict - - [ ] `def has_any_validation(self) -> bool:` - Returns True if any command is configured - - [ ] Commit: "feat(domain): add ValidationConfig helper methods" - - [ ] **B1.6** [Hamza] Define `ContextConfig` Pydantic model: - - [ ] **B1.6a** [Hamza] Define all fields: - - [ ] `ignore_patterns: list[str] = Field(default_factory=list, description="Gitignore-style patterns to exclude from indexing")` - - [ ] `include_patterns: list[str] | None = Field(default=None, description="If set, only files matching these patterns are included")` - - [ ] `max_file_size: int = Field(default=1_000_000, description="Maximum file size in bytes to index (default 1MB)")` - - [ ] `max_files: int = Field(default=100_000, description="Maximum number of files to index")` - - [ ] `indexing_strategy: str = Field(default="full_text", description="How to index: full_text, embeddings, or both")` - - [ ] `chunking_policy: str = Field(default="smart", description="How to chunk large files: fixed, semantic, or smart")` - - [ ] `chunk_size: int = Field(default=1000, description="Target chunk size in tokens for chunking")` - - [ ] Commit: "feat(domain): define ContextConfig fields" - - [ ] **B1.6b** [Hamza] Add default ignore patterns: - - [ ] `@field_validator('ignore_patterns', mode='before')` - Merge with defaults if not explicitly empty - - [ ] Default patterns: `[".git/", "node_modules/", "__pycache__/", ".venv/", "*.pyc", ".DS_Store"]` - - [ ] Commit: "feat(domain): add ContextConfig default ignore patterns" - - [ ] **B1.1** [Hamza] Define `Project` Pydantic model in `src/cleveragents/domain/models/core/project.py`: - - [ ] **B1.1a** [Hamza] Import Resource model and create Project class: - - [ ] Import `Resource` from resource module - - [ ] Import `ValidationConfig`, `ContextConfig` from same file - - [ ] Create `class Project(BaseModel):` with proper config - - [ ] Commit: "feat(domain): add Project model scaffold" - - [ ] **B1.1b** [Hamza] Define identity fields: - - [ ] `project_id: str = Field(..., description="ULID primary identifier")` - - [ ] `name: str = Field(..., min_length=1, max_length=100, description="Project display name")` - - [ ] `namespace: str = Field(default="local", description="Namespace: local/, username/, orgname/")` - - [ ] `description: str | None = Field(default=None, max_length=500, description="Optional project description")` - - [ ] Commit: "feat(domain): define Project identity fields" - - [ ] **B1.1c** [Hamza] Define categorization and resource fields: - - [ ] `tags: list[str] = Field(default_factory=list, description="Categorization tags (e.g., 'python', 'backend')")` - - [ ] `resources: list[Resource] = Field(default_factory=list, description="Resources associated with this project")` - - [ ] `validation_config: ValidationConfig | None = Field(default=None, description="Project-level validation commands")` - - [ ] `context_config: ContextConfig = Field(default_factory=ContextConfig, description="Context indexing configuration")` - - [ ] Commit: "feat(domain): define Project categorization and resource fields" - - [ ] **B1.1d** [Hamza] Define timestamp fields: - - [ ] `created_at: datetime = Field(default_factory=datetime.utcnow)` - - [ ] `updated_at: datetime = Field(default_factory=datetime.utcnow)` - - [ ] Commit: "feat(domain): define Project timestamp fields" - - [ ] **B1.1e** [Hamza] Add computed property for is_remote: - - [ ] `@property def is_remote(self) -> bool:` - Returns True only if ALL resources have is_remote=True - - [ ] Empty resources list: return False (local by default) - - [ ] Mixed local/remote: return False (has local resources, so project is local) - - [ ] All remote: return True (can execute on server) - - [ ] Commit: "feat(domain): add Project.is_remote computed property" - - [ ] **B1.1f** [Hamza] Add namespace validator: - - [ ] `@field_validator('namespace')` - Validate namespace format - - [ ] Must match pattern: `^(local|[a-z][a-z0-9_]{0,49})$` (local or valid identifier) - - [ ] Reserved namespaces: `["openai", "anthropic", "google", "cleveragents"]` - reject these - - [ ] Commit: "feat(domain): add Project namespace validator" - - [ ] **B1.1g** [Hamza] Add namespaced_name property and helpers: - - [ ] `@property def namespaced_name(self) -> str:` - Returns f"{self.namespace}/{self.name}" - - [ ] `@classmethod def parse_namespaced_name(cls, full_name: str) -> tuple[str, str]:` - Split into (namespace, name) - - [ ] `def add_resource(self, resource: Resource) -> 'Project':` - Returns new project with resource added (immutable pattern) - - [ ] `def remove_resource(self, resource_id: str) -> 'Project':` - Returns new project without resource - - [ ] `def get_resource(self, name: str) -> Resource | None:` - Find resource by name - - [ ] Commit: "feat(domain): add Project helper methods" - - [ ] Tests: Behave scenarios for model validation - - [ ] **B1.7** [Rui] Write 25 Behave scenarios in `features/project_model.feature`: - - [ ] **B1.7a** [Rui] Project creation scenarios: - - [ ] Scenario: Create valid project with all required fields - - [ ] Scenario: Create project with optional description - - [ ] Scenario: Create project with multiple tags - - [ ] Scenario: Project creation fails with empty name - - [ ] Scenario: Project creation fails with name > 100 chars - - [ ] Commit: "test(behave): add project creation scenarios" - - [ ] **B1.7b** [Rui] Namespace validation scenarios: - - [ ] Scenario: Project namespace "local" is valid - - [ ] Scenario: Project namespace "myuser" is valid - - [ ] Scenario: Project namespace "my_org_name" is valid - - [ ] Scenario: Project namespace starting with number is invalid - - [ ] Scenario: Project namespace "openai" (reserved) is rejected - - [ ] Scenario: Project namespaced_name returns "namespace/name" format - - [ ] Commit: "test(behave): add namespace validation scenarios" - - [ ] **B1.7c** [Rui] is_remote derivation scenarios: - - [ ] Scenario: Project with no resources has is_remote=False - - [ ] Scenario: Project with one local resource has is_remote=False - - [ ] Scenario: Project with one remote resource has is_remote=True - - [ ] Scenario: Project with mixed local/remote resources has is_remote=False - - [ ] Scenario: Project with all remote resources has is_remote=True - - [ ] Commit: "test(behave): add is_remote derivation scenarios" - - [ ] **B1.7d** [Rui] Resource model scenarios: - - [ ] Scenario: Resource with each ResourceType value validates correctly - - [ ] Scenario: Resource with GIT_WORKTREE strategy on GIT_REPOSITORY is valid - - [ ] Scenario: Resource with COPY_ON_WRITE strategy on FILESYSTEM is valid - - [ ] Scenario: Resource with TRANSACTION_ROLLBACK on DATABASE is valid - - [ ] Scenario: Resource with read_only=True rejects write operations - - [ ] Scenario: Resource location validated based on type - - [ ] Commit: "test(behave): add resource model scenarios" - - [ ] **B1.7e** [Rui] ValidationConfig scenarios: - - [ ] Scenario: ValidationConfig with all commands validates - - [ ] Scenario: ValidationConfig with only test_command validates - - [ ] Scenario: ValidationConfig get_all_commands returns non-None commands - - [ ] Scenario: ValidationConfig custom_commands are included - - [ ] Commit: "test(behave): add ValidationConfig scenarios" - - [ ] **B1.7f** [Rui] ContextConfig scenarios: - - [ ] Scenario: ContextConfig ignore patterns accept glob syntax - - [ ] Scenario: ContextConfig default ignore patterns applied - - [ ] Scenario: ContextConfig max_file_size enforced - - [ ] Commit: "test(behave): add ContextConfig scenarios" - - [ ] **B1.7g** [Rui] Serialization scenarios: - - [ ] Scenario: Project JSON serialization round-trips correctly - - [ ] Scenario: Resource JSON serialization preserves enum values - - [ ] Scenario: Project with nested resources serializes completely - - [ ] Commit: "test(behave): add serialization round-trip scenarios" + **SEQUENTIAL ORDER**: B1.3 (ResourceType) → B1.4 (SandboxStrategy) → B1.2 (Resource) → B1.5 (ValidationConfig) → B1.6 (ContextConfig) → B1.1 (Project) + The order matters because Project depends on Resource, which depends on enums. + - [ ] Code: Create Project domain model + - [ ] **B1.3** [Hamza] Define `ResourceType` enum in `src/cleveragents/domain/models/core/resource.py`: + - [ ] **B1.3a** [Hamza] Create the file with proper imports: + - [ ] Import `Enum` from enum module + - [ ] Import `str` for string mixin: `class ResourceType(str, Enum):` + - [ ] Add module docstring explaining resource types + - [ ] Commit: "feat(domain): create resource.py with ResourceType enum scaffold" + - [ ] **B1.3b** [Hamza] Define enum values with descriptive docstrings: + - [ ] `GIT_REPOSITORY = "git_repository"` - Git repo (local path or remote URL), supports worktree sandboxing + - [ ] `FILESYSTEM = "filesystem"` - Local directory or file tree, supports copy-on-write sandboxing + - [ ] `DATABASE = "database"` - SQL/NoSQL database endpoint, supports transaction-based sandboxing + - [ ] `API_ENDPOINT = "api_endpoint"` - REST/GraphQL API, typically cannot be sandboxed + - [ ] `DOCUMENT_CORPUS = "document_corpus"` - Collection of documents (PDFs, markdown, wikis) + - [ ] `CLOUD_INFRASTRUCTURE = "cloud_infrastructure"` - Cloud resources (AWS, GCP, Azure) + - [ ] Commit: "feat(domain): define ResourceType enum values" + - [ ] **B1.4** [Hamza] Define `SandboxStrategy` enum in same file: + - [ ] **B1.4a** [Hamza] Define enum with string values: + - [ ] `GIT_WORKTREE = "git_worktree"` - Use `git worktree add` for isolation, efficient for git repos + - [ ] `COPY_ON_WRITE = "copy_on_write"` - Copy directory to temp location, universal but can be slow for large dirs + - [ ] `OVERLAY = "overlay"` - Use overlayfs (Linux only), efficient copy-on-write for large directories + - [ ] `TRANSACTION_ROLLBACK = "transaction_rollback"` - Database transaction that can be rolled back + - [ ] `VERSIONING = "versioning"` - Use versioning features (e.g., S3 versioning) + - [ ] `NONE = "none"` - No sandboxing possible, modifications are immediate and irreversible + - [ ] Commit: "feat(domain): define SandboxStrategy enum values" + - [ ] **B1.4b** [Hamza] Add helper method to check sandbox capabilities: + - [ ] `@classmethod def supports_rollback(cls, strategy: 'SandboxStrategy') -> bool:` - Returns True for all except NONE + - [ ] `@classmethod def is_copy_based(cls, strategy: 'SandboxStrategy') -> bool:` - Returns True for COPY_ON_WRITE, OVERLAY + - [ ] Commit: "feat(domain): add SandboxStrategy helper methods" + - [ ] **B1.2** [Hamza] Define `Resource` Pydantic model in `src/cleveragents/domain/models/core/resource.py`: + - [ ] **B1.2a** [Hamza] Create basic model structure: + - [ ] Import `BaseModel, Field, field_validator` from pydantic + - [ ] Import `datetime` for timestamps + - [ ] Import `Any` from typing for metadata dict + - [ ] Create `class Resource(BaseModel):` with `model_config = ConfigDict(frozen=True)` + - [ ] Commit: "feat(domain): add Resource model scaffold" + - [ ] **B1.2b** [Hamza] Define all fields with proper types and descriptions: + - [ ] `resource_id: str = Field(..., description="ULID primary identifier")` - Required, no default + - [ ] `name: str = Field(..., min_length=1, max_length=100, description="Human-readable resource name")` + - [ ] `type: ResourceType = Field(..., description="Type of resource determining available operations")` + - [ ] `location: str = Field(..., description="Path, URL, or connection string")` + - [ ] `is_remote: bool = Field(default=False, description="Whether resource is network-accessible")` + - [ ] `sandbox_strategy: SandboxStrategy = Field(..., description="How to sandbox this resource during execution")` + - [ ] `read_only: bool = Field(default=False, description="If True, write operations are blocked")` + - [ ] `metadata: dict[str, Any] = Field(default_factory=dict, description="Additional type-specific metadata")` + - [ ] `created_at: datetime = Field(default_factory=datetime.utcnow, description="Creation timestamp")` + - [ ] Commit: "feat(domain): define Resource model fields" + - [ ] **B1.2c** [Hamza] Add validators: + - [ ] `@field_validator('resource_id')` - Validate ULID format (26 alphanumeric chars) + - [ ] `@field_validator('location')` - Validate based on type (path for filesystem, URL for git remote, etc.) + - [ ] `@field_validator('sandbox_strategy')` - Warn if incompatible with resource type (e.g., GIT_WORKTREE on FILESYSTEM) + - [ ] Add `@model_validator(mode='after')` to check sandbox_strategy is compatible with type + - [ ] Commit: "feat(domain): add Resource model validators" + - [ ] **B1.2d** [Hamza] Add computed properties and helper methods: + - [ ] `@property def supports_sandbox(self) -> bool:` - Returns True if sandbox_strategy != NONE + - [ ] `@property def can_write(self) -> bool:` - Returns True if not read_only + - [ ] `def get_sandbox_path(self, base_dir: str) -> str:` - Generate sandbox path for this resource + - [ ] Commit: "feat(domain): add Resource helper methods" + - [ ] **B1.5** [Hamza] Define `ValidationConfig` Pydantic model in `src/cleveragents/domain/models/core/project.py`: + - [ ] **B1.5a** [Hamza] Create file with ValidationConfig model: + - [ ] Import necessary Pydantic types + - [ ] Create `class ValidationConfig(BaseModel):` + - [ ] Commit: "feat(domain): create project.py with ValidationConfig scaffold" + - [ ] **B1.5b** [Hamza] Define all fields: + - [ ] `test_command: str | None = Field(default=None, description="Shell command to run tests (e.g., 'pytest')")` + - [ ] `lint_command: str | None = Field(default=None, description="Shell command to run linter (e.g., 'ruff check .')")` + - [ ] `type_check_command: str | None = Field(default=None, description="Shell command for type checking (e.g., 'pyright')")` + - [ ] `build_command: str | None = Field(default=None, description="Shell command to build project (e.g., 'npm run build')")` + - [ ] `custom_commands: dict[str, str] = Field(default_factory=dict, description="Named custom validation commands")` + - [ ] `timeout_seconds: int = Field(default=300, description="Maximum time for each validation command")` + - [ ] `fail_on_lint_error: bool = Field(default=True, description="Whether lint errors should block apply")` + - [ ] Commit: "feat(domain): define ValidationConfig fields" + - [ ] **B1.5c** [Hamza] Add helper methods: + - [ ] `def get_all_commands(self) -> dict[str, str]:` - Returns all non-None commands as dict + - [ ] `def has_any_validation(self) -> bool:` - Returns True if any command is configured + - [ ] Commit: "feat(domain): add ValidationConfig helper methods" + - [ ] **B1.6** [Hamza] Define `ContextConfig` Pydantic model: + - [ ] **B1.6a** [Hamza] Define all fields: + - [ ] `ignore_patterns: list[str] = Field(default_factory=list, description="Gitignore-style patterns to exclude from indexing")` + - [ ] `include_patterns: list[str] | None = Field(default=None, description="If set, only files matching these patterns are included")` + - [ ] `max_file_size: int = Field(default=1_000_000, description="Maximum file size in bytes to index (default 1MB)")` + - [ ] `max_files: int = Field(default=100_000, description="Maximum number of files to index")` + - [ ] `indexing_strategy: str = Field(default="full_text", description="How to index: full_text, embeddings, or both")` + - [ ] `chunking_policy: str = Field(default="smart", description="How to chunk large files: fixed, semantic, or smart")` + - [ ] `chunk_size: int = Field(default=1000, description="Target chunk size in tokens for chunking")` + - [ ] Commit: "feat(domain): define ContextConfig fields" + - [ ] **B1.6b** [Hamza] Add default ignore patterns: + - [ ] `@field_validator('ignore_patterns', mode='before')` - Merge with defaults if not explicitly empty + - [ ] Default patterns: `[".git/", "node_modules/", "__pycache__/", ".venv/", "*.pyc", ".DS_Store"]` + - [ ] Commit: "feat(domain): add ContextConfig default ignore patterns" + - [ ] **B1.1** [Hamza] Define `Project` Pydantic model in `src/cleveragents/domain/models/core/project.py`: + - [ ] **B1.1a** [Hamza] Import Resource model and create Project class: + - [ ] Import `Resource` from resource module + - [ ] Import `ValidationConfig`, `ContextConfig` from same file + - [ ] Create `class Project(BaseModel):` with proper config + - [ ] Commit: "feat(domain): add Project model scaffold" + - [ ] **B1.1b** [Hamza] Define identity fields: + - [ ] `project_id: str = Field(..., description="ULID primary identifier")` + - [ ] `name: str = Field(..., min_length=1, max_length=100, description="Project display name")` + - [ ] `namespace: str = Field(default="local", description="Namespace: local/, username/, orgname/")` + - [ ] `description: str | None = Field(default=None, max_length=500, description="Optional project description")` + - [ ] Commit: "feat(domain): define Project identity fields" + - [ ] **B1.1c** [Hamza] Define categorization and resource fields: + - [ ] `tags: list[str] = Field(default_factory=list, description="Categorization tags (e.g., 'python', 'backend')")` + - [ ] `resources: list[Resource] = Field(default_factory=list, description="Resources associated with this project")` + - [ ] `validation_config: ValidationConfig | None = Field(default=None, description="Project-level validation commands")` + - [ ] `context_config: ContextConfig = Field(default_factory=ContextConfig, description="Context indexing configuration")` + - [ ] Commit: "feat(domain): define Project categorization and resource fields" + - [ ] **B1.1d** [Hamza] Define timestamp fields: + - [ ] `created_at: datetime = Field(default_factory=datetime.utcnow)` + - [ ] `updated_at: datetime = Field(default_factory=datetime.utcnow)` + - [ ] Commit: "feat(domain): define Project timestamp fields" + - [ ] **B1.1e** [Hamza] Add computed property for is_remote: + - [ ] `@property def is_remote(self) -> bool:` - Returns True only if ALL resources have is_remote=True + - [ ] Empty resources list: return False (local by default) + - [ ] Mixed local/remote: return False (has local resources, so project is local) + - [ ] All remote: return True (can execute on server) + - [ ] Commit: "feat(domain): add Project.is_remote computed property" + - [ ] **B1.1f** [Hamza] Add namespace validator: + - [ ] `@field_validator('namespace')` - Validate namespace format + - [ ] Must match pattern: `^(local|[a-z][a-z0-9_]{0,49})$` (local or valid identifier) + - [ ] Reserved namespaces: `["openai", "anthropic", "google", "cleveragents"]` - reject these + - [ ] Commit: "feat(domain): add Project namespace validator" + - [ ] **B1.1g** [Hamza] Add namespaced_name property and helpers: + - [ ] `@property def namespaced_name(self) -> str:` - Returns f"{self.namespace}/{self.name}" + - [ ] `@classmethod def parse_namespaced_name(cls, full_name: str) -> tuple[str, str]:` - Split into (namespace, name) + - [ ] `def add_resource(self, resource: Resource) -> 'Project':` - Returns new project with resource added (immutable pattern) + - [ ] `def remove_resource(self, resource_id: str) -> 'Project':` - Returns new project without resource + - [ ] `def get_resource(self, name: str) -> Resource | None:` - Find resource by name + - [ ] Commit: "feat(domain): add Project helper methods" + - [ ] Tests: Behave scenarios for model validation + - [ ] **B1.7** [Rui] Write 25 Behave scenarios in `features/project_model.feature`: + - [ ] **B1.7a** [Rui] Project creation scenarios: + - [ ] Scenario: Create valid project with all required fields + - [ ] Scenario: Create project with optional description + - [ ] Scenario: Create project with multiple tags + - [ ] Scenario: Project creation fails with empty name + - [ ] Scenario: Project creation fails with name > 100 chars + - [ ] Commit: "test(behave): add project creation scenarios" + - [ ] **B1.7b** [Rui] Namespace validation scenarios: + - [ ] Scenario: Project namespace "local" is valid + - [ ] Scenario: Project namespace "myuser" is valid + - [ ] Scenario: Project namespace "my_org_name" is valid + - [ ] Scenario: Project namespace starting with number is invalid + - [ ] Scenario: Project namespace "openai" (reserved) is rejected + - [ ] Scenario: Project namespaced_name returns "namespace/name" format + - [ ] Commit: "test(behave): add namespace validation scenarios" + - [ ] **B1.7c** [Rui] is_remote derivation scenarios: + - [ ] Scenario: Project with no resources has is_remote=False + - [ ] Scenario: Project with one local resource has is_remote=False + - [ ] Scenario: Project with one remote resource has is_remote=True + - [ ] Scenario: Project with mixed local/remote resources has is_remote=False + - [ ] Scenario: Project with all remote resources has is_remote=True + - [ ] Commit: "test(behave): add is_remote derivation scenarios" + - [ ] **B1.7d** [Rui] Resource model scenarios: + - [ ] Scenario: Resource with each ResourceType value validates correctly + - [ ] Scenario: Resource with GIT_WORKTREE strategy on GIT_REPOSITORY is valid + - [ ] Scenario: Resource with COPY_ON_WRITE strategy on FILESYSTEM is valid + - [ ] Scenario: Resource with TRANSACTION_ROLLBACK on DATABASE is valid + - [ ] Scenario: Resource with read_only=True rejects write operations + - [ ] Scenario: Resource location validated based on type + - [ ] Commit: "test(behave): add resource model scenarios" + - [ ] **B1.7e** [Rui] ValidationConfig scenarios: + - [ ] Scenario: ValidationConfig with all commands validates + - [ ] Scenario: ValidationConfig with only test_command validates + - [ ] Scenario: ValidationConfig get_all_commands returns non-None commands + - [ ] Scenario: ValidationConfig custom_commands are included + - [ ] Commit: "test(behave): add ValidationConfig scenarios" + - [ ] **B1.7f** [Rui] ContextConfig scenarios: + - [ ] Scenario: ContextConfig ignore patterns accept glob syntax + - [ ] Scenario: ContextConfig default ignore patterns applied + - [ ] Scenario: ContextConfig max_file_size enforced + - [ ] Commit: "test(behave): add ContextConfig scenarios" + - [ ] **B1.7g** [Rui] Serialization scenarios: + - [ ] Scenario: Project JSON serialization round-trips correctly + - [ ] Scenario: Resource JSON serialization preserves enum values + - [ ] Scenario: Project with nested resources serializes completely + - [ ] Commit: "test(behave): add serialization round-trip scenarios" - [ ] **Stage B2: Project CLI Commands** (Day 3-4) **[Hamza]** - **SEQUENTIAL ORDER**: B2.1 (File scaffold) → B2.2 (Create) → B2.3 (Add resource) → B2.4 (Remove resource) → B2.5 (List) → B2.6 (Show) → B2.7 (Validation) → B2.8 (Delete) → B2.9 (Register) + **SEQUENTIAL ORDER**: B2.1 (File scaffold) → B2.2 (Create) → B2.3 (Add resource) → B2.4 (Remove resource) → B2.5 (List) → B2.6 (Show) → B2.7 (Validation) → B2.8 (Delete) → B2.9 (Register) + - [ ] Code: Implement project CLI + - [ ] **B2.1** [Hamza] Create `src/cleveragents/cli/commands/project.py` scaffold: + - [ ] **B2.1a** [Hamza] Create file with imports and Click group: + - [ ] Import `click` for CLI framework + - [ ] Import `rich.console.Console`, `rich.table.Table` for output + - [ ] Import Project, Resource models from domain + - [ ] Import ProjectService from application.services + - [ ] Create `@click.group(name="project")` decorator + - [ ] Add docstring: "Manage projects and their resources" + - [ ] Commit: "feat(cli): create project.py with Click group scaffold" + - [ ] **B2.1b** [Hamza] Create ProjectService in `src/cleveragents/application/services/project_service.py`: + - [ ] Import ProjectRepository, ResourceRepository + - [ ] Define `class ProjectService:` + - [ ] Add `__init__(self, project_repo: ProjectRepository, resource_repo: ResourceRepository)` + - [ ] Add stub methods: `create_project()`, `get_project()`, `list_projects()`, `delete_project()` + - [ ] Commit: "feat(service): add ProjectService scaffold" + - [ ] **B2.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project create` command: + - [ ] **B2.2a** [Hamza] Define command signature: - - [ ] Code: Implement project CLI - - [ ] **B2.1** [Hamza] Create `src/cleveragents/cli/commands/project.py` scaffold: - - [ ] **B2.1a** [Hamza] Create file with imports and Click group: - - [ ] Import `click` for CLI framework - - [ ] Import `rich.console.Console`, `rich.table.Table` for output - - [ ] Import Project, Resource models from domain - - [ ] Import ProjectService from application.services - - [ ] Create `@click.group(name="project")` decorator - - [ ] Add docstring: "Manage projects and their resources" - - [ ] Commit: "feat(cli): create project.py with Click group scaffold" - - [ ] **B2.1b** [Hamza] Create ProjectService in `src/cleveragents/application/services/project_service.py`: - - [ ] Import ProjectRepository, ResourceRepository - - [ ] Define `class ProjectService:` - - [ ] Add `__init__(self, project_repo: ProjectRepository, resource_repo: ResourceRepository)` - - [ ] Add stub methods: `create_project()`, `get_project()`, `list_projects()`, `delete_project()` - - [ ] Commit: "feat(service): add ProjectService scaffold" - - [ ] **B2.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project create` command: - - [ ] **B2.2a** [Hamza] Define command signature: - ```python - @project.command("create") - @click.option("--name", "-n", required=True, help="Project name (namespace/name format)") - @click.option("--description", "-d", default=None, help="Project description") - @click.option("--tag", "-t", multiple=True, help="Project tags (can specify multiple)") - def create_project(name: str, description: str | None, tag: tuple[str, ...]): - ``` - - [ ] Commit: "feat(cli): add project create command signature" - - [ ] **B2.2b** [Hamza] Implement namespace parsing: - - [ ] Split name on "/" to get (namespace, short_name) - - [ ] If no "/" present, default namespace to "local" - - [ ] Validate namespace: must match `^(local|[a-z][a-z0-9_]{0,49})$` - - [ ] Validate short_name: 1-100 chars, alphanumeric + hyphens - - [ ] Raise `click.BadParameter` on validation failure - - [ ] Commit: "feat(cli): implement namespace parsing in project create" - - [ ] **B2.2c** [Hamza] Implement project creation: - - [ ] Generate ULID for project_id: `ulid.new().str` - - [ ] Create `Project` domain model with all fields - - [ ] Call `project_service.create_project(project)` - - [ ] Handle `DuplicateProjectError` - display user-friendly message - - [ ] Commit: "feat(cli): implement project creation logic" - - [ ] **B2.2d** [Hamza] Implement success output: - - [ ] Display: "Created project: {namespace}/{short_name}" - - [ ] Display: "Project ID: {project_id}" - - [ ] Display: "Tags: {tags}" if any - - [ ] Use Rich console for colored output (green for success) - - [ ] Commit: "feat(cli): add project create success output" - - [ ] **B2.2e** [Hamza] Add ProjectService.create_project() implementation: - - [ ] Validate project.name is unique via repository - - [ ] If duplicate, raise `DuplicateProjectError(name=project.name)` - - [ ] Persist via `self._project_repo.create(project)` - - [ ] Return created project - - [ ] Commit: "feat(service): implement ProjectService.create_project()" - - [ ] **B2.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project add-resource` command: - - [ ] **B2.3a** [Hamza] Define command signature: - ```python - @project.command("add-resource") - @click.option("--project", "-p", required=True, help="Project name (namespace/name)") - @click.option("--name", "-n", required=True, help="Resource name") - @click.option("--type", "-t", "resource_type", required=True, - type=click.Choice(["git_repository", "filesystem", "database", "api_endpoint"])) - @click.option("--location", "-l", required=True, help="Path, URL, or connection string") - @click.option("--sandbox-strategy", "-s", required=True, - type=click.Choice(["git_worktree", "copy_on_write", "transaction_rollback", "none"])) - @click.option("--read-only", is_flag=True, help="Mark resource as read-only") - @click.option("--metadata", "-m", multiple=True, help="Key=value metadata pairs") - ``` - - [ ] Commit: "feat(cli): add project add-resource command signature" - - [ ] **B2.3b** [Hamza] Implement resource type validation: - - [ ] Map CLI type string to ResourceType enum - - [ ] Validate sandbox strategy is compatible with resource type: - - [ ] git_repository: allows git_worktree, copy_on_write, none - - [ ] filesystem: allows copy_on_write, overlay, none - - [ ] database: allows transaction_rollback, none - - [ ] api_endpoint: only allows none - - [ ] Raise `click.BadParameter` if incompatible - - [ ] Commit: "feat(cli): validate resource type and sandbox strategy compatibility" - - [ ] **B2.3c** [Hamza] Implement location validation: - - [ ] For git_repository: validate path exists or URL is valid git URL - - [ ] For filesystem: validate path exists and is directory - - [ ] For database: validate connection string format (basic check) - - [ ] For api_endpoint: validate URL format - - [ ] Commit: "feat(cli): validate resource location by type" - - [ ] **B2.3d** [Hamza] Implement metadata parsing: - - [ ] Parse each `--metadata` value as "key=value" - - [ ] Build dict from all pairs - - [ ] Handle missing "=" gracefully (error) - - [ ] Commit: "feat(cli): parse metadata key=value pairs" - - [ ] **B2.3e** [Hamza] Implement resource creation and linking: - - [ ] Fetch project by namespaced name - - [ ] Raise `ProjectNotFoundError` if not exists - - [ ] Check resource name is unique within project - - [ ] Create Resource with ULID and all fields - - [ ] Add resource to project: `project_service.add_resource(project_id, resource)` - - [ ] Display success: "Added resource '{name}' to project '{project_name}'" - - [ ] Commit: "feat(cli): implement add-resource creation and linking" - - [ ] **B2.3f** [Hamza] Add ProjectService.add_resource() implementation: - - [ ] Fetch project from repository - - [ ] Check for duplicate resource name in project - - [ ] Create new project with resource added (immutable pattern) - - [ ] Recompute project.is_remote based on all resources - - [ ] Update project in repository - - [ ] Create resource in resource repository - - [ ] Return updated project - - [ ] Commit: "feat(service): implement ProjectService.add_resource()" - - [ ] **B2.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project remove-resource` command: - - [ ] **B2.4a** [Hamza] Define command signature: - ```python - @project.command("remove-resource") - @click.option("--project", "-p", required=True, help="Project name") - @click.option("--name", "-n", required=True, help="Resource name to remove") - @click.option("--yes", is_flag=True, help="Skip confirmation") - ``` - - [ ] Commit: "feat(cli): add project remove-resource command signature" - - [ ] **B2.4b** [Hamza] Implement removal logic: - - [ ] Fetch project and validate resource exists - - [ ] If not --yes, prompt for confirmation: "Remove resource '{name}'? [y/N]" - - [ ] Call `project_service.remove_resource(project_id, resource_name)` - - [ ] Display success: "Removed resource '{name}' from project" - - [ ] Commit: "feat(cli): implement remove-resource logic" - - [ ] **B2.4c** [Hamza] Add ProjectService.remove_resource() implementation: - - [ ] Fetch project - - [ ] Find resource by name - - [ ] Raise `ResourceNotFoundError` if not exists - - [ ] Create new project without resource (immutable pattern) - - [ ] Recompute is_remote - - [ ] Update project - - [ ] Delete resource from resource repository - - [ ] Return updated project - - [ ] Commit: "feat(service): implement ProjectService.remove_resource()" - - [ ] **B2.5** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project list` command: - - [ ] **B2.5a** [Hamza] Define command signature: - ```python - @project.command("list") - @click.option("--namespace", "-n", default=None, help="Filter by namespace") - @click.option("--tag", "-t", default=None, help="Filter by tag") - @click.option("--format", "output_format", type=click.Choice(["table", "json"]), default="table") - ``` - - [ ] Commit: "feat(cli): add project list command signature" - - [ ] **B2.5b** [Hamza] Implement query and filtering: - - [ ] Call `project_service.list_projects(namespace=namespace, tag=tag)` - - [ ] If namespace provided, filter by namespace - - [ ] If tag provided, filter projects that have this tag - - [ ] Commit: "feat(cli): implement project list filtering" - - [ ] **B2.5c** [Hamza] Implement table output: - - [ ] Create Rich Table with columns: ID, Name, Resources, Tags, Remote - - [ ] Add row for each project: - - [ ] ID: first 8 chars of project_id - - [ ] Name: namespaced_name - - [ ] Resources: count of resources - - [ ] Tags: comma-separated tags (truncate if >3) - - [ ] Remote: "Yes" or "No" based on is_remote - - [ ] Display table via console.print() - - [ ] If no projects found, display "No projects found" - - [ ] Commit: "feat(cli): implement project list table output" - - [ ] **B2.5d** [Hamza] Implement JSON output: - - [ ] If format=json, serialize projects to JSON - - [ ] Use model_dump_json() for each project - - [ ] Print to stdout (for piping to jq, etc.) - - [ ] Commit: "feat(cli): implement project list JSON output" - - [ ] **B2.5e** [Hamza] Add ProjectService.list_projects() implementation: - - [ ] Call `project_repo.list_all()` - - [ ] Apply namespace filter if provided - - [ ] Apply tag filter if provided - - [ ] Return filtered list - - [ ] Commit: "feat(service): implement ProjectService.list_projects()" - - [ ] **B2.6** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project show` command: - - [ ] **B2.6a** [Hamza] Define command signature: - ```python - @project.command("show") - @click.argument("name") - @click.option("--format", "output_format", type=click.Choice(["rich", "json"]), default="rich") - ``` - - [ ] Commit: "feat(cli): add project show command signature" - - [ ] **B2.6b** [Hamza] Implement project fetch and rich display: - - [ ] Fetch project by namespaced name via service - - [ ] If not found, display error and exit(1) - - [ ] Display project details using Rich panels: - ``` - ╭─ Project: local/my-project ────────────────────────╮ - │ ID: 01ARZ3NDEKTSV4RRFFQ69G5FAV │ - │ Description: My awesome project │ - │ Tags: python, backend │ - │ Remote: No │ - │ Created: 2024-01-15 10:30:00 │ - ╰───────────────────────────────────────────────────╯ + ```python + @project.command("create") + @click.option("--name", "-n", required=True, help="Project name (namespace/name format)") + @click.option("--description", "-d", default=None, help="Project description") + @click.option("--tag", "-t", multiple=True, help="Project tags (can specify multiple)") + def create_project(name: str, description: str | None, tag: tuple[str, ...]): + ``` - Resources (2): - ┌─────────────────┬──────────────────┬─────────────────┬──────────┐ - │ Name │ Type │ Location │ Strategy │ - ├─────────────────┼──────────────────┼─────────────────┼──────────┤ - │ source │ git_repository │ /path/to/repo │ worktree │ - │ config │ filesystem │ /path/to/config │ copy │ - └─────────────────┴──────────────────┴─────────────────┴──────────┘ + - [ ] Commit: "feat(cli): add project create command signature" - Validation Config: - Test: pytest tests/ - Lint: ruff check src/ - Type Check: pyright src/ - ``` - - [ ] Commit: "feat(cli): implement project show rich display" - - [ ] **B2.6c** [Hamza] Implement JSON output: - - [ ] If format=json, output full project as JSON - - [ ] Include all resources and validation config - - [ ] Commit: "feat(cli): implement project show JSON output" - - [ ] **B2.6d** [Hamza] Add ProjectService.get_project() implementation: - - [ ] Parse namespaced name to (namespace, short_name) - - [ ] Query by namespaced_name OR by project_id (support both) - - [ ] Return Project with resources loaded - - [ ] Commit: "feat(service): implement ProjectService.get_project()" - - [ ] **B2.7** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project set-validation` command: - - [ ] **B2.7a** [Hamza] Define command signature: - ```python - @project.command("set-validation") - @click.option("--project", "-p", required=True, help="Project name") - @click.option("--test-command", default=None, help="Command to run tests") - @click.option("--lint-command", default=None, help="Command to run linter") - @click.option("--type-check-command", default=None, help="Command for type checking") - @click.option("--build-command", default=None, help="Command to build project") - @click.option("--timeout", default=300, type=int, help="Timeout for each command (seconds)") - @click.option("--clear", is_flag=True, help="Clear all validation config") - ``` - - [ ] Commit: "feat(cli): add project set-validation command signature" - - [ ] **B2.7b** [Hamza] Implement validation config update: - - [ ] Fetch project - - [ ] If --clear, set validation_config to None - - [ ] Otherwise, create ValidationConfig with provided commands - - [ ] Only set commands that were explicitly provided (preserve existing if not specified) - - [ ] Call `project_service.update_validation(project_id, config)` - - [ ] Display updated config summary - - [ ] Commit: "feat(cli): implement set-validation logic" - - [ ] **B2.7c** [Hamza] Add ProjectService.update_validation() implementation: - - [ ] Fetch project - - [ ] Merge new config with existing (if not --clear) - - [ ] Update project with new validation_config - - [ ] Persist via repository - - [ ] Commit: "feat(service): implement ProjectService.update_validation()" - - [ ] **B2.8** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project delete` command: - - [ ] **B2.8a** [Hamza] Define command signature: - ```python - @project.command("delete") - @click.argument("name") - @click.option("--force", "-f", is_flag=True, help="Force delete even if plans exist") - @click.option("--yes", is_flag=True, help="Skip confirmation") - ``` - - [ ] Commit: "feat(cli): add project delete command signature" - - [ ] **B2.8b** [Hamza] Implement deletion checks: - - [ ] Fetch project - - [ ] Check for active plans using this project: `plan_repo.count(project_id=project.project_id)` - - [ ] If plans exist and not --force: - - [ ] Display error: "Cannot delete project with {n} active plans. Use --force to delete anyway." - - [ ] List plan IDs (first 5) - - [ ] Exit with code 1 - - [ ] If --force, display warning: "Deleting project with {n} active plans" - - [ ] Commit: "feat(cli): implement project delete safety checks" - - [ ] **B2.8c** [Hamza] Implement deletion: - - [ ] Prompt for confirmation: "Delete project '{name}'? This cannot be undone. [y/N]" - - [ ] Support `--yes` to bypass confirmation - - [ ] If confirmed (or --yes), call `project_service.delete_project(project_id)` - - [ ] Display success: "Deleted project '{name}'" - - [ ] Commit: "feat(cli): implement project delete confirmation and execution" - - [ ] **B2.8d** [Hamza] Add ProjectService.delete_project() implementation: - - [ ] Delete all resources for project via resource_repo - - [ ] Delete project via project_repo - - [ ] Return True on success - - [ ] Commit: "feat(service): implement ProjectService.delete_project()" - - [ ] **B2.9** [Hamza] Register project commands in `src/cleveragents/cli/main.py`: - - [ ] **B2.9a** [Hamza] Import and register: - - [ ] Add `from cleveragents.cli.commands.project import project as project_group` - - [ ] Add `app.add_command(project_group)` in main app setup - - [ ] Verify `agents [--data-dir PATH] [--config-path PATH] project --help` shows all subcommands - - [ ] Commit: "feat(cli): register project commands in main CLI" - - [ ] **B2.9b** [Hamza] Add DI wiring for ProjectService: - - [ ] Update container.py to provide ProjectService - - [ ] Inject into CLI commands via Click context or similar pattern - - [ ] Commit: "feat(di): wire ProjectService into CLI" - - [ ] Tests: Behave + Robot for all project CLI commands - - [ ] **B2.10** [Rui] Write Behave scenarios in `features/project_cli.feature`: - - [ ] **B2.10a** [Rui] Project creation scenarios: - - [ ] Scenario: Create project with valid name succeeds - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/test-project` - - [ ] Then the output contains "Created project: local/test-project" - - [ ] And the output contains "Project ID:" - - [ ] Scenario: Create project with description and tags - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/test --description "My project" --tag python --tag backend` - - [ ] Then the project has description "My project" - - [ ] And the project has tags "python", "backend" - - [ ] Scenario: Create project with duplicate name fails - - [ ] Given a project "local/existing" exists - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/existing` - - [ ] Then the exit code is 1 - - [ ] And the output contains "already exists" - - [ ] Scenario: Create project with invalid namespace fails - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name 123invalid/test` - - [ ] Then the exit code is 1 - - [ ] And the output contains "Invalid namespace" - - [ ] Commit: "test(behave): add project create CLI scenarios" - - [ ] **B2.10b** [Rui] Add resource scenarios: - - [ ] Scenario: Add git repository resource to project - - [ ] Given a project "local/test" exists - - [ ] And a git repository exists at "/tmp/test-repo" - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name source --type git_repository --location /tmp/test-repo --sandbox-strategy git_worktree` - - [ ] Then the output contains "Added resource 'source'" - - [ ] Scenario: Add filesystem resource to project - - [ ] Given a project "local/test" exists - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name config --type filesystem --location /tmp/config --sandbox-strategy copy_on_write` - - [ ] Then the output contains "Added resource 'config'" - - [ ] Scenario: Add resource with incompatible sandbox strategy fails - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name api --type api_endpoint --location https://api.example.com --sandbox-strategy git_worktree` - - [ ] Then the exit code is 1 - - [ ] And the output contains "incompatible" - - [ ] Scenario: Add resource with metadata - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource ... --metadata branch=main --metadata remote=origin` - - [ ] Then the resource has metadata key "branch" with value "main" - - [ ] Commit: "test(behave): add resource CLI scenarios" - - [ ] **B2.10c** [Rui] Remove resource scenarios: - - [ ] Scenario: Remove resource from project succeeds - - [ ] Given project "local/test" has resource "source" - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project remove-resource --project local/test --name source --yes` - - [ ] Then the output contains "Removed resource 'source'" - - [ ] Scenario: Remove non-existent resource fails gracefully - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project remove-resource --project local/test --name nonexistent --yes` - - [ ] Then the exit code is 1 - - [ ] And the output contains "not found" - - [ ] Commit: "test(behave): add remove-resource CLI scenarios" - - [ ] **B2.10d** [Rui] List and show scenarios: - - [ ] Scenario: List projects shows all projects - - [ ] Given projects "local/proj1" and "local/proj2" exist - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list` - - [ ] Then the output contains "proj1" - - [ ] And the output contains "proj2" - - [ ] Scenario: List projects with namespace filter works - - [ ] Given projects "local/proj1" and "team/proj2" exist - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list --namespace local` - - [ ] Then the output contains "proj1" - - [ ] And the output does not contain "proj2" - - [ ] Scenario: List projects JSON format works - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list --format json` - - [ ] Then the output is valid JSON - - [ ] Scenario: Show project displays full details - - [ ] Given project "local/test" with 2 resources exists - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project show local/test` - - [ ] Then the output contains "local/test" - - [ ] And the output contains "Resources (2)" - - [ ] Commit: "test(behave): add list and show CLI scenarios" - - [ ] **B2.10e** [Rui] Validation config scenarios: - - [ ] Scenario: Set validation commands persists correctly - - [ ] Given project "local/test" exists - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project set-validation --project local/test --test-command "pytest" --lint-command "ruff check"` - - [ ] Then project "local/test" has test_command "pytest" - - [ ] And project "local/test" has lint_command "ruff check" - - [ ] Scenario: Clear validation config works - - [ ] Given project "local/test" has validation config - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project set-validation --project local/test --clear` - - [ ] Then project "local/test" has no validation config - - [ ] Commit: "test(behave): add validation config CLI scenarios" - - [ ] **B2.10f** [Rui] Delete scenarios: - - [ ] Scenario: Delete project succeeds - - [ ] Given project "local/test" exists with no plans - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --yes` - - [ ] Then the output contains "Deleted project" - - [ ] And project "local/test" no longer exists - - [ ] Scenario: Delete project with active plans blocked without --force - - [ ] Given project "local/test" exists - - [ ] And a plan uses project "local/test" - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --yes` - - [ ] Then the exit code is 1 - - [ ] And the output contains "active plans" - - [ ] Scenario: Delete project with active plans succeeds with --force - - [ ] Given project "local/test" exists with active plans - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --force --yes` - - [ ] Then the output contains "Deleted project" - - [ ] Commit: "test(behave): add delete CLI scenarios" - - [ ] **B2.11** [Rui] Write Robot integration test `robot/project_cli_integration.robot`: - - [ ] **B2.11a** [Rui] Full lifecycle test: - - [ ] Test: Full project lifecycle - - [ ] Create project with description and tags - - [ ] Add git repository resource - - [ ] Add filesystem resource - - [ ] Show project and verify all details - - [ ] Set validation commands - - [ ] List projects and verify presence - - [ ] Remove one resource - - [ ] Delete project - - [ ] Verify project no longer exists - - [ ] Commit: "test(robot): add full project lifecycle e2e test" - - [ ] **B2.11b** [Rui] Multi-resource test: - - [ ] Test: Project with multiple resources of different types - - [ ] Create project - - [ ] Add git repo resource (primary code) - - [ ] Add filesystem resource (docs) - - [ ] Add database resource (read-only) - - [ ] Verify is_remote is computed correctly (should be False - has local resources) - - [ ] Show project and verify all resources listed - - [ ] Commit: "test(robot): add multi-resource project e2e test" + - [ ] **B2.2b** [Hamza] Implement namespace parsing: + - [ ] Split name on "/" to get (namespace, short_name) + - [ ] If no "/" present, default namespace to "local" + - [ ] Validate namespace: must match `^(local|[a-z][a-z0-9_]{0,49})$` + - [ ] Validate short_name: 1-100 chars, alphanumeric + hyphens + - [ ] Raise `click.BadParameter` on validation failure + - [ ] Commit: "feat(cli): implement namespace parsing in project create" + - [ ] **B2.2c** [Hamza] Implement project creation: + - [ ] Generate ULID for project_id: `ulid.new().str` + - [ ] Create `Project` domain model with all fields + - [ ] Call `project_service.create_project(project)` + - [ ] Handle `DuplicateProjectError` - display user-friendly message + - [ ] Commit: "feat(cli): implement project creation logic" + - [ ] **B2.2d** [Hamza] Implement success output: + - [ ] Display: "Created project: {namespace}/{short_name}" + - [ ] Display: "Project ID: {project_id}" + - [ ] Display: "Tags: {tags}" if any + - [ ] Use Rich console for colored output (green for success) + - [ ] Commit: "feat(cli): add project create success output" + - [ ] **B2.2e** [Hamza] Add ProjectService.create_project() implementation: + - [ ] Validate project.name is unique via repository + - [ ] If duplicate, raise `DuplicateProjectError(name=project.name)` + - [ ] Persist via `self._project_repo.create(project)` + - [ ] Return created project + - [ ] Commit: "feat(service): implement ProjectService.create_project()" + + - [ ] **B2.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project add-resource` command: + - [ ] **B2.3a** [Hamza] Define command signature: + + ```python + @project.command("add-resource") + @click.option("--project", "-p", required=True, help="Project name (namespace/name)") + @click.option("--name", "-n", required=True, help="Resource name") + @click.option("--type", "-t", "resource_type", required=True, + type=click.Choice(["git_repository", "filesystem", "database", "api_endpoint"])) + @click.option("--location", "-l", required=True, help="Path, URL, or connection string") + @click.option("--sandbox-strategy", "-s", required=True, + type=click.Choice(["git_worktree", "copy_on_write", "transaction_rollback", "none"])) + @click.option("--read-only", is_flag=True, help="Mark resource as read-only") + @click.option("--metadata", "-m", multiple=True, help="Key=value metadata pairs") + ``` + + - [ ] Commit: "feat(cli): add project add-resource command signature" + + - [ ] **B2.3b** [Hamza] Implement resource type validation: + - [ ] Map CLI type string to ResourceType enum + - [ ] Validate sandbox strategy is compatible with resource type: + - [ ] git_repository: allows git_worktree, copy_on_write, none + - [ ] filesystem: allows copy_on_write, overlay, none + - [ ] database: allows transaction_rollback, none + - [ ] api_endpoint: only allows none + - [ ] Raise `click.BadParameter` if incompatible + - [ ] Commit: "feat(cli): validate resource type and sandbox strategy compatibility" + - [ ] **B2.3c** [Hamza] Implement location validation: + - [ ] For git_repository: validate path exists or URL is valid git URL + - [ ] For filesystem: validate path exists and is directory + - [ ] For database: validate connection string format (basic check) + - [ ] For api_endpoint: validate URL format + - [ ] Commit: "feat(cli): validate resource location by type" + - [ ] **B2.3d** [Hamza] Implement metadata parsing: + - [ ] Parse each `--metadata` value as "key=value" + - [ ] Build dict from all pairs + - [ ] Handle missing "=" gracefully (error) + - [ ] Commit: "feat(cli): parse metadata key=value pairs" + - [ ] **B2.3e** [Hamza] Implement resource creation and linking: + - [ ] Fetch project by namespaced name + - [ ] Raise `ProjectNotFoundError` if not exists + - [ ] Check resource name is unique within project + - [ ] Create Resource with ULID and all fields + - [ ] Add resource to project: `project_service.add_resource(project_id, resource)` + - [ ] Display success: "Added resource '{name}' to project '{project_name}'" + - [ ] Commit: "feat(cli): implement add-resource creation and linking" + - [ ] **B2.3f** [Hamza] Add ProjectService.add_resource() implementation: + - [ ] Fetch project from repository + - [ ] Check for duplicate resource name in project + - [ ] Create new project with resource added (immutable pattern) + - [ ] Recompute project.is_remote based on all resources + - [ ] Update project in repository + - [ ] Create resource in resource repository + - [ ] Return updated project + - [ ] Commit: "feat(service): implement ProjectService.add_resource()" + + - [ ] **B2.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project remove-resource` command: + - [ ] **B2.4a** [Hamza] Define command signature: + + ```python + @project.command("remove-resource") + @click.option("--project", "-p", required=True, help="Project name") + @click.option("--name", "-n", required=True, help="Resource name to remove") + @click.option("--yes", is_flag=True, help="Skip confirmation") + ``` + + - [ ] Commit: "feat(cli): add project remove-resource command signature" + + - [ ] **B2.4b** [Hamza] Implement removal logic: + - [ ] Fetch project and validate resource exists + - [ ] If not --yes, prompt for confirmation: "Remove resource '{name}'? [y/N]" + - [ ] Call `project_service.remove_resource(project_id, resource_name)` + - [ ] Display success: "Removed resource '{name}' from project" + - [ ] Commit: "feat(cli): implement remove-resource logic" + - [ ] **B2.4c** [Hamza] Add ProjectService.remove_resource() implementation: + - [ ] Fetch project + - [ ] Find resource by name + - [ ] Raise `ResourceNotFoundError` if not exists + - [ ] Create new project without resource (immutable pattern) + - [ ] Recompute is_remote + - [ ] Update project + - [ ] Delete resource from resource repository + - [ ] Return updated project + - [ ] Commit: "feat(service): implement ProjectService.remove_resource()" + + - [ ] **B2.5** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project list` command: + - [ ] **B2.5a** [Hamza] Define command signature: + + ```python + @project.command("list") + @click.option("--namespace", "-n", default=None, help="Filter by namespace") + @click.option("--tag", "-t", default=None, help="Filter by tag") + @click.option("--format", "output_format", type=click.Choice(["table", "json"]), default="table") + ``` + + - [ ] Commit: "feat(cli): add project list command signature" + + - [ ] **B2.5b** [Hamza] Implement query and filtering: + - [ ] Call `project_service.list_projects(namespace=namespace, tag=tag)` + - [ ] If namespace provided, filter by namespace + - [ ] If tag provided, filter projects that have this tag + - [ ] Commit: "feat(cli): implement project list filtering" + - [ ] **B2.5c** [Hamza] Implement table output: + - [ ] Create Rich Table with columns: ID, Name, Resources, Tags, Remote + - [ ] Add row for each project: + - [ ] ID: first 8 chars of project_id + - [ ] Name: namespaced_name + - [ ] Resources: count of resources + - [ ] Tags: comma-separated tags (truncate if >3) + - [ ] Remote: "Yes" or "No" based on is_remote + - [ ] Display table via console.print() + - [ ] If no projects found, display "No projects found" + - [ ] Commit: "feat(cli): implement project list table output" + - [ ] **B2.5d** [Hamza] Implement JSON output: + - [ ] If format=json, serialize projects to JSON + - [ ] Use model_dump_json() for each project + - [ ] Print to stdout (for piping to jq, etc.) + - [ ] Commit: "feat(cli): implement project list JSON output" + - [ ] **B2.5e** [Hamza] Add ProjectService.list_projects() implementation: + - [ ] Call `project_repo.list_all()` + - [ ] Apply namespace filter if provided + - [ ] Apply tag filter if provided + - [ ] Return filtered list + - [ ] Commit: "feat(service): implement ProjectService.list_projects()" + + - [ ] **B2.6** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project show` command: + - [ ] **B2.6a** [Hamza] Define command signature: + + ```python + @project.command("show") + @click.argument("name") + @click.option("--format", "output_format", type=click.Choice(["rich", "json"]), default="rich") + ``` + + - [ ] Commit: "feat(cli): add project show command signature" + + - [ ] **B2.6b** [Hamza] Implement project fetch and rich display: + - [ ] Fetch project by namespaced name via service + - [ ] If not found, display error and exit(1) + - [ ] Display project details using Rich panels: + + ``` + ╭─ Project: local/my-project ────────────────────────╮ + │ ID: 01ARZ3NDEKTSV4RRFFQ69G5FAV │ + │ Description: My awesome project │ + │ Tags: python, backend │ + │ Remote: No │ + │ Created: 2024-01-15 10:30:00 │ + ╰───────────────────────────────────────────────────╯ + + Resources (2): + ┌─────────────────┬──────────────────┬─────────────────┬──────────┐ + │ Name │ Type │ Location │ Strategy │ + ├─────────────────┼──────────────────┼─────────────────┼──────────┤ + │ source │ git_repository │ /path/to/repo │ worktree │ + │ config │ filesystem │ /path/to/config │ copy │ + └─────────────────┴──────────────────┴─────────────────┴──────────┘ + + Validation Config: + Test: pytest tests/ + Lint: ruff check src/ + Type Check: pyright src/ + ``` + + - [ ] Commit: "feat(cli): implement project show rich display" + + - [ ] **B2.6c** [Hamza] Implement JSON output: + - [ ] If format=json, output full project as JSON + - [ ] Include all resources and validation config + - [ ] Commit: "feat(cli): implement project show JSON output" + - [ ] **B2.6d** [Hamza] Add ProjectService.get_project() implementation: + - [ ] Parse namespaced name to (namespace, short_name) + - [ ] Query by namespaced_name OR by project_id (support both) + - [ ] Return Project with resources loaded + - [ ] Commit: "feat(service): implement ProjectService.get_project()" + + - [ ] **B2.7** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project set-validation` command: + - [ ] **B2.7a** [Hamza] Define command signature: + + ```python + @project.command("set-validation") + @click.option("--project", "-p", required=True, help="Project name") + @click.option("--test-command", default=None, help="Command to run tests") + @click.option("--lint-command", default=None, help="Command to run linter") + @click.option("--type-check-command", default=None, help="Command for type checking") + @click.option("--build-command", default=None, help="Command to build project") + @click.option("--timeout", default=300, type=int, help="Timeout for each command (seconds)") + @click.option("--clear", is_flag=True, help="Clear all validation config") + ``` + + - [ ] Commit: "feat(cli): add project set-validation command signature" + + - [ ] **B2.7b** [Hamza] Implement validation config update: + - [ ] Fetch project + - [ ] If --clear, set validation_config to None + - [ ] Otherwise, create ValidationConfig with provided commands + - [ ] Only set commands that were explicitly provided (preserve existing if not specified) + - [ ] Call `project_service.update_validation(project_id, config)` + - [ ] Display updated config summary + - [ ] Commit: "feat(cli): implement set-validation logic" + - [ ] **B2.7c** [Hamza] Add ProjectService.update_validation() implementation: + - [ ] Fetch project + - [ ] Merge new config with existing (if not --clear) + - [ ] Update project with new validation_config + - [ ] Persist via repository + - [ ] Commit: "feat(service): implement ProjectService.update_validation()" + + - [ ] **B2.8** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project delete` command: + - [ ] **B2.8a** [Hamza] Define command signature: + + ```python + @project.command("delete") + @click.argument("name") + @click.option("--force", "-f", is_flag=True, help="Force delete even if plans exist") + @click.option("--yes", is_flag=True, help="Skip confirmation") + ``` + + - [ ] Commit: "feat(cli): add project delete command signature" + + - [ ] **B2.8b** [Hamza] Implement deletion checks: + - [ ] Fetch project + - [ ] Check for active plans using this project: `plan_repo.count(project_id=project.project_id)` + - [ ] If plans exist and not --force: + - [ ] Display error: "Cannot delete project with {n} active plans. Use --force to delete anyway." + - [ ] List plan IDs (first 5) + - [ ] Exit with code 1 + - [ ] If --force, display warning: "Deleting project with {n} active plans" + - [ ] Commit: "feat(cli): implement project delete safety checks" + - [ ] **B2.8c** [Hamza] Implement deletion: + - [ ] Prompt for confirmation: "Delete project '{name}'? This cannot be undone. [y/N]" + - [ ] Support `--yes` to bypass confirmation + - [ ] If confirmed (or --yes), call `project_service.delete_project(project_id)` + - [ ] Display success: "Deleted project '{name}'" + - [ ] Commit: "feat(cli): implement project delete confirmation and execution" + - [ ] **B2.8d** [Hamza] Add ProjectService.delete_project() implementation: + - [ ] Delete all resources for project via resource_repo + - [ ] Delete project via project_repo + - [ ] Return True on success + - [ ] Commit: "feat(service): implement ProjectService.delete_project()" + + - [ ] **B2.9** [Hamza] Register project commands in `src/cleveragents/cli/main.py`: + - [ ] **B2.9a** [Hamza] Import and register: + - [ ] Add `from cleveragents.cli.commands.project import project as project_group` + - [ ] Add `app.add_command(project_group)` in main app setup + - [ ] Verify `agents [--data-dir PATH] [--config-path PATH] project --help` shows all subcommands + - [ ] Commit: "feat(cli): register project commands in main CLI" + - [ ] **B2.9b** [Hamza] Add DI wiring for ProjectService: + - [ ] Update container.py to provide ProjectService + - [ ] Inject into CLI commands via Click context or similar pattern + - [ ] Commit: "feat(di): wire ProjectService into CLI" + + - [ ] Tests: Behave + Robot for all project CLI commands + - [ ] **B2.10** [Rui] Write Behave scenarios in `features/project_cli.feature`: + - [ ] **B2.10a** [Rui] Project creation scenarios: + - [ ] Scenario: Create project with valid name succeeds + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/test-project` + - [ ] Then the output contains "Created project: local/test-project" + - [ ] And the output contains "Project ID:" + - [ ] Scenario: Create project with description and tags + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/test --description "My project" --tag python --tag backend` + - [ ] Then the project has description "My project" + - [ ] And the project has tags "python", "backend" + - [ ] Scenario: Create project with duplicate name fails + - [ ] Given a project "local/existing" exists + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name local/existing` + - [ ] Then the exit code is 1 + - [ ] And the output contains "already exists" + - [ ] Scenario: Create project with invalid namespace fails + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project create --name 123invalid/test` + - [ ] Then the exit code is 1 + - [ ] And the output contains "Invalid namespace" + - [ ] Commit: "test(behave): add project create CLI scenarios" + - [ ] **B2.10b** [Rui] Add resource scenarios: + - [ ] Scenario: Add git repository resource to project + - [ ] Given a project "local/test" exists + - [ ] And a git repository exists at "/tmp/test-repo" + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name source --type git_repository --location /tmp/test-repo --sandbox-strategy git_worktree` + - [ ] Then the output contains "Added resource 'source'" + - [ ] Scenario: Add filesystem resource to project + - [ ] Given a project "local/test" exists + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name config --type filesystem --location /tmp/config --sandbox-strategy copy_on_write` + - [ ] Then the output contains "Added resource 'config'" + - [ ] Scenario: Add resource with incompatible sandbox strategy fails + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource --project local/test --name api --type api_endpoint --location https://api.example.com --sandbox-strategy git_worktree` + - [ ] Then the exit code is 1 + - [ ] And the output contains "incompatible" + - [ ] Scenario: Add resource with metadata + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project add-resource ... --metadata branch=main --metadata remote=origin` + - [ ] Then the resource has metadata key "branch" with value "main" + - [ ] Commit: "test(behave): add resource CLI scenarios" + - [ ] **B2.10c** [Rui] Remove resource scenarios: + - [ ] Scenario: Remove resource from project succeeds + - [ ] Given project "local/test" has resource "source" + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project remove-resource --project local/test --name source --yes` + - [ ] Then the output contains "Removed resource 'source'" + - [ ] Scenario: Remove non-existent resource fails gracefully + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project remove-resource --project local/test --name nonexistent --yes` + - [ ] Then the exit code is 1 + - [ ] And the output contains "not found" + - [ ] Commit: "test(behave): add remove-resource CLI scenarios" + - [ ] **B2.10d** [Rui] List and show scenarios: + - [ ] Scenario: List projects shows all projects + - [ ] Given projects "local/proj1" and "local/proj2" exist + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list` + - [ ] Then the output contains "proj1" + - [ ] And the output contains "proj2" + - [ ] Scenario: List projects with namespace filter works + - [ ] Given projects "local/proj1" and "team/proj2" exist + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list --namespace local` + - [ ] Then the output contains "proj1" + - [ ] And the output does not contain "proj2" + - [ ] Scenario: List projects JSON format works + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project list --format json` + - [ ] Then the output is valid JSON + - [ ] Scenario: Show project displays full details + - [ ] Given project "local/test" with 2 resources exists + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project show local/test` + - [ ] Then the output contains "local/test" + - [ ] And the output contains "Resources (2)" + - [ ] Commit: "test(behave): add list and show CLI scenarios" + - [ ] **B2.10e** [Rui] Validation config scenarios: + - [ ] Scenario: Set validation commands persists correctly + - [ ] Given project "local/test" exists + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project set-validation --project local/test --test-command "pytest" --lint-command "ruff check"` + - [ ] Then project "local/test" has test_command "pytest" + - [ ] And project "local/test" has lint_command "ruff check" + - [ ] Scenario: Clear validation config works + - [ ] Given project "local/test" has validation config + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project set-validation --project local/test --clear` + - [ ] Then project "local/test" has no validation config + - [ ] Commit: "test(behave): add validation config CLI scenarios" + - [ ] **B2.10f** [Rui] Delete scenarios: + - [ ] Scenario: Delete project succeeds + - [ ] Given project "local/test" exists with no plans + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --yes` + - [ ] Then the output contains "Deleted project" + - [ ] And project "local/test" no longer exists + - [ ] Scenario: Delete project with active plans blocked without --force + - [ ] Given project "local/test" exists + - [ ] And a plan uses project "local/test" + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --yes` + - [ ] Then the exit code is 1 + - [ ] And the output contains "active plans" + - [ ] Scenario: Delete project with active plans succeeds with --force + - [ ] Given project "local/test" exists with active plans + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] project delete local/test --force --yes` + - [ ] Then the output contains "Deleted project" + - [ ] Commit: "test(behave): add delete CLI scenarios" + - [ ] **B2.11** [Rui] Write Robot integration test `robot/project_cli_integration.robot`: + - [ ] **B2.11a** [Rui] Full lifecycle test: + - [ ] Test: Full project lifecycle + - [ ] Create project with description and tags + - [ ] Add git repository resource + - [ ] Add filesystem resource + - [ ] Show project and verify all details + - [ ] Set validation commands + - [ ] List projects and verify presence + - [ ] Remove one resource + - [ ] Delete project + - [ ] Verify project no longer exists + - [ ] Commit: "test(robot): add full project lifecycle e2e test" + - [ ] **B2.11b** [Rui] Multi-resource test: + - [ ] Test: Project with multiple resources of different types + - [ ] Create project + - [ ] Add git repo resource (primary code) + - [ ] Add filesystem resource (docs) + - [ ] Add database resource (read-only) + - [ ] Verify is_remote is computed correctly (should be False - has local resources) + - [ ] Show project and verify all resources listed + - [ ] Commit: "test(robot): add multi-resource project e2e test" - [ ] **Stage B3: Sandbox Framework** (Day 3-5) **[Luis + Hamza - Architectural, CRITICAL PATH]** - **PARALLEL SUBTRACKS**: - - TRACK B3.protocol [Luis - Day 3 AM]: Protocol + Status + Factory (B3.1, B3.2, B3.5, B3.6) - - TRACK B3.git [Hamza - Day 3 PM - Day 4]: Git Worktree Implementation (B3.3) - - TRACK B3.fs [Hamza - Day 4]: Filesystem Implementation (B3.4) - - TRACK B3.manager [Luis - Day 4-5]: Manager + Merge (B3.7, B3.8) - - TRACK B3.tests [Rui - Day 3-5]: Tests in parallel with implementation + **PARALLEL SUBTRACKS**: + - TRACK B3.protocol [Luis - Day 3 AM]: Protocol + Status + Factory (B3.1, B3.2, B3.5, B3.6) + - TRACK B3.git [Hamza - Day 3 PM - Day 4]: Git Worktree Implementation (B3.3) + - TRACK B3.fs [Hamza - Day 4]: Filesystem Implementation (B3.4) + - TRACK B3.manager [Luis - Day 4-5]: Manager + Merge (B3.7, B3.8) + - TRACK B3.tests [Rui - Day 3-5]: Tests in parallel with implementation - - [ ] Code: Implement sandbox abstraction - - [ ] **B3.1** [Luis] Define `Sandbox` protocol in `src/cleveragents/infrastructure/sandbox/protocol.py`: - - [ ] **B3.1a** [Luis] Create file with necessary imports: - - [ ] Import `Protocol, runtime_checkable` from typing - - [ ] Import `ABC, abstractmethod` from abc - - [ ] Import `dataclasses` for result types - - [ ] Add module docstring explaining sandbox abstraction purpose - - [ ] Commit: "feat(sandbox): create protocol.py with imports" - - [ ] **B3.1b** [Luis] Define `SandboxContext` dataclass: - - [ ] `sandbox_id: str` - Unique identifier (ULID) for this sandbox instance - - [ ] `sandbox_path: str` - Root path where sandboxed files live - - [ ] `original_path: str` - Original resource location - - [ ] `resource_id: str` - ID of resource being sandboxed - - [ ] `plan_id: str` - ID of plan that created this sandbox - - [ ] `created_at: datetime` - When sandbox was created - - [ ] `metadata: dict[str, Any]` - Implementation-specific data (e.g., git branch name) - - [ ] Commit: "feat(sandbox): define SandboxContext dataclass" - - [ ] **B3.1c** [Luis] Define `CommitResult` dataclass: - - [ ] `sandbox_id: str` - Which sandbox was committed - - [ ] `success: bool` - Whether commit succeeded - - [ ] `commit_ref: str | None` - Git commit hash or equivalent reference - - [ ] `changed_files: list[str]` - List of files that were changed - - [ ] `added_files: list[str]` - List of files that were created - - [ ] `deleted_files: list[str]` - List of files that were removed - - [ ] `error: str | None` - Error message if success=False - - [ ] `timestamp: datetime` - When commit occurred - - [ ] Commit: "feat(sandbox): define CommitResult dataclass" - - [ ] **B3.1d** [Luis] Define `Sandbox` protocol: - ```python - @runtime_checkable - class Sandbox(Protocol): - """Protocol for resource sandboxing implementations.""" + - [ ] Code: Implement sandbox abstraction + - [ ] **B3.1** [Luis] Define `Sandbox` protocol in `src/cleveragents/infrastructure/sandbox/protocol.py`: + - [ ] **B3.1a** [Luis] Create file with necessary imports: + - [ ] Import `Protocol, runtime_checkable` from typing + - [ ] Import `ABC, abstractmethod` from abc + - [ ] Import `dataclasses` for result types + - [ ] Add module docstring explaining sandbox abstraction purpose + - [ ] Commit: "feat(sandbox): create protocol.py with imports" + - [ ] **B3.1b** [Luis] Define `SandboxContext` dataclass: + - [ ] `sandbox_id: str` - Unique identifier (ULID) for this sandbox instance + - [ ] `sandbox_path: str` - Root path where sandboxed files live + - [ ] `original_path: str` - Original resource location + - [ ] `resource_id: str` - ID of resource being sandboxed + - [ ] `plan_id: str` - ID of plan that created this sandbox + - [ ] `created_at: datetime` - When sandbox was created + - [ ] `metadata: dict[str, Any]` - Implementation-specific data (e.g., git branch name) + - [ ] Commit: "feat(sandbox): define SandboxContext dataclass" + - [ ] **B3.1c** [Luis] Define `CommitResult` dataclass: + - [ ] `sandbox_id: str` - Which sandbox was committed + - [ ] `success: bool` - Whether commit succeeded + - [ ] `commit_ref: str | None` - Git commit hash or equivalent reference + - [ ] `changed_files: list[str]` - List of files that were changed + - [ ] `added_files: list[str]` - List of files that were created + - [ ] `deleted_files: list[str]` - List of files that were removed + - [ ] `error: str | None` - Error message if success=False + - [ ] `timestamp: datetime` - When commit occurred + - [ ] Commit: "feat(sandbox): define CommitResult dataclass" + - [ ] **B3.1d** [Luis] Define `Sandbox` protocol: - @property - def sandbox_id(self) -> str: - """Unique identifier for this sandbox.""" - ... + ```python + @runtime_checkable + class Sandbox(Protocol): + """Protocol for resource sandboxing implementations.""" - @property - def resource(self) -> Resource: - """The resource being sandboxed.""" - ... + @property + def sandbox_id(self) -> str: + """Unique identifier for this sandbox.""" + ... - @property - def status(self) -> SandboxStatus: - """Current status of the sandbox.""" - ... + @property + def resource(self) -> Resource: + """The resource being sandboxed.""" + ... - @property - def context(self) -> SandboxContext | None: - """Context after sandbox is created, None before.""" - ... + @property + def status(self) -> SandboxStatus: + """Current status of the sandbox.""" + ... - def create(self, plan_id: str) -> SandboxContext: - """Initialize sandbox environment. Returns context with paths.""" - ... + @property + def context(self) -> SandboxContext | None: + """Context after sandbox is created, None before.""" + ... - def get_path(self, resource_path: str) -> str: - """Translate resource-relative path to sandbox absolute path.""" - ... + def create(self, plan_id: str) -> SandboxContext: + """Initialize sandbox environment. Returns context with paths.""" + ... - def commit(self, message: str | None = None) -> CommitResult: - """Finalize sandbox changes. Returns result with changed files.""" - ... + def get_path(self, resource_path: str) -> str: + """Translate resource-relative path to sandbox absolute path.""" + ... - def rollback(self) -> None: - """Discard all sandbox changes. Sandbox can still be used.""" - ... + def commit(self, message: str | None = None) -> CommitResult: + """Finalize sandbox changes. Returns result with changed files.""" + ... - def cleanup(self) -> None: - """Remove sandbox artifacts. Sandbox cannot be used after this.""" - ... - ``` - - [ ] Commit: "feat(sandbox): define Sandbox protocol" - - [ ] **B3.2** [Luis] Define `SandboxStatus` enum in same file: - - [ ] **B3.2a** [Luis] Define status values with docstrings: - - [ ] `PENDING = "pending"` - Sandbox created but not yet initialized - - [ ] `CREATED = "created"` - Sandbox initialized, ready for use - - [ ] `ACTIVE = "active"` - Sandbox has been written to - - [ ] `COMMITTED = "committed"` - Changes have been applied to original - - [ ] `ROLLED_BACK = "rolled_back"` - Changes have been discarded - - [ ] `CLEANED_UP = "cleaned_up"` - Sandbox artifacts removed, terminal state - - [ ] `ERRORED = "errored"` - Sandbox operation failed - - [ ] Commit: "feat(sandbox): define SandboxStatus enum" - - [ ] **B3.2b** [Luis] Add status transition validation: - - [ ] `@classmethod def valid_transitions(cls) -> dict[SandboxStatus, list[SandboxStatus]]:` - Define allowed transitions - - [ ] PENDING → CREATED, ERRORED - - [ ] CREATED → ACTIVE, COMMITTED (no changes), CLEANED_UP - - [ ] ACTIVE → COMMITTED, ROLLED_BACK, ERRORED - - [ ] COMMITTED → CLEANED_UP - - [ ] ROLLED_BACK → ACTIVE (retry), CLEANED_UP - - [ ] ERRORED → CLEANED_UP - - [ ] CLEANED_UP → (terminal, no transitions) - - [ ] Commit: "feat(sandbox): add SandboxStatus transition validation" - - [ ] **B3.3** [Hamza] Implement `GitWorktreeSandbox` in `src/cleveragents/infrastructure/sandbox/git_worktree.py`: - - [ ] **B3.3a** [Hamza] Create class scaffold with constructor: - - [ ] Import subprocess for git commands - - [ ] Import logging, tempfile, shutil, os - - [ ] Import protocol types from protocol.py - - [ ] Define `class GitWorktreeSandbox:` implementing Sandbox protocol - - [ ] Constructor: `__init__(self, resource: Resource)`: - - [ ] Validate `resource.type == ResourceType.GIT_REPOSITORY` - - [ ] Validate `resource.sandbox_strategy == SandboxStrategy.GIT_WORKTREE` - - [ ] Store resource reference - - [ ] Initialize `_sandbox_id = ulid.new().str` - - [ ] Initialize `_status = SandboxStatus.PENDING` - - [ ] Initialize `_context: SandboxContext | None = None` - - [ ] Initialize `_worktree_path: str | None = None` - - [ ] Initialize `_branch_name: str | None = None` - - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox class scaffold" - - [ ] **B3.3b** [Hamza] Implement `create(plan_id: str) -> SandboxContext`: - - [ ] Validate status is PENDING - - [ ] Generate unique branch name: `f"cleveragents-sandbox-{self._sandbox_id}"` - - [ ] Generate worktree path: `tempfile.mkdtemp(prefix=f"ca_git_worktree_{plan_id}_")` - - [ ] Determine repo root from resource.location (handle both path and URL) - - [ ] If resource is remote URL, first clone to temp location - - [ ] Run git command: `git worktree add {worktree_path} -b {branch_name}` - - [ ] Handle errors: if worktree fails, cleanup and raise - - [ ] Store paths in instance variables - - [ ] Create and store SandboxContext - - [ ] Update status to CREATED - - [ ] Log: "Created git worktree sandbox {sandbox_id} at {worktree_path}" - - [ ] Return context - - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.create()" - - [ ] **B3.3c** [Hamza] Implement helper method for git commands: - - [ ] `def _run_git(self, args: list[str], cwd: str | None = None) -> subprocess.CompletedProcess`: - - [ ] Build full command: `["git"] + args` - - [ ] Run with `subprocess.run(capture_output=True, text=True, check=False)` - - [ ] Log command and output at DEBUG level - - [ ] If returncode != 0, log error at WARNING level - - [ ] Return CompletedProcess for caller to handle - - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox._run_git() helper" - - [ ] **B3.3d** [Hamza] Implement `get_path(resource_path: str) -> str`: - - [ ] Validate sandbox is CREATED or ACTIVE - - [ ] Validate resource_path does not escape sandbox (no `..` traversal) - - [ ] Return `os.path.join(self._worktree_path, resource_path)` - - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.get_path()" - - [ ] **B3.3e** [Hamza] Implement `commit(message: str | None = None) -> CommitResult`: - - [ ] Validate status is CREATED or ACTIVE - - [ ] Default message: `f"CleverAgents sandbox commit [{self._sandbox_id}]"` - - [ ] Run `git status --porcelain` to check for changes - - [ ] If no changes, return CommitResult(success=True, changed_files=[]) - - [ ] Run `git add -A` to stage all changes - - [ ] Parse `git diff --cached --name-status` to get changed/added/deleted lists - - [ ] Run `git commit -m "{message}"` to commit - - [ ] Get commit hash with `git rev-parse HEAD` - - [ ] Update status to COMMITTED - - [ ] Return CommitResult with all fields populated - - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.commit()" - - [ ] **B3.3f** [Hamza] Implement `rollback() -> None`: - - [ ] Validate status is CREATED or ACTIVE - - [ ] Run `git checkout .` to discard modified files - - [ ] Run `git clean -fd` to remove untracked files - - [ ] Run `git reset HEAD` to unstage any staged changes - - [ ] Update status to ROLLED_BACK - - [ ] Log: "Rolled back git worktree sandbox {sandbox_id}" - - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.rollback()" - - [ ] **B3.3g** [Hamza] Implement `cleanup() -> None`: - - [ ] Can be called from any non-terminal status - - [ ] Run `git worktree remove {worktree_path} --force` from repo root - - [ ] If worktree remove fails, try `shutil.rmtree(self._worktree_path)` as fallback - - [ ] Optionally delete the sandbox branch: `git branch -D {branch_name}` - - [ ] Update status to CLEANED_UP - - [ ] Clear context and paths - - [ ] Log: "Cleaned up git worktree sandbox {sandbox_id}" - - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.cleanup()" - - [ ] **B3.3h** [Hamza] Add proper error handling: - - [ ] Define `SandboxError` base exception in protocol.py - - [ ] Define `SandboxCreationError(SandboxError)` - failed to create - - [ ] Define `SandboxCommitError(SandboxError)` - failed to commit - - [ ] Define `SandboxRollbackError(SandboxError)` - failed to rollback - - [ ] All methods should catch subprocess errors and wrap in SandboxError - - [ ] On error, update status to ERRORED and include original exception - - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox error handling" - - [ ] **B3.4** [Hamza] Implement `FilesystemSandbox` in `src/cleveragents/infrastructure/sandbox/filesystem.py`: - - [ ] **B3.4a** [Hamza] Create class scaffold: - - [ ] Similar structure to GitWorktreeSandbox - - [ ] Constructor validates `resource.type == ResourceType.FILESYSTEM` - - [ ] Constructor validates `resource.sandbox_strategy == SandboxStrategy.COPY_ON_WRITE` - - [ ] Commit: "feat(sandbox): add FilesystemSandbox class scaffold" - - [ ] **B3.4b** [Hamza] Implement `create(plan_id: str) -> SandboxContext`: - - [ ] Generate sandbox path: `tempfile.mkdtemp(prefix=f"ca_fs_sandbox_{plan_id}_")` - - [ ] Copy resource directory to sandbox: `shutil.copytree(resource.location, sandbox_path, dirs_exist_ok=True)` - - [ ] Use `shutil.ignore_patterns()` to skip .git, node_modules, __pycache__ - - [ ] Record file hashes of original for diff detection later - - [ ] Create and store SandboxContext - - [ ] Update status to CREATED - - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.create()" - - [ ] **B3.4c** [Hamza] Implement `get_path(resource_path: str) -> str`: - - [ ] Same pattern as GitWorktreeSandbox - - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.get_path()" - - [ ] **B3.4d** [Hamza] Implement `commit(message: str | None = None) -> CommitResult`: - - [ ] Compare sandbox files with original (using recorded hashes) - - [ ] Build lists of changed/added/deleted files - - [ ] For each changed file: `shutil.copy2(sandbox_file, original_file)` - - [ ] For each new file: copy and create parent dirs as needed - - [ ] For each deleted file: `os.remove(original_file)` - - [ ] Use atomic operations where possible (write to .tmp then rename) - - [ ] Update status to COMMITTED - - [ ] Return CommitResult with file lists - - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.commit()" - - [ ] **B3.4e** [Hamza] Implement `rollback() -> None`: - - [ ] For filesystem, rollback is implicit - just don't commit - - [ ] Re-copy original to sandbox to reset: `shutil.rmtree(sandbox); shutil.copytree(original, sandbox)` - - [ ] Update status to ROLLED_BACK - - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.rollback()" - - [ ] **B3.4f** [Hamza] Implement `cleanup() -> None`: - - [ ] Remove sandbox directory: `shutil.rmtree(self._sandbox_path, ignore_errors=True)` - - [ ] Update status to CLEANED_UP - - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.cleanup()" - - [ ] **B3.5** [Luis] Implement `NoSandbox` in `src/cleveragents/infrastructure/sandbox/no_sandbox.py`: - - [ ] **B3.5a** [Luis] Create class for non-sandboxable resources: - - [ ] For APIs, some cloud resources, etc. - - [ ] Constructor: accept any Resource with sandbox_strategy=NONE - - [ ] Commit: "feat(sandbox): add NoSandbox class scaffold" - - [ ] **B3.5b** [Luis] Implement all methods as passthrough or warnings: - - [ ] `create()`: Log WARNING "Resource {name} is not sandboxed - changes are immediate" - - [ ] `get_path(resource_path)`: Return original resource path unchanged - - [ ] `commit()`: Return CommitResult(success=True) - changes already applied - - [ ] `rollback()`: Log ERROR "Rollback not possible for non-sandboxed resource" - - [ ] `cleanup()`: No-op, status to CLEANED_UP - - [ ] Commit: "feat(sandbox): implement NoSandbox methods" - - [ ] **B3.6** [Luis] Implement `SandboxFactory` in `src/cleveragents/infrastructure/sandbox/factory.py`: - - [ ] **B3.6a** [Luis] Create factory class: - - [ ] Import all sandbox implementations - - [ ] Define `class SandboxFactory:` - - [ ] Commit: "feat(sandbox): add SandboxFactory scaffold" - - [ ] **B3.6b** [Luis] Implement `create_sandbox(resource: Resource) -> Sandbox`: - - [ ] Match on `resource.sandbox_strategy`: - ```python - match resource.sandbox_strategy: - case SandboxStrategy.GIT_WORKTREE: - return GitWorktreeSandbox(resource) - case SandboxStrategy.COPY_ON_WRITE: - return FilesystemSandbox(resource) - case SandboxStrategy.OVERLAY: - # Overlay not implemented yet, fall back to copy - logger.warning("Overlay not implemented, using copy-on-write") - return FilesystemSandbox(resource) - case SandboxStrategy.TRANSACTION_ROLLBACK: - raise NotImplementedError("Database sandboxing not yet implemented") - case SandboxStrategy.NONE: - return NoSandbox(resource) - case _: - raise ValueError(f"Unknown sandbox strategy: {resource.sandbox_strategy}") - ``` - - [ ] Commit: "feat(sandbox): implement SandboxFactory.create_sandbox()" - - [ ] **B3.6c** [Luis] Add validation helper: - - [ ] `@staticmethod def is_supported(resource: Resource) -> bool:` - Check if sandboxing is supported - - [ ] `@staticmethod def get_supported_strategies(resource_type: ResourceType) -> list[SandboxStrategy]:` - Valid combos - - [ ] Commit: "feat(sandbox): add SandboxFactory validation helpers" - - [ ] **B3.7** [Luis] Implement sandbox lifecycle management: - - [ ] **B3.7a** [Luis] Create `SandboxManager` in `src/cleveragents/infrastructure/sandbox/manager.py`: - - [ ] Import threading for lock management - - [ ] Import factory and protocol types - - [ ] Define `class SandboxManager:` - - [ ] Instance variables: - - [ ] `_factory: SandboxFactory` - injected via constructor - - [ ] `_active_sandboxes: dict[str, dict[str, Sandbox]]` - plan_id -> resource_id -> Sandbox - - [ ] `_lock: threading.RLock` - thread safety for sandbox tracking - - [ ] `_cleanup_on_exit: bool` - whether to cleanup on process exit (default True) - - [ ] Commit: "feat(sandbox): add SandboxManager scaffold" - - [ ] **B3.7b** [Luis] Implement `get_or_create_sandbox(plan_id: str, resource: Resource) -> Sandbox`: - - [ ] Acquire lock - - [ ] Check if sandbox already exists for this plan+resource - - [ ] If exists and status is usable (CREATED, ACTIVE, ROLLED_BACK), return it - - [ ] If exists but cleaned up, remove from tracking - - [ ] Create new sandbox via factory - - [ ] Call sandbox.create(plan_id) to initialize - - [ ] Store in _active_sandboxes - - [ ] Release lock - - [ ] Return sandbox - - [ ] This is the LAZY sandboxing pattern - only create when needed - - [ ] Commit: "feat(sandbox): implement SandboxManager.get_or_create_sandbox()" - - [ ] **B3.7c** [Luis] Implement `commit_all(plan_id: str) -> list[CommitResult]`: - - [ ] Get all sandboxes for plan_id - - [ ] For each sandbox with status ACTIVE: - - [ ] Call sandbox.commit() - - [ ] Collect CommitResult - - [ ] Return list of all results - - [ ] If any commit fails, don't rollback others (partial commit possible, caller decides) - - [ ] Commit: "feat(sandbox): implement SandboxManager.commit_all()" - - [ ] **B3.7d** [Luis] Implement `rollback_all(plan_id: str) -> None`: - - [ ] Get all sandboxes for plan_id - - [ ] For each sandbox with status ACTIVE: - - [ ] Call sandbox.rollback() - - [ ] Log any rollback failures but continue with others - - [ ] Commit: "feat(sandbox): implement SandboxManager.rollback_all()" - - [ ] **B3.7e** [Luis] Implement `cleanup_all(plan_id: str) -> None`: - - [ ] Get all sandboxes for plan_id - - [ ] For each sandbox: - - [ ] Call sandbox.cleanup() - - [ ] Remove plan_id entry from _active_sandboxes - - [ ] Commit: "feat(sandbox): implement SandboxManager.cleanup_all()" - - [ ] **B3.7f** [Luis] Implement `cleanup_abandoned() -> int`: - - [ ] Find sandbox directories matching pattern that aren't tracked - - [ ] Check if creating process is still alive (via PID file or lock) - - [ ] If process dead, clean up the directory - - [ ] Return count of cleaned sandboxes - - [ ] This is called on application startup - - [ ] Commit: "feat(sandbox): implement SandboxManager.cleanup_abandoned()" - - [ ] **B3.7g** [Luis] Add atexit handler for graceful cleanup: - - [ ] Register `atexit.register(self._cleanup_on_exit_handler)` - - [ ] Handler calls cleanup_all for all tracked plans - - [ ] Commit: "feat(sandbox): add SandboxManager atexit cleanup" - - [ ] **B3.8** [Luis] Implement merge strategies in `src/cleveragents/infrastructure/sandbox/merge.py`: - - [ ] **B3.8a** [Luis] Define merge types and protocol: - - [ ] Define `MergeResult` dataclass: - - [ ] `success: bool` - - [ ] `content: str | bytes` - merged content - - [ ] `has_conflicts: bool` - - [ ] `conflict_markers: list[tuple[int, int]]` - line ranges with conflicts - - [ ] Define `MergeStrategy(Protocol)`: - - [ ] Method `merge(base: str, ours: str, theirs: str) -> MergeResult` - - [ ] Commit: "feat(sandbox): define merge types and protocol" - - [ ] **B3.8b** [Luis] Implement `GitMergeStrategy`: - - [ ] Use `git merge-file` for three-way merge - - [ ] Write base, ours, theirs to temp files - - [ ] Run `git merge-file -p ours base theirs` - - [ ] Parse output for conflict markers (`<<<<<<<`, `=======`, `>>>>>>>`) - - [ ] Return MergeResult with content and conflict info - - [ ] Commit: "feat(sandbox): implement GitMergeStrategy" - - [ ] **B3.8c** [Luis] Implement `SequentialMergeStrategy`: - - [ ] For non-mergeable resources, apply changes in order - - [ ] "Theirs" (second change) always wins - - [ ] Return MergeResult(success=True, content=theirs) - - [ ] Commit: "feat(sandbox): implement SequentialMergeStrategy" - - [ ] **B3.8d** [Luis] Implement `JsonMergeStrategy`: - - [ ] Parse both as JSON - - [ ] Deep merge objects (recursive dict merge) - - [ ] Arrays: concatenate or last-wins based on config - - [ ] Return serialized merged JSON - - [ ] Commit: "feat(sandbox): implement JsonMergeStrategy" - - [ ] Tests: Integration tests for each sandbox type - - [ ] **B3.9** [Rui] Write Behave scenarios in `features/sandbox_git_worktree.feature`: - - [ ] **B3.9a** [Rui] Creation scenarios: - - [ ] Scenario: Create git worktree sandbox from local git repo - - [ ] Given a git repository at "/tmp/test-repo" with files - - [ ] When I create a GitWorktreeSandbox for that resource - - [ ] And I call sandbox.create("plan-123") - - [ ] Then a worktree exists at the sandbox path - - [ ] And the sandbox status is CREATED - - [ ] And the original repo is unchanged - - [ ] Scenario: Create sandbox from remote git URL (if applicable) - - [ ] Scenario: Create fails for non-git resource type - - [ ] Commit: "test(behave): add git worktree creation scenarios" - - [ ] **B3.9b** [Rui] Modification isolation scenarios: - - [ ] Scenario: Modify file in sandbox does not affect original - - [ ] Given a created git worktree sandbox - - [ ] When I write "modified content" to sandbox path "test.py" - - [ ] Then the original repo's "test.py" is unchanged - - [ ] And the sandbox status is ACTIVE - - [ ] Scenario: Create new file in sandbox does not appear in original - - [ ] Scenario: Delete file in sandbox does not delete original - - [ ] Commit: "test(behave): add git worktree isolation scenarios" - - [ ] **B3.9c** [Rui] Commit scenarios: - - [ ] Scenario: Commit sandbox creates git commit - - [ ] Given a sandbox with modified files - - [ ] When I call sandbox.commit("Test commit") - - [ ] Then a git commit exists with message "Test commit" - - [ ] And CommitResult.changed_files contains "test.py" - - [ ] And sandbox status is COMMITTED - - [ ] Scenario: Commit with no changes succeeds with empty file list - - [ ] Scenario: Commit includes all staged and unstaged changes - - [ ] Commit: "test(behave): add git worktree commit scenarios" - - [ ] **B3.9d** [Rui] Rollback and cleanup scenarios: - - [ ] Scenario: Rollback sandbox discards all changes - - [ ] Given a sandbox with modified files - - [ ] When I call sandbox.rollback() - - [ ] Then the sandbox files match original - - [ ] And sandbox status is ROLLED_BACK - - [ ] Scenario: Cleanup removes worktree and branch - - [ ] Scenario: Multiple sandboxes from same repo are isolated - - [ ] Commit: "test(behave): add git worktree rollback/cleanup scenarios" - - [ ] **B3.10** [Rui] Write Behave scenarios in `features/sandbox_filesystem.feature`: - - [ ] Similar structure to B3.9 but for filesystem sandbox - - [ ] Scenario: Create filesystem sandbox copies directory - - [ ] Scenario: Large directory copy uses efficient patterns - - [ ] Scenario: Ignore patterns (node_modules, .git) are skipped during copy - - [ ] Scenario: Modify file in sandbox does not affect original - - [ ] Scenario: Commit sandbox applies changes to original atomically - - [ ] Scenario: Rollback sandbox resets to original state - - [ ] Scenario: Cleanup removes temp directory - - [ ] Commit: "test(behave): add filesystem sandbox scenarios" - - [ ] **B3.11** [Rui] Write Robot integration test `robot/sandbox_integration.robot`: - - [ ] **B3.11a** [Rui] Full lifecycle test: - - [ ] Create real git repo with multiple files - - [ ] Create sandbox, modify files, commit - - [ ] Verify changes appear in repo - - [ ] Create second sandbox, rollback, verify no changes - - [ ] Cleanup, verify temp directories removed - - [ ] Commit: "test(robot): add full sandbox lifecycle e2e test" - - [ ] **B3.11b** [Rui] Parallel sandbox test: - - [ ] Create two sandboxes on same repo - - [ ] Modify different files in each - - [ ] Commit both - - [ ] Verify no cross-contamination - - [ ] Commit: "test(robot): add parallel sandbox isolation e2e test" - - [ ] Tests: Parallel execution isolation tests - - [ ] **B3.12** [Rui] Write Behave scenarios in `features/sandbox_isolation.feature`: - - [ ] Scenario: Two plans with sandboxes on same resource are isolated - - [ ] Given Plan A creates sandbox for resource R - - [ ] And Plan B creates sandbox for same resource R - - [ ] When Plan A writes "A content" to file.txt - - [ ] And Plan B writes "B content" to file.txt - - [ ] Then Plan A's sandbox shows "A content" - - [ ] And Plan B's sandbox shows "B content" - - [ ] And original file is unchanged - - [ ] Scenario: Plan A cannot see Plan B's intermediate changes - - [ ] Scenario: Commits from different plans require merge - - [ ] Commit: "test(behave): add sandbox isolation scenarios" - - [ ] Tests: Merge conflict resolution tests - - [ ] **B3.13** [Rui] Write Behave scenarios in `features/sandbox_merge.feature`: - - [ ] Scenario: Git merge strategy handles non-conflicting changes - - [ ] Given base content "line1\nline2\nline3" - - [ ] And ours changes line1 to "modified1" - - [ ] And theirs changes line3 to "modified3" - - [ ] When I merge with GitMergeStrategy - - [ ] Then result is "modified1\nline2\nmodified3" - - [ ] And has_conflicts is False - - [ ] Scenario: Git merge strategy marks conflicts appropriately - - [ ] Given both ours and theirs change line2 - - [ ] When I merge - - [ ] Then has_conflicts is True - - [ ] And content contains conflict markers - - [ ] Scenario: Sequential merge uses theirs content - - [ ] Scenario: JSON merge combines object properties - - [ ] Commit: "test(behave): add merge strategy scenarios" + def rollback(self) -> None: + """Discard all sandbox changes. Sandbox can still be used.""" + ... + + def cleanup(self) -> None: + """Remove sandbox artifacts. Sandbox cannot be used after this.""" + ... + ``` + + - [ ] Commit: "feat(sandbox): define Sandbox protocol" + + - [ ] **B3.2** [Luis] Define `SandboxStatus` enum in same file: + - [ ] **B3.2a** [Luis] Define status values with docstrings: + - [ ] `PENDING = "pending"` - Sandbox created but not yet initialized + - [ ] `CREATED = "created"` - Sandbox initialized, ready for use + - [ ] `ACTIVE = "active"` - Sandbox has been written to + - [ ] `COMMITTED = "committed"` - Changes have been applied to original + - [ ] `ROLLED_BACK = "rolled_back"` - Changes have been discarded + - [ ] `CLEANED_UP = "cleaned_up"` - Sandbox artifacts removed, terminal state + - [ ] `ERRORED = "errored"` - Sandbox operation failed + - [ ] Commit: "feat(sandbox): define SandboxStatus enum" + - [ ] **B3.2b** [Luis] Add status transition validation: + - [ ] `@classmethod def valid_transitions(cls) -> dict[SandboxStatus, list[SandboxStatus]]:` - Define allowed transitions + - [ ] PENDING → CREATED, ERRORED + - [ ] CREATED → ACTIVE, COMMITTED (no changes), CLEANED_UP + - [ ] ACTIVE → COMMITTED, ROLLED_BACK, ERRORED + - [ ] COMMITTED → CLEANED_UP + - [ ] ROLLED_BACK → ACTIVE (retry), CLEANED_UP + - [ ] ERRORED → CLEANED_UP + - [ ] CLEANED_UP → (terminal, no transitions) + - [ ] Commit: "feat(sandbox): add SandboxStatus transition validation" + - [ ] **B3.3** [Hamza] Implement `GitWorktreeSandbox` in `src/cleveragents/infrastructure/sandbox/git_worktree.py`: + - [ ] **B3.3a** [Hamza] Create class scaffold with constructor: + - [ ] Import subprocess for git commands + - [ ] Import logging, tempfile, shutil, os + - [ ] Import protocol types from protocol.py + - [ ] Define `class GitWorktreeSandbox:` implementing Sandbox protocol + - [ ] Constructor: `__init__(self, resource: Resource)`: + - [ ] Validate `resource.type == ResourceType.GIT_REPOSITORY` + - [ ] Validate `resource.sandbox_strategy == SandboxStrategy.GIT_WORKTREE` + - [ ] Store resource reference + - [ ] Initialize `_sandbox_id = ulid.new().str` + - [ ] Initialize `_status = SandboxStatus.PENDING` + - [ ] Initialize `_context: SandboxContext | None = None` + - [ ] Initialize `_worktree_path: str | None = None` + - [ ] Initialize `_branch_name: str | None = None` + - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox class scaffold" + - [ ] **B3.3b** [Hamza] Implement `create(plan_id: str) -> SandboxContext`: + - [ ] Validate status is PENDING + - [ ] Generate unique branch name: `f"cleveragents-sandbox-{self._sandbox_id}"` + - [ ] Generate worktree path: `tempfile.mkdtemp(prefix=f"ca_git_worktree_{plan_id}_")` + - [ ] Determine repo root from resource.location (handle both path and URL) + - [ ] If resource is remote URL, first clone to temp location + - [ ] Run git command: `git worktree add {worktree_path} -b {branch_name}` + - [ ] Handle errors: if worktree fails, cleanup and raise + - [ ] Store paths in instance variables + - [ ] Create and store SandboxContext + - [ ] Update status to CREATED + - [ ] Log: "Created git worktree sandbox {sandbox_id} at {worktree_path}" + - [ ] Return context + - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.create()" + - [ ] **B3.3c** [Hamza] Implement helper method for git commands: + - [ ] `def _run_git(self, args: list[str], cwd: str | None = None) -> subprocess.CompletedProcess`: + - [ ] Build full command: `["git"] + args` + - [ ] Run with `subprocess.run(capture_output=True, text=True, check=False)` + - [ ] Log command and output at DEBUG level + - [ ] If returncode != 0, log error at WARNING level + - [ ] Return CompletedProcess for caller to handle + - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox.\_run_git() helper" + - [ ] **B3.3d** [Hamza] Implement `get_path(resource_path: str) -> str`: + - [ ] Validate sandbox is CREATED or ACTIVE + - [ ] Validate resource_path does not escape sandbox (no `..` traversal) + - [ ] Return `os.path.join(self._worktree_path, resource_path)` + - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.get_path()" + - [ ] **B3.3e** [Hamza] Implement `commit(message: str | None = None) -> CommitResult`: + - [ ] Validate status is CREATED or ACTIVE + - [ ] Default message: `f"CleverAgents sandbox commit [{self._sandbox_id}]"` + - [ ] Run `git status --porcelain` to check for changes + - [ ] If no changes, return CommitResult(success=True, changed_files=[]) + - [ ] Run `git add -A` to stage all changes + - [ ] Parse `git diff --cached --name-status` to get changed/added/deleted lists + - [ ] Run `git commit -m "{message}"` to commit + - [ ] Get commit hash with `git rev-parse HEAD` + - [ ] Update status to COMMITTED + - [ ] Return CommitResult with all fields populated + - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.commit()" + - [ ] **B3.3f** [Hamza] Implement `rollback() -> None`: + - [ ] Validate status is CREATED or ACTIVE + - [ ] Run `git checkout .` to discard modified files + - [ ] Run `git clean -fd` to remove untracked files + - [ ] Run `git reset HEAD` to unstage any staged changes + - [ ] Update status to ROLLED_BACK + - [ ] Log: "Rolled back git worktree sandbox {sandbox_id}" + - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.rollback()" + - [ ] **B3.3g** [Hamza] Implement `cleanup() -> None`: + - [ ] Can be called from any non-terminal status + - [ ] Run `git worktree remove {worktree_path} --force` from repo root + - [ ] If worktree remove fails, try `shutil.rmtree(self._worktree_path)` as fallback + - [ ] Optionally delete the sandbox branch: `git branch -D {branch_name}` + - [ ] Update status to CLEANED_UP + - [ ] Clear context and paths + - [ ] Log: "Cleaned up git worktree sandbox {sandbox_id}" + - [ ] Commit: "feat(sandbox): implement GitWorktreeSandbox.cleanup()" + - [ ] **B3.3h** [Hamza] Add proper error handling: + - [ ] Define `SandboxError` base exception in protocol.py + - [ ] Define `SandboxCreationError(SandboxError)` - failed to create + - [ ] Define `SandboxCommitError(SandboxError)` - failed to commit + - [ ] Define `SandboxRollbackError(SandboxError)` - failed to rollback + - [ ] All methods should catch subprocess errors and wrap in SandboxError + - [ ] On error, update status to ERRORED and include original exception + - [ ] Commit: "feat(sandbox): add GitWorktreeSandbox error handling" + - [ ] **B3.4** [Hamza] Implement `FilesystemSandbox` in `src/cleveragents/infrastructure/sandbox/filesystem.py`: + - [ ] **B3.4a** [Hamza] Create class scaffold: + - [ ] Similar structure to GitWorktreeSandbox + - [ ] Constructor validates `resource.type == ResourceType.FILESYSTEM` + - [ ] Constructor validates `resource.sandbox_strategy == SandboxStrategy.COPY_ON_WRITE` + - [ ] Commit: "feat(sandbox): add FilesystemSandbox class scaffold" + - [ ] **B3.4b** [Hamza] Implement `create(plan_id: str) -> SandboxContext`: + - [ ] Generate sandbox path: `tempfile.mkdtemp(prefix=f"ca_fs_sandbox_{plan_id}_")` + - [ ] Copy resource directory to sandbox: `shutil.copytree(resource.location, sandbox_path, dirs_exist_ok=True)` + - [ ] Use `shutil.ignore_patterns()` to skip .git, node_modules, **pycache** + - [ ] Record file hashes of original for diff detection later + - [ ] Create and store SandboxContext + - [ ] Update status to CREATED + - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.create()" + - [ ] **B3.4c** [Hamza] Implement `get_path(resource_path: str) -> str`: + - [ ] Same pattern as GitWorktreeSandbox + - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.get_path()" + - [ ] **B3.4d** [Hamza] Implement `commit(message: str | None = None) -> CommitResult`: + - [ ] Compare sandbox files with original (using recorded hashes) + - [ ] Build lists of changed/added/deleted files + - [ ] For each changed file: `shutil.copy2(sandbox_file, original_file)` + - [ ] For each new file: copy and create parent dirs as needed + - [ ] For each deleted file: `os.remove(original_file)` + - [ ] Use atomic operations where possible (write to .tmp then rename) + - [ ] Update status to COMMITTED + - [ ] Return CommitResult with file lists + - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.commit()" + - [ ] **B3.4e** [Hamza] Implement `rollback() -> None`: + - [ ] For filesystem, rollback is implicit - just don't commit + - [ ] Re-copy original to sandbox to reset: `shutil.rmtree(sandbox); shutil.copytree(original, sandbox)` + - [ ] Update status to ROLLED_BACK + - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.rollback()" + - [ ] **B3.4f** [Hamza] Implement `cleanup() -> None`: + - [ ] Remove sandbox directory: `shutil.rmtree(self._sandbox_path, ignore_errors=True)` + - [ ] Update status to CLEANED_UP + - [ ] Commit: "feat(sandbox): implement FilesystemSandbox.cleanup()" + - [ ] **B3.5** [Luis] Implement `NoSandbox` in `src/cleveragents/infrastructure/sandbox/no_sandbox.py`: + - [ ] **B3.5a** [Luis] Create class for non-sandboxable resources: + - [ ] For APIs, some cloud resources, etc. + - [ ] Constructor: accept any Resource with sandbox_strategy=NONE + - [ ] Commit: "feat(sandbox): add NoSandbox class scaffold" + - [ ] **B3.5b** [Luis] Implement all methods as passthrough or warnings: + - [ ] `create()`: Log WARNING "Resource {name} is not sandboxed - changes are immediate" + - [ ] `get_path(resource_path)`: Return original resource path unchanged + - [ ] `commit()`: Return CommitResult(success=True) - changes already applied + - [ ] `rollback()`: Log ERROR "Rollback not possible for non-sandboxed resource" + - [ ] `cleanup()`: No-op, status to CLEANED_UP + - [ ] Commit: "feat(sandbox): implement NoSandbox methods" + - [ ] **B3.6** [Luis] Implement `SandboxFactory` in `src/cleveragents/infrastructure/sandbox/factory.py`: + - [ ] **B3.6a** [Luis] Create factory class: + - [ ] Import all sandbox implementations + - [ ] Define `class SandboxFactory:` + - [ ] Commit: "feat(sandbox): add SandboxFactory scaffold" + - [ ] **B3.6b** [Luis] Implement `create_sandbox(resource: Resource) -> Sandbox`: + - [ ] Match on `resource.sandbox_strategy`: + ```python + match resource.sandbox_strategy: + case SandboxStrategy.GIT_WORKTREE: + return GitWorktreeSandbox(resource) + case SandboxStrategy.COPY_ON_WRITE: + return FilesystemSandbox(resource) + case SandboxStrategy.OVERLAY: + # Overlay not implemented yet, fall back to copy + logger.warning("Overlay not implemented, using copy-on-write") + return FilesystemSandbox(resource) + case SandboxStrategy.TRANSACTION_ROLLBACK: + raise NotImplementedError("Database sandboxing not yet implemented") + case SandboxStrategy.NONE: + return NoSandbox(resource) + case _: + raise ValueError(f"Unknown sandbox strategy: {resource.sandbox_strategy}") + ``` + - [ ] Commit: "feat(sandbox): implement SandboxFactory.create_sandbox()" + - [ ] **B3.6c** [Luis] Add validation helper: + - [ ] `@staticmethod def is_supported(resource: Resource) -> bool:` - Check if sandboxing is supported + - [ ] `@staticmethod def get_supported_strategies(resource_type: ResourceType) -> list[SandboxStrategy]:` - Valid combos + - [ ] Commit: "feat(sandbox): add SandboxFactory validation helpers" + - [ ] **B3.7** [Luis] Implement sandbox lifecycle management: + - [ ] **B3.7a** [Luis] Create `SandboxManager` in `src/cleveragents/infrastructure/sandbox/manager.py`: + - [ ] Import threading for lock management + - [ ] Import factory and protocol types + - [ ] Define `class SandboxManager:` + - [ ] Instance variables: + - [ ] `_factory: SandboxFactory` - injected via constructor + - [ ] `_active_sandboxes: dict[str, dict[str, Sandbox]]` - plan_id -> resource_id -> Sandbox + - [ ] `_lock: threading.RLock` - thread safety for sandbox tracking + - [ ] `_cleanup_on_exit: bool` - whether to cleanup on process exit (default True) + - [ ] Commit: "feat(sandbox): add SandboxManager scaffold" + - [ ] **B3.7b** [Luis] Implement `get_or_create_sandbox(plan_id: str, resource: Resource) -> Sandbox`: + - [ ] Acquire lock + - [ ] Check if sandbox already exists for this plan+resource + - [ ] If exists and status is usable (CREATED, ACTIVE, ROLLED_BACK), return it + - [ ] If exists but cleaned up, remove from tracking + - [ ] Create new sandbox via factory + - [ ] Call sandbox.create(plan_id) to initialize + - [ ] Store in \_active_sandboxes + - [ ] Release lock + - [ ] Return sandbox + - [ ] This is the LAZY sandboxing pattern - only create when needed + - [ ] Commit: "feat(sandbox): implement SandboxManager.get_or_create_sandbox()" + - [ ] **B3.7c** [Luis] Implement `commit_all(plan_id: str) -> list[CommitResult]`: + - [ ] Get all sandboxes for plan_id + - [ ] For each sandbox with status ACTIVE: + - [ ] Call sandbox.commit() + - [ ] Collect CommitResult + - [ ] Return list of all results + - [ ] If any commit fails, don't rollback others (partial commit possible, caller decides) + - [ ] Commit: "feat(sandbox): implement SandboxManager.commit_all()" + - [ ] **B3.7d** [Luis] Implement `rollback_all(plan_id: str) -> None`: + - [ ] Get all sandboxes for plan_id + - [ ] For each sandbox with status ACTIVE: + - [ ] Call sandbox.rollback() + - [ ] Log any rollback failures but continue with others + - [ ] Commit: "feat(sandbox): implement SandboxManager.rollback_all()" + - [ ] **B3.7e** [Luis] Implement `cleanup_all(plan_id: str) -> None`: + - [ ] Get all sandboxes for plan_id + - [ ] For each sandbox: + - [ ] Call sandbox.cleanup() + - [ ] Remove plan_id entry from \_active_sandboxes + - [ ] Commit: "feat(sandbox): implement SandboxManager.cleanup_all()" + - [ ] **B3.7f** [Luis] Implement `cleanup_abandoned() -> int`: + - [ ] Find sandbox directories matching pattern that aren't tracked + - [ ] Check if creating process is still alive (via PID file or lock) + - [ ] If process dead, clean up the directory + - [ ] Return count of cleaned sandboxes + - [ ] This is called on application startup + - [ ] Commit: "feat(sandbox): implement SandboxManager.cleanup_abandoned()" + - [ ] **B3.7g** [Luis] Add atexit handler for graceful cleanup: + - [ ] Register `atexit.register(self._cleanup_on_exit_handler)` + - [ ] Handler calls cleanup_all for all tracked plans + - [ ] Commit: "feat(sandbox): add SandboxManager atexit cleanup" + - [ ] **B3.8** [Luis] Implement merge strategies in `src/cleveragents/infrastructure/sandbox/merge.py`: + - [ ] **B3.8a** [Luis] Define merge types and protocol: + - [ ] Define `MergeResult` dataclass: + - [ ] `success: bool` + - [ ] `content: str | bytes` - merged content + - [ ] `has_conflicts: bool` + - [ ] `conflict_markers: list[tuple[int, int]]` - line ranges with conflicts + - [ ] Define `MergeStrategy(Protocol)`: + - [ ] Method `merge(base: str, ours: str, theirs: str) -> MergeResult` + - [ ] Commit: "feat(sandbox): define merge types and protocol" + - [ ] **B3.8b** [Luis] Implement `GitMergeStrategy`: + - [ ] Use `git merge-file` for three-way merge + - [ ] Write base, ours, theirs to temp files + - [ ] Run `git merge-file -p ours base theirs` + - [ ] Parse output for conflict markers (`<<<<<<<`, `=======`, `>>>>>>>`) + - [ ] Return MergeResult with content and conflict info + - [ ] Commit: "feat(sandbox): implement GitMergeStrategy" + - [ ] **B3.8c** [Luis] Implement `SequentialMergeStrategy`: + - [ ] For non-mergeable resources, apply changes in order + - [ ] "Theirs" (second change) always wins + - [ ] Return MergeResult(success=True, content=theirs) + - [ ] Commit: "feat(sandbox): implement SequentialMergeStrategy" + - [ ] **B3.8d** [Luis] Implement `JsonMergeStrategy`: + - [ ] Parse both as JSON + - [ ] Deep merge objects (recursive dict merge) + - [ ] Arrays: concatenate or last-wins based on config + - [ ] Return serialized merged JSON + - [ ] Commit: "feat(sandbox): implement JsonMergeStrategy" + + - [ ] Tests: Integration tests for each sandbox type + - [ ] **B3.9** [Rui] Write Behave scenarios in `features/sandbox_git_worktree.feature`: + - [ ] **B3.9a** [Rui] Creation scenarios: + - [ ] Scenario: Create git worktree sandbox from local git repo + - [ ] Given a git repository at "/tmp/test-repo" with files + - [ ] When I create a GitWorktreeSandbox for that resource + - [ ] And I call sandbox.create("plan-123") + - [ ] Then a worktree exists at the sandbox path + - [ ] And the sandbox status is CREATED + - [ ] And the original repo is unchanged + - [ ] Scenario: Create sandbox from remote git URL (if applicable) + - [ ] Scenario: Create fails for non-git resource type + - [ ] Commit: "test(behave): add git worktree creation scenarios" + - [ ] **B3.9b** [Rui] Modification isolation scenarios: + - [ ] Scenario: Modify file in sandbox does not affect original + - [ ] Given a created git worktree sandbox + - [ ] When I write "modified content" to sandbox path "test.py" + - [ ] Then the original repo's "test.py" is unchanged + - [ ] And the sandbox status is ACTIVE + - [ ] Scenario: Create new file in sandbox does not appear in original + - [ ] Scenario: Delete file in sandbox does not delete original + - [ ] Commit: "test(behave): add git worktree isolation scenarios" + - [ ] **B3.9c** [Rui] Commit scenarios: + - [ ] Scenario: Commit sandbox creates git commit + - [ ] Given a sandbox with modified files + - [ ] When I call sandbox.commit("Test commit") + - [ ] Then a git commit exists with message "Test commit" + - [ ] And CommitResult.changed_files contains "test.py" + - [ ] And sandbox status is COMMITTED + - [ ] Scenario: Commit with no changes succeeds with empty file list + - [ ] Scenario: Commit includes all staged and unstaged changes + - [ ] Commit: "test(behave): add git worktree commit scenarios" + - [ ] **B3.9d** [Rui] Rollback and cleanup scenarios: + - [ ] Scenario: Rollback sandbox discards all changes + - [ ] Given a sandbox with modified files + - [ ] When I call sandbox.rollback() + - [ ] Then the sandbox files match original + - [ ] And sandbox status is ROLLED_BACK + - [ ] Scenario: Cleanup removes worktree and branch + - [ ] Scenario: Multiple sandboxes from same repo are isolated + - [ ] Commit: "test(behave): add git worktree rollback/cleanup scenarios" + - [ ] **B3.10** [Rui] Write Behave scenarios in `features/sandbox_filesystem.feature`: + - [ ] Similar structure to B3.9 but for filesystem sandbox + - [ ] Scenario: Create filesystem sandbox copies directory + - [ ] Scenario: Large directory copy uses efficient patterns + - [ ] Scenario: Ignore patterns (node_modules, .git) are skipped during copy + - [ ] Scenario: Modify file in sandbox does not affect original + - [ ] Scenario: Commit sandbox applies changes to original atomically + - [ ] Scenario: Rollback sandbox resets to original state + - [ ] Scenario: Cleanup removes temp directory + - [ ] Commit: "test(behave): add filesystem sandbox scenarios" + - [ ] **B3.11** [Rui] Write Robot integration test `robot/sandbox_integration.robot`: + - [ ] **B3.11a** [Rui] Full lifecycle test: + - [ ] Create real git repo with multiple files + - [ ] Create sandbox, modify files, commit + - [ ] Verify changes appear in repo + - [ ] Create second sandbox, rollback, verify no changes + - [ ] Cleanup, verify temp directories removed + - [ ] Commit: "test(robot): add full sandbox lifecycle e2e test" + - [ ] **B3.11b** [Rui] Parallel sandbox test: + - [ ] Create two sandboxes on same repo + - [ ] Modify different files in each + - [ ] Commit both + - [ ] Verify no cross-contamination + - [ ] Commit: "test(robot): add parallel sandbox isolation e2e test" + - [ ] Tests: Parallel execution isolation tests + - [ ] **B3.12** [Rui] Write Behave scenarios in `features/sandbox_isolation.feature`: + - [ ] Scenario: Two plans with sandboxes on same resource are isolated + - [ ] Given Plan A creates sandbox for resource R + - [ ] And Plan B creates sandbox for same resource R + - [ ] When Plan A writes "A content" to file.txt + - [ ] And Plan B writes "B content" to file.txt + - [ ] Then Plan A's sandbox shows "A content" + - [ ] And Plan B's sandbox shows "B content" + - [ ] And original file is unchanged + - [ ] Scenario: Plan A cannot see Plan B's intermediate changes + - [ ] Scenario: Commits from different plans require merge + - [ ] Commit: "test(behave): add sandbox isolation scenarios" + - [ ] Tests: Merge conflict resolution tests + - [ ] **B3.13** [Rui] Write Behave scenarios in `features/sandbox_merge.feature`: + - [ ] Scenario: Git merge strategy handles non-conflicting changes + - [ ] Given base content "line1\nline2\nline3" + - [ ] And ours changes line1 to "modified1" + - [ ] And theirs changes line3 to "modified3" + - [ ] When I merge with GitMergeStrategy + - [ ] Then result is "modified1\nline2\nmodified3" + - [ ] And has_conflicts is False + - [ ] Scenario: Git merge strategy marks conflicts appropriately + - [ ] Given both ours and theirs change line2 + - [ ] When I merge + - [ ] Then has_conflicts is True + - [ ] And content contains conflict markers + - [ ] Scenario: Sequential merge uses theirs content + - [ ] Scenario: JSON merge combines object properties + - [ ] Commit: "test(behave): add merge strategy scenarios" - [ ] **Stage B4: Resource Integration** (Day 6-7) **[Hamza]** - **SEQUENTIAL ORDER**: B4.1 (Types) → B4.2 (Service scaffold) → B4.3 (Access) → B4.4 (Lazy sandbox) → B4.5 (Commit/Rollback) → B4.6 (Cleanup hooks) → B4.7 (Lifecycle integration) + **SEQUENTIAL ORDER**: B4.1 (Types) → B4.2 (Service scaffold) → B4.3 (Access) → B4.4 (Lazy sandbox) → B4.5 (Commit/Rollback) → B4.6 (Cleanup hooks) → B4.7 (Lifecycle integration) + - [ ] Code: Connect resources to plan execution + - [ ] **B4.1** [Hamza] Define resource access types in `src/cleveragents/domain/models/core/resource_access.py`: + - [ ] **B4.1a** [Hamza] Define `AccessMode` enum: + - [ ] `READ = "read"` - Read-only access, may use original or sandbox + - [ ] `WRITE = "write"` - Write access, requires sandbox + - [ ] `EXECUTE = "execute"` - Execute commands in context + - [ ] Commit: "feat(domain): define AccessMode enum" + - [ ] **B4.1b** [Hamza] Define `ResourceAccess` dataclass: + - [ ] `resource_id: str` - Which resource is being accessed + - [ ] `plan_id: str` - Which plan is accessing + - [ ] `mode: AccessMode` - How resource is being accessed + - [ ] `sandbox: Sandbox | None` - Sandbox if write mode + - [ ] `effective_path: str` - Resolved path (sandbox or original) + - [ ] `accessed_at: datetime` - When access was granted + - [ ] `is_sandboxed: bool` - Whether using sandbox or original + - [ ] Commit: "feat(domain): define ResourceAccess dataclass" + - [ ] **B4.1c** [Hamza] Define `ResourceAccessTracker` dataclass: + - [ ] `plan_id: str` - Plan being tracked + - [ ] `accesses: dict[str, ResourceAccess]` - resource_id -> access + - [ ] `read_resources: set[str]` - Resources accessed for read + - [ ] `write_resources: set[str]` - Resources accessed for write + - [ ] `first_write_at: datetime | None` - When first write occurred + - [ ] Commit: "feat(domain): define ResourceAccessTracker" + - [ ] **B4.2** [Hamza] Create `ResourceService` scaffold in `src/cleveragents/application/services/resource_service.py`: + - [ ] **B4.2a** [Hamza] Define class with dependencies: - - [ ] Code: Connect resources to plan execution - - [ ] **B4.1** [Hamza] Define resource access types in `src/cleveragents/domain/models/core/resource_access.py`: - - [ ] **B4.1a** [Hamza] Define `AccessMode` enum: - - [ ] `READ = "read"` - Read-only access, may use original or sandbox - - [ ] `WRITE = "write"` - Write access, requires sandbox - - [ ] `EXECUTE = "execute"` - Execute commands in context - - [ ] Commit: "feat(domain): define AccessMode enum" - - [ ] **B4.1b** [Hamza] Define `ResourceAccess` dataclass: - - [ ] `resource_id: str` - Which resource is being accessed - - [ ] `plan_id: str` - Which plan is accessing - - [ ] `mode: AccessMode` - How resource is being accessed - - [ ] `sandbox: Sandbox | None` - Sandbox if write mode - - [ ] `effective_path: str` - Resolved path (sandbox or original) - - [ ] `accessed_at: datetime` - When access was granted - - [ ] `is_sandboxed: bool` - Whether using sandbox or original - - [ ] Commit: "feat(domain): define ResourceAccess dataclass" - - [ ] **B4.1c** [Hamza] Define `ResourceAccessTracker` dataclass: - - [ ] `plan_id: str` - Plan being tracked - - [ ] `accesses: dict[str, ResourceAccess]` - resource_id -> access - - [ ] `read_resources: set[str]` - Resources accessed for read - - [ ] `write_resources: set[str]` - Resources accessed for write - - [ ] `first_write_at: datetime | None` - When first write occurred - - [ ] Commit: "feat(domain): define ResourceAccessTracker" - - [ ] **B4.2** [Hamza] Create `ResourceService` scaffold in `src/cleveragents/application/services/resource_service.py`: - - [ ] **B4.2a** [Hamza] Define class with dependencies: - ```python - class ResourceService: - def __init__( - self, - sandbox_manager: SandboxManager, - project_repo: ProjectRepository, - resource_repo: ResourceRepository, - config: ResourceServiceConfig - ): - self._sandbox_manager = sandbox_manager - self._project_repo = project_repo - self._resource_repo = resource_repo - self._config = config - self._trackers: dict[str, ResourceAccessTracker] = {} # plan_id -> tracker - self._lock = threading.RLock() - ``` - - [ ] Commit: "feat(service): add ResourceService scaffold with dependencies" - - [ ] **B4.2b** [Hamza] Define `ResourceServiceConfig` in `src/cleveragents/config/settings.py`: - - [ ] `force_sandbox_for_reads: bool = False` - Always sandbox, even for reads - - [ ] `preserve_sandbox_on_failure: bool = True` - Keep sandbox for debugging on error - - [ ] `auto_cleanup_abandoned: bool = True` - Cleanup orphaned sandboxes on startup - - [ ] `max_sandboxes_per_plan: int = 10` - Limit sandboxes per plan - - [ ] Commit: "feat(config): add ResourceServiceConfig" - - [ ] **B4.3** [Hamza] Implement `access_resource()` method: - - [ ] **B4.3a** [Hamza] Core method signature: - ```python - def access_resource( - self, - plan_id: str, - resource: Resource, - mode: AccessMode = AccessMode.READ - ) -> ResourceAccess: - ``` - - [ ] Commit: "feat(service): add access_resource() signature" - - [ ] **B4.3b** [Hamza] Implement tracker initialization: - - [ ] Acquire lock - - [ ] If no tracker for plan_id, create one - - [ ] Check if resource already accessed - - [ ] If already accessed with same or higher mode, return existing access - - [ ] Commit: "feat(service): implement access_resource() tracker init" - - [ ] **B4.3c** [Hamza] Implement read access logic: - - [ ] If mode is READ and not force_sandbox_for_reads: - - [ ] Return access with effective_path = resource.location - - [ ] Set is_sandboxed = False - - [ ] Record in tracker's read_resources - - [ ] Commit: "feat(service): implement read access without sandbox" - - [ ] **B4.3d** [Hamza] Implement write access logic: - - [ ] If mode is WRITE: - - [ ] Call `sandbox_manager.get_or_create_sandbox(plan_id, resource)` - - [ ] Get sandbox.context.sandbox_path - - [ ] Set effective_path = sandbox_path - - [ ] Set is_sandboxed = True - - [ ] Record in tracker's write_resources - - [ ] Set first_write_at if not set - - [ ] Commit: "feat(service): implement write access with sandbox" - - [ ] **B4.3e** [Hamza] Implement access upgrade: - - [ ] If resource was accessed as READ but now needs WRITE: - - [ ] Create sandbox if not exists - - [ ] Update tracker to reflect write mode - - [ ] Return new ResourceAccess with sandboxed path - - [ ] Commit: "feat(service): implement access mode upgrade" - - [ ] **B4.4** [Hamza] Implement lazy sandboxing pattern: - - [ ] **B4.4a** [Hamza] Sandbox created only when write occurs: - - [ ] `access_resource(plan_id, resource, READ)` - no sandbox - - [ ] First `access_resource(plan_id, resource, WRITE)` - creates sandbox - - [ ] Subsequent writes to same resource - reuses existing sandbox - - [ ] Log: "Created sandbox for resource {name} on first write" - - [ ] Commit: "feat(service): implement lazy sandbox creation" - - [ ] **B4.4b** [Hamza] Track sandbox lifecycle per plan: - - [ ] Method `get_plan_sandboxes(plan_id: str) -> list[Sandbox]`: - - [ ] Return all sandboxes associated with plan - - [ ] Method `has_pending_changes(plan_id: str) -> bool`: - - [ ] Check if any sandbox has uncommitted changes - - [ ] Commit: "feat(service): add sandbox tracking methods" - - [ ] **B4.5** [Hamza] Implement commit and rollback methods: - - [ ] **B4.5a** [Hamza] Implement `commit_plan_resources()`: - ```python - def commit_plan_resources(self, plan_id: str, message: str | None = None) -> list[CommitResult]: - """Commit all sandbox changes for a plan.""" - results = [] - sandboxes = self._sandbox_manager.get_sandboxes(plan_id) - for sandbox in sandboxes: - if sandbox.status == SandboxStatus.ACTIVE: - result = sandbox.commit(message or f"CleverAgents plan {plan_id}") - results.append(result) - self._log_commit_result(result) - return results - ``` - - [ ] Commit: "feat(service): implement commit_plan_resources()" - - [ ] **B4.5b** [Hamza] Implement `rollback_plan_resources()`: - ```python - def rollback_plan_resources(self, plan_id: str) -> None: - """Rollback all sandbox changes for a plan.""" - sandboxes = self._sandbox_manager.get_sandboxes(plan_id) - for sandbox in sandboxes: - if sandbox.status in (SandboxStatus.CREATED, SandboxStatus.ACTIVE): - sandbox.rollback() - logger.info(f"Rolled back sandbox {sandbox.sandbox_id}") - ``` - - [ ] Commit: "feat(service): implement rollback_plan_resources()" - - [ ] **B4.5c** [Hamza] Implement `cleanup_plan_resources()`: - ```python - def cleanup_plan_resources(self, plan_id: str) -> None: - """Clean up all sandbox artifacts for a plan.""" - self._sandbox_manager.cleanup_all(plan_id) - # Remove tracker - with self._lock: - if plan_id in self._trackers: - del self._trackers[plan_id] - logger.info(f"Cleaned up all resources for plan {plan_id}") - ``` - - [ ] Commit: "feat(service): implement cleanup_plan_resources()" - - [ ] **B4.6** [Hamza] Add sandbox cleanup hooks: - - [ ] **B4.6a** [Hamza] Implement plan completion hook: - - [ ] Create `PlanCompletionHandler` that listens for plan state changes - - [ ] On transition to APPLIED: commit then cleanup - - [ ] On transition to CANCELLED: rollback then cleanup - - [ ] Commit: "feat(service): add plan completion cleanup hook" - - [ ] **B4.6b** [Hamza] Implement failure handling hook: - - [ ] On plan transition to ERRORED: - - [ ] If preserve_sandbox_on_failure: keep sandbox for debugging - - [ ] Log: "Sandbox preserved for debugging: {sandbox_path}" - - [ ] Otherwise: rollback and cleanup - - [ ] Commit: "feat(service): add failure handling hook" - - [ ] **B4.6c** [Hamza] Implement application exit hook: - - [ ] Register `atexit.register(self._cleanup_all_on_exit)` - - [ ] Handler iterates all active trackers - - [ ] Cleanup all sandboxes (don't commit - exit is unexpected) - - [ ] Log warning if any uncommitted changes lost - - [ ] Commit: "feat(service): add atexit cleanup hook" - - [ ] **B4.6d** [Hamza] Implement startup cleanup: - - [ ] Method `cleanup_abandoned_sandboxes() -> int`: - - [ ] Called on application startup - - [ ] Call `sandbox_manager.cleanup_abandoned()` - - [ ] Return count of cleaned sandboxes - - [ ] Log: "Cleaned up {n} abandoned sandboxes from previous session" - - [ ] Commit: "feat(service): add startup cleanup for abandoned sandboxes" - - [ ] **B4.7** [Hamza] Integrate with plan execution lifecycle: - - [ ] **B4.7a** [Hamza] Update `PlanLifecycleService.apply_plan()`: - - [ ] Before apply: verify all sandboxes have changes - - [ ] Call `resource_service.commit_plan_resources(plan_id)` - - [ ] If any commit fails, rollback all and raise - - [ ] On success: cleanup resources - - [ ] Commit: "feat(service): integrate ResourceService with apply_plan()" - - [ ] **B4.7b** [Hamza] Update `PlanLifecycleService.cancel_plan()`: - - [ ] Call `resource_service.rollback_plan_resources(plan_id)` - - [ ] Call `resource_service.cleanup_plan_resources(plan_id)` - - [ ] Commit: "feat(service): integrate ResourceService with cancel_plan()" - - [ ] **B4.7c** [Hamza] Add DI wiring for ResourceService: - - [ ] Update container.py to provide ResourceService - - [ ] Inject into PlanLifecycleService - - [ ] Commit: "feat(di): wire ResourceService into container" - - [ ] Tests: End-to-end tests for plan execution with sandboxed resources - - [ ] **B4.8** [Rui] Write Behave scenarios in `features/resource_service.feature`: - - [ ] **B4.8a** [Rui] Access mode scenarios: - - [ ] Scenario: First write access creates sandbox - - [ ] Given a plan "plan-123" and resource "repo" with sandbox_strategy=git_worktree - - [ ] When I call `resource_service.access_resource("plan-123", repo, WRITE)` - - [ ] Then a sandbox is created for the resource - - [ ] And the returned ResourceAccess.is_sandboxed is True - - [ ] And the effective_path points to the sandbox location - - [ ] Scenario: Read access without write uses original - - [ ] When I call `resource_service.access_resource("plan-123", repo, READ)` - - [ ] Then no sandbox is created - - [ ] And ResourceAccess.is_sandboxed is False - - [ ] And effective_path points to original resource.location - - [ ] Commit: "test(behave): add resource access mode scenarios" - - [ ] **B4.8b** [Rui] Sandbox reuse scenarios: - - [ ] Scenario: Multiple writes use same sandbox - - [ ] Given I accessed resource for WRITE once - - [ ] When I access the same resource for WRITE again - - [ ] Then the same sandbox is returned - - [ ] And only one sandbox exists for this plan+resource - - [ ] Scenario: Access upgrade from READ to WRITE creates sandbox - - [ ] Given I accessed resource for READ (no sandbox) - - [ ] When I access the same resource for WRITE - - [ ] Then a sandbox is created - - [ ] And the effective_path changes to sandbox path - - [ ] Commit: "test(behave): add sandbox reuse scenarios" - - [ ] **B4.8c** [Rui] Commit and rollback scenarios: - - [ ] Scenario: Plan completion commits and cleans up sandbox - - [ ] Given a plan with sandbox containing changes - - [ ] When the plan transitions to APPLIED - - [ ] Then commit_plan_resources is called - - [ ] And all changes are committed to the original resource - - [ ] And the sandbox is cleaned up - - [ ] Scenario: Plan failure rolls back sandbox - - [ ] Given a plan with sandbox containing changes - - [ ] When the plan transitions to ERRORED - - [ ] Then rollback_plan_resources is called (if not preserve_sandbox_on_failure) - - [ ] And no changes are committed - - [ ] Scenario: Plan failure preserves sandbox for debugging - - [ ] Given preserve_sandbox_on_failure=True - - [ ] When the plan transitions to ERRORED - - [ ] Then the sandbox is NOT cleaned up - - [ ] And a log message indicates sandbox location for debugging - - [ ] Commit: "test(behave): add commit/rollback scenarios" - - [ ] **B4.8d** [Rui] Cleanup scenarios: - - [ ] Scenario: Application exit cleans up all sandboxes - - [ ] Given multiple plans with active sandboxes - - [ ] When the application exits - - [ ] Then all sandbox directories are removed - - [ ] Scenario: Startup cleans up abandoned sandboxes - - [ ] Given orphaned sandbox directories from previous crash - - [ ] When the application starts - - [ ] Then abandoned sandboxes are cleaned up - - [ ] And a log message indicates how many were cleaned - - [ ] Commit: "test(behave): add cleanup scenarios" - - [ ] **B4.9** [Rui] Write Robot integration test `robot/resource_service_integration.robot`: - - [ ] **B4.9a** [Rui] Full lifecycle test: - - [ ] Test: Full plan execution with sandboxed git resource - - [ ] Create a real git repository with files - - [ ] Create a project with git resource - - [ ] Create a plan targeting the project - - [ ] Access resource for write (sandbox created) - - [ ] Modify files via sandbox path - - [ ] Complete plan (commit and cleanup) - - [ ] Verify changes appear in original git repo - - [ ] Verify sandbox directory is removed - - [ ] Commit: "test(robot): add full resource service e2e test" - - [ ] **B4.9b** [Rui] Multi-resource test: - - [ ] Test: Plan with multiple resources - - [ ] Create project with git repo + filesystem resources - - [ ] Access both for write - - [ ] Modify files in both sandboxes - - [ ] Commit all - - [ ] Verify both original resources have changes - - [ ] Commit: "test(robot): add multi-resource plan e2e test" + ```python + class ResourceService: + def __init__( + self, + sandbox_manager: SandboxManager, + project_repo: ProjectRepository, + resource_repo: ResourceRepository, + config: ResourceServiceConfig + ): + self._sandbox_manager = sandbox_manager + self._project_repo = project_repo + self._resource_repo = resource_repo + self._config = config + self._trackers: dict[str, ResourceAccessTracker] = {} # plan_id -> tracker + self._lock = threading.RLock() + ``` + + - [ ] Commit: "feat(service): add ResourceService scaffold with dependencies" + + - [ ] **B4.2b** [Hamza] Define `ResourceServiceConfig` in `src/cleveragents/config/settings.py`: + - [ ] `force_sandbox_for_reads: bool = False` - Always sandbox, even for reads + - [ ] `preserve_sandbox_on_failure: bool = True` - Keep sandbox for debugging on error + - [ ] `auto_cleanup_abandoned: bool = True` - Cleanup orphaned sandboxes on startup + - [ ] `max_sandboxes_per_plan: int = 10` - Limit sandboxes per plan + - [ ] Commit: "feat(config): add ResourceServiceConfig" + + - [ ] **B4.3** [Hamza] Implement `access_resource()` method: + - [ ] **B4.3a** [Hamza] Core method signature: + + ```python + def access_resource( + self, + plan_id: str, + resource: Resource, + mode: AccessMode = AccessMode.READ + ) -> ResourceAccess: + ``` + + - [ ] Commit: "feat(service): add access_resource() signature" + + - [ ] **B4.3b** [Hamza] Implement tracker initialization: + - [ ] Acquire lock + - [ ] If no tracker for plan_id, create one + - [ ] Check if resource already accessed + - [ ] If already accessed with same or higher mode, return existing access + - [ ] Commit: "feat(service): implement access_resource() tracker init" + - [ ] **B4.3c** [Hamza] Implement read access logic: + - [ ] If mode is READ and not force_sandbox_for_reads: + - [ ] Return access with effective_path = resource.location + - [ ] Set is_sandboxed = False + - [ ] Record in tracker's read_resources + - [ ] Commit: "feat(service): implement read access without sandbox" + - [ ] **B4.3d** [Hamza] Implement write access logic: + - [ ] If mode is WRITE: + - [ ] Call `sandbox_manager.get_or_create_sandbox(plan_id, resource)` + - [ ] Get sandbox.context.sandbox_path + - [ ] Set effective_path = sandbox_path + - [ ] Set is_sandboxed = True + - [ ] Record in tracker's write_resources + - [ ] Set first_write_at if not set + - [ ] Commit: "feat(service): implement write access with sandbox" + - [ ] **B4.3e** [Hamza] Implement access upgrade: + - [ ] If resource was accessed as READ but now needs WRITE: + - [ ] Create sandbox if not exists + - [ ] Update tracker to reflect write mode + - [ ] Return new ResourceAccess with sandboxed path + - [ ] Commit: "feat(service): implement access mode upgrade" + + - [ ] **B4.4** [Hamza] Implement lazy sandboxing pattern: + - [ ] **B4.4a** [Hamza] Sandbox created only when write occurs: + - [ ] `access_resource(plan_id, resource, READ)` - no sandbox + - [ ] First `access_resource(plan_id, resource, WRITE)` - creates sandbox + - [ ] Subsequent writes to same resource - reuses existing sandbox + - [ ] Log: "Created sandbox for resource {name} on first write" + - [ ] Commit: "feat(service): implement lazy sandbox creation" + - [ ] **B4.4b** [Hamza] Track sandbox lifecycle per plan: + - [ ] Method `get_plan_sandboxes(plan_id: str) -> list[Sandbox]`: + - [ ] Return all sandboxes associated with plan + - [ ] Method `has_pending_changes(plan_id: str) -> bool`: + - [ ] Check if any sandbox has uncommitted changes + - [ ] Commit: "feat(service): add sandbox tracking methods" + - [ ] **B4.5** [Hamza] Implement commit and rollback methods: + - [ ] **B4.5a** [Hamza] Implement `commit_plan_resources()`: + + ```python + def commit_plan_resources(self, plan_id: str, message: str | None = None) -> list[CommitResult]: + """Commit all sandbox changes for a plan.""" + results = [] + sandboxes = self._sandbox_manager.get_sandboxes(plan_id) + for sandbox in sandboxes: + if sandbox.status == SandboxStatus.ACTIVE: + result = sandbox.commit(message or f"CleverAgents plan {plan_id}") + results.append(result) + self._log_commit_result(result) + return results + ``` + + - [ ] Commit: "feat(service): implement commit_plan_resources()" + + - [ ] **B4.5b** [Hamza] Implement `rollback_plan_resources()`: + + ```python + def rollback_plan_resources(self, plan_id: str) -> None: + """Rollback all sandbox changes for a plan.""" + sandboxes = self._sandbox_manager.get_sandboxes(plan_id) + for sandbox in sandboxes: + if sandbox.status in (SandboxStatus.CREATED, SandboxStatus.ACTIVE): + sandbox.rollback() + logger.info(f"Rolled back sandbox {sandbox.sandbox_id}") + ``` + + - [ ] Commit: "feat(service): implement rollback_plan_resources()" + + - [ ] **B4.5c** [Hamza] Implement `cleanup_plan_resources()`: + + ```python + def cleanup_plan_resources(self, plan_id: str) -> None: + """Clean up all sandbox artifacts for a plan.""" + self._sandbox_manager.cleanup_all(plan_id) + # Remove tracker + with self._lock: + if plan_id in self._trackers: + del self._trackers[plan_id] + logger.info(f"Cleaned up all resources for plan {plan_id}") + ``` + + - [ ] Commit: "feat(service): implement cleanup_plan_resources()" + + - [ ] **B4.6** [Hamza] Add sandbox cleanup hooks: + - [ ] **B4.6a** [Hamza] Implement plan completion hook: + - [ ] Create `PlanCompletionHandler` that listens for plan state changes + - [ ] On transition to APPLIED: commit then cleanup + - [ ] On transition to CANCELLED: rollback then cleanup + - [ ] Commit: "feat(service): add plan completion cleanup hook" + - [ ] **B4.6b** [Hamza] Implement failure handling hook: + - [ ] On plan transition to ERRORED: + - [ ] If preserve_sandbox_on_failure: keep sandbox for debugging + - [ ] Log: "Sandbox preserved for debugging: {sandbox_path}" + - [ ] Otherwise: rollback and cleanup + - [ ] Commit: "feat(service): add failure handling hook" + - [ ] **B4.6c** [Hamza] Implement application exit hook: + - [ ] Register `atexit.register(self._cleanup_all_on_exit)` + - [ ] Handler iterates all active trackers + - [ ] Cleanup all sandboxes (don't commit - exit is unexpected) + - [ ] Log warning if any uncommitted changes lost + - [ ] Commit: "feat(service): add atexit cleanup hook" + - [ ] **B4.6d** [Hamza] Implement startup cleanup: + - [ ] Method `cleanup_abandoned_sandboxes() -> int`: + - [ ] Called on application startup + - [ ] Call `sandbox_manager.cleanup_abandoned()` + - [ ] Return count of cleaned sandboxes + - [ ] Log: "Cleaned up {n} abandoned sandboxes from previous session" + - [ ] Commit: "feat(service): add startup cleanup for abandoned sandboxes" + - [ ] **B4.7** [Hamza] Integrate with plan execution lifecycle: + - [ ] **B4.7a** [Hamza] Update `PlanLifecycleService.apply_plan()`: + - [ ] Before apply: verify all sandboxes have changes + - [ ] Call `resource_service.commit_plan_resources(plan_id)` + - [ ] If any commit fails, rollback all and raise + - [ ] On success: cleanup resources + - [ ] Commit: "feat(service): integrate ResourceService with apply_plan()" + - [ ] **B4.7b** [Hamza] Update `PlanLifecycleService.cancel_plan()`: + - [ ] Call `resource_service.rollback_plan_resources(plan_id)` + - [ ] Call `resource_service.cleanup_plan_resources(plan_id)` + - [ ] Commit: "feat(service): integrate ResourceService with cancel_plan()" + - [ ] **B4.7c** [Hamza] Add DI wiring for ResourceService: + - [ ] Update container.py to provide ResourceService + - [ ] Inject into PlanLifecycleService + - [ ] Commit: "feat(di): wire ResourceService into container" + + - [ ] Tests: End-to-end tests for plan execution with sandboxed resources + - [ ] **B4.8** [Rui] Write Behave scenarios in `features/resource_service.feature`: + - [ ] **B4.8a** [Rui] Access mode scenarios: + - [ ] Scenario: First write access creates sandbox + - [ ] Given a plan "plan-123" and resource "repo" with sandbox_strategy=git_worktree + - [ ] When I call `resource_service.access_resource("plan-123", repo, WRITE)` + - [ ] Then a sandbox is created for the resource + - [ ] And the returned ResourceAccess.is_sandboxed is True + - [ ] And the effective_path points to the sandbox location + - [ ] Scenario: Read access without write uses original + - [ ] When I call `resource_service.access_resource("plan-123", repo, READ)` + - [ ] Then no sandbox is created + - [ ] And ResourceAccess.is_sandboxed is False + - [ ] And effective_path points to original resource.location + - [ ] Commit: "test(behave): add resource access mode scenarios" + - [ ] **B4.8b** [Rui] Sandbox reuse scenarios: + - [ ] Scenario: Multiple writes use same sandbox + - [ ] Given I accessed resource for WRITE once + - [ ] When I access the same resource for WRITE again + - [ ] Then the same sandbox is returned + - [ ] And only one sandbox exists for this plan+resource + - [ ] Scenario: Access upgrade from READ to WRITE creates sandbox + - [ ] Given I accessed resource for READ (no sandbox) + - [ ] When I access the same resource for WRITE + - [ ] Then a sandbox is created + - [ ] And the effective_path changes to sandbox path + - [ ] Commit: "test(behave): add sandbox reuse scenarios" + - [ ] **B4.8c** [Rui] Commit and rollback scenarios: + - [ ] Scenario: Plan completion commits and cleans up sandbox + - [ ] Given a plan with sandbox containing changes + - [ ] When the plan transitions to APPLIED + - [ ] Then commit_plan_resources is called + - [ ] And all changes are committed to the original resource + - [ ] And the sandbox is cleaned up + - [ ] Scenario: Plan failure rolls back sandbox + - [ ] Given a plan with sandbox containing changes + - [ ] When the plan transitions to ERRORED + - [ ] Then rollback_plan_resources is called (if not preserve_sandbox_on_failure) + - [ ] And no changes are committed + - [ ] Scenario: Plan failure preserves sandbox for debugging + - [ ] Given preserve_sandbox_on_failure=True + - [ ] When the plan transitions to ERRORED + - [ ] Then the sandbox is NOT cleaned up + - [ ] And a log message indicates sandbox location for debugging + - [ ] Commit: "test(behave): add commit/rollback scenarios" + - [ ] **B4.8d** [Rui] Cleanup scenarios: + - [ ] Scenario: Application exit cleans up all sandboxes + - [ ] Given multiple plans with active sandboxes + - [ ] When the application exits + - [ ] Then all sandbox directories are removed + - [ ] Scenario: Startup cleans up abandoned sandboxes + - [ ] Given orphaned sandbox directories from previous crash + - [ ] When the application starts + - [ ] Then abandoned sandboxes are cleaned up + - [ ] And a log message indicates how many were cleaned + - [ ] Commit: "test(behave): add cleanup scenarios" + - [ ] **B4.9** [Rui] Write Robot integration test `robot/resource_service_integration.robot`: + - [ ] **B4.9a** [Rui] Full lifecycle test: + - [ ] Test: Full plan execution with sandboxed git resource + - [ ] Create a real git repository with files + - [ ] Create a project with git resource + - [ ] Create a plan targeting the project + - [ ] Access resource for write (sandbox created) + - [ ] Modify files via sandbox path + - [ ] Complete plan (commit and cleanup) + - [ ] Verify changes appear in original git repo + - [ ] Verify sandbox directory is removed + - [ ] Commit: "test(robot): add full resource service e2e test" + - [ ] **B4.9b** [Rui] Multi-resource test: + - [ ] Test: Plan with multiple resources + - [ ] Create project with git repo + filesystem resources + - [ ] Access both for write + - [ ] Modify files in both sandboxes + - [ ] Commit all + - [ ] Verify both original resources have changes + - [ ] Commit: "test(robot): add multi-resource plan e2e test" - [ ] **Stage B5: Project Persistence** (Day 7-8) **[Hamza]** - **SEQUENTIAL ORDER**: B5.1 (Projects migration) → B5.2 (Resources migration) → B5.3 (Project model) → B5.4 (Resource model) → B5.5 (ProjectRepository) → B5.6 (ResourceRepository) → B5.7 (Tests) + **SEQUENTIAL ORDER**: B5.1 (Projects migration) → B5.2 (Resources migration) → B5.3 (Project model) → B5.4 (Resource model) → B5.5 (ProjectRepository) → B5.6 (ResourceRepository) → B5.7 (Tests) + - [ ] Code: Project/Resource database schema + - [ ] **B5.1** [Hamza] Create Alembic migration for `projects` table: + - [ ] **B5.1a** [Hamza] Generate migration file: + - [ ] Run `alembic revision --autogenerate -m "create_projects_table"` + - [ ] Commit: "chore(db): generate projects table migration" + - [ ] **B5.1b** [Hamza] Define schema: - - [ ] Code: Project/Resource database schema - - [ ] **B5.1** [Hamza] Create Alembic migration for `projects` table: - - [ ] **B5.1a** [Hamza] Generate migration file: - - [ ] Run `alembic revision --autogenerate -m "create_projects_table"` - - [ ] Commit: "chore(db): generate projects table migration" - - [ ] **B5.1b** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'projects', - sa.Column('project_id', sa.Text(), nullable=False), - sa.Column('name', sa.Text(), nullable=False), - sa.Column('namespace', sa.Text(), nullable=False), - sa.Column('short_name', sa.Text(), nullable=False), - sa.Column('description', sa.Text(), nullable=True), - sa.Column('tags', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('is_remote', sa.Boolean(), nullable=False, server_default='false'), - sa.Column('validation_config', sa.JSON(), nullable=True), - sa.Column('context_config', sa.JSON(), nullable=True), - sa.Column('created_at', sa.Text(), nullable=False), - sa.Column('updated_at', sa.Text(), nullable=False), - sa.PrimaryKeyConstraint('project_id') - ) - op.create_index('ix_projects_name', 'projects', ['name'], unique=True) - op.create_index('ix_projects_namespace', 'projects', ['namespace']) - op.create_index('ix_projects_namespace_short_name', 'projects', ['namespace', 'short_name'], unique=True) - ``` - - [ ] Commit: "feat(db): add projects table schema" - - [ ] **B5.1c** [Hamza] Add downgrade: - ```python - def downgrade(): - op.drop_index('ix_projects_namespace_short_name') - op.drop_index('ix_projects_namespace') - op.drop_index('ix_projects_name') - op.drop_table('projects') - ``` - - [ ] Commit: "feat(db): add projects table downgrade" - - [ ] **B5.2** [Hamza] Create Alembic migration for `resources` table: - - [ ] **B5.2a** [Hamza] Generate migration file: - - [ ] Run `alembic revision --autogenerate -m "create_resources_table"` - - [ ] Commit: "chore(db): generate resources table migration" - - [ ] **B5.2b** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'resources', - sa.Column('resource_id', sa.Text(), nullable=False), - sa.Column('project_id', sa.Text(), nullable=False), - sa.Column('name', sa.Text(), nullable=False), - sa.Column('type', sa.Text(), nullable=False), - sa.Column('location', sa.Text(), nullable=False), - sa.Column('is_remote', sa.Boolean(), nullable=False, server_default='false'), - sa.Column('sandbox_strategy', sa.Text(), nullable=False), - sa.Column('read_only', sa.Boolean(), nullable=False, server_default='false'), - sa.Column('metadata', sa.JSON(), nullable=False, server_default='{}'), - sa.Column('created_at', sa.Text(), nullable=False), - sa.PrimaryKeyConstraint('resource_id'), - sa.ForeignKeyConstraint(['project_id'], ['projects.project_id'], ondelete='CASCADE') - ) - op.create_index('ix_resources_project_id', 'resources', ['project_id']) - op.create_unique_constraint('uq_resources_project_name', 'resources', ['project_id', 'name']) - ``` - - [ ] Commit: "feat(db): add resources table schema with FK to projects" - - [ ] **B5.2c** [Hamza] Add downgrade: - - [ ] Drop constraint, index, and table - - [ ] Commit: "feat(db): add resources table downgrade" - - [ ] **B5.3** [Hamza] Create `ProjectModel` in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **B5.3a** [Hamza] Define SQLAlchemy model: - ```python - class ProjectModel(Base): - __tablename__ = 'projects' + ```python + def upgrade(): + op.create_table( + 'projects', + sa.Column('project_id', sa.Text(), nullable=False), + sa.Column('name', sa.Text(), nullable=False), + sa.Column('namespace', sa.Text(), nullable=False), + sa.Column('short_name', sa.Text(), nullable=False), + sa.Column('description', sa.Text(), nullable=True), + sa.Column('tags', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('is_remote', sa.Boolean(), nullable=False, server_default='false'), + sa.Column('validation_config', sa.JSON(), nullable=True), + sa.Column('context_config', sa.JSON(), nullable=True), + sa.Column('created_at', sa.Text(), nullable=False), + sa.Column('updated_at', sa.Text(), nullable=False), + sa.PrimaryKeyConstraint('project_id') + ) + op.create_index('ix_projects_name', 'projects', ['name'], unique=True) + op.create_index('ix_projects_namespace', 'projects', ['namespace']) + op.create_index('ix_projects_namespace_short_name', 'projects', ['namespace', 'short_name'], unique=True) + ``` - project_id = Column(Text, primary_key=True) - name = Column(Text, nullable=False, unique=True) - namespace = Column(Text, nullable=False, index=True) - short_name = Column(Text, nullable=False) - description = Column(Text, nullable=True) - tags = Column(JSON, nullable=False, default=list) - is_remote = Column(Boolean, nullable=False, default=False) - validation_config = Column(JSON, nullable=True) - context_config = Column(JSON, nullable=True) - created_at = Column(Text, nullable=False) - updated_at = Column(Text, nullable=False) + - [ ] Commit: "feat(db): add projects table schema" - # Relationship to resources - resources = relationship("ResourceModel", back_populates="project", cascade="all, delete-orphan") - ``` - - [ ] Commit: "feat(db): add ProjectModel SQLAlchemy class" - - [ ] **B5.3b** [Hamza] Add domain conversion methods: - ```python - def to_domain(self) -> Project: - return Project( - project_id=self.project_id, - name=self.name, - namespace=self.namespace, - description=self.description, - tags=self.tags or [], - resources=[r.to_domain() for r in self.resources], - validation_config=ValidationConfig(**self.validation_config) if self.validation_config else None, - context_config=ContextConfig(**self.context_config) if self.context_config else ContextConfig(), - created_at=datetime.fromisoformat(self.created_at), - updated_at=datetime.fromisoformat(self.updated_at), - ) + - [ ] **B5.1c** [Hamza] Add downgrade: - @classmethod - def from_domain(cls, project: Project) -> "ProjectModel": - return cls( - project_id=project.project_id, - name=project.namespaced_name, - namespace=project.namespace, - short_name=project.name, - description=project.description, - tags=project.tags, - is_remote=project.is_remote, - validation_config=project.validation_config.model_dump() if project.validation_config else None, - context_config=project.context_config.model_dump(), - created_at=project.created_at.isoformat(), - updated_at=project.updated_at.isoformat(), - ) - ``` - - [ ] Commit: "feat(db): add ProjectModel domain conversion methods" - - [ ] **B5.4** [Hamza] Create `ResourceModel` in same file: - - [ ] **B5.4a** [Hamza] Define SQLAlchemy model: - ```python - class ResourceModel(Base): - __tablename__ = 'resources' + ```python + def downgrade(): + op.drop_index('ix_projects_namespace_short_name') + op.drop_index('ix_projects_namespace') + op.drop_index('ix_projects_name') + op.drop_table('projects') + ``` - resource_id = Column(Text, primary_key=True) - project_id = Column(Text, ForeignKey('projects.project_id', ondelete='CASCADE'), nullable=False) - name = Column(Text, nullable=False) - type = Column(Text, nullable=False) - location = Column(Text, nullable=False) - is_remote = Column(Boolean, nullable=False, default=False) - sandbox_strategy = Column(Text, nullable=False) - read_only = Column(Boolean, nullable=False, default=False) - metadata = Column(JSON, nullable=False, default=dict) - created_at = Column(Text, nullable=False) + - [ ] Commit: "feat(db): add projects table downgrade" - # Relationship back to project - project = relationship("ProjectModel", back_populates="resources") + - [ ] **B5.2** [Hamza] Create Alembic migration for `resources` table: + - [ ] **B5.2a** [Hamza] Generate migration file: + - [ ] Run `alembic revision --autogenerate -m "create_resources_table"` + - [ ] Commit: "chore(db): generate resources table migration" + - [ ] **B5.2b** [Hamza] Define schema: - __table_args__ = ( - UniqueConstraint('project_id', 'name', name='uq_resources_project_name'), - ) - ``` - - [ ] Commit: "feat(db): add ResourceModel SQLAlchemy class" - - [ ] **B5.4b** [Hamza] Add domain conversion methods: - - [ ] `to_domain()` - Convert to Resource domain model - - [ ] `from_domain()` - Create from Resource domain model - - [ ] Handle enum conversions (ResourceType, SandboxStrategy) - - [ ] Commit: "feat(db): add ResourceModel domain conversion methods" - - [ ] **B5.5** [Hamza] Implement `ProjectRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **B5.5a** [Hamza] Define class with session factory: - ```python - class ProjectRepository: - def __init__(self, session_factory: Callable[[], Session]): - self._session_factory = session_factory - ``` - - [ ] Commit: "feat(repo): add ProjectRepository scaffold" - - [ ] **B5.5b** [Hamza] Implement `create(project: Project) -> Project`: - - [ ] Create ProjectModel from domain - - [ ] Add to session - - [ ] Handle duplicate name: raise `DuplicateProjectError` - - [ ] Commit transaction - - [ ] Return created project - - [ ] Commit: "feat(repo): implement ProjectRepository.create()" - - [ ] **B5.5c** [Hamza] Implement `get_by_id(project_id: str) -> Project | None`: - - [ ] Query by primary key with eager load of resources - - [ ] Convert to domain or return None - - [ ] Commit: "feat(repo): implement ProjectRepository.get_by_id()" - - [ ] **B5.5d** [Hamza] Implement `get_by_name(name: str) -> Project | None`: - - [ ] Query by namespaced name (unique index) - - [ ] Eager load resources - - [ ] Convert to domain - - [ ] Commit: "feat(repo): implement ProjectRepository.get_by_name()" - - [ ] **B5.5e** [Hamza] Implement `get_with_resources(project_id: str) -> Project | None`: - - [ ] Same as get_by_id but ensures resources are loaded - - [ ] Use `options(joinedload(ProjectModel.resources))` - - [ ] Commit: "feat(repo): implement ProjectRepository.get_with_resources()" - - [ ] **B5.5f** [Hamza] Implement `list_all(namespace: str | None = None) -> list[Project]`: - - [ ] Query all projects - - [ ] Filter by namespace if provided - - [ ] Order by namespace ASC, short_name ASC - - [ ] Convert all to domain - - [ ] Commit: "feat(repo): implement ProjectRepository.list_all()" - - [ ] **B5.5g** [Hamza] Implement `update(project: Project) -> Project`: - - [ ] Fetch existing by project_id - - [ ] Update all fields from domain - - [ ] Update `updated_at` to now - - [ ] Commit and return - - [ ] Commit: "feat(repo): implement ProjectRepository.update()" - - [ ] **B5.5h** [Hamza] Implement `delete(project_id: str) -> bool`: - - [ ] Delete project (cascade deletes resources via FK) - - [ ] Return True if deleted - - [ ] Commit: "feat(repo): implement ProjectRepository.delete()" - - [ ] **B5.6** [Hamza] Implement `ResourceRepository` in same file: - - [ ] **B5.6a** [Hamza] Define class: - - [ ] Same pattern as ProjectRepository - - [ ] Commit: "feat(repo): add ResourceRepository scaffold" - - [ ] **B5.6b** [Hamza] Implement `create(resource: Resource, project_id: str) -> Resource`: - - [ ] Create ResourceModel with project_id link - - [ ] Handle duplicate name within project: raise `DuplicateResourceError` - - [ ] Commit: "feat(repo): implement ResourceRepository.create()" - - [ ] **B5.6c** [Hamza] Implement `get_by_project(project_id: str) -> list[Resource]`: - - [ ] Query all resources with given project_id - - [ ] Order by name ASC - - [ ] Commit: "feat(repo): implement ResourceRepository.get_by_project()" - - [ ] **B5.6d** [Hamza] Implement `get_by_name(project_id: str, name: str) -> Resource | None`: - - [ ] Query by unique (project_id, name) pair - - [ ] Commit: "feat(repo): implement ResourceRepository.get_by_name()" - - [ ] **B5.6e** [Hamza] Implement `delete(resource_id: str) -> bool`: - - [ ] Delete resource by ID - - [ ] Commit: "feat(repo): implement ResourceRepository.delete()" - - [ ] Tests: Integration tests for persistence - - [ ] **B5.7** [Rui] Write Behave scenarios in `features/project_persistence.feature`: - - [ ] **B5.7a** [Rui] Project persistence scenarios: - - [ ] Scenario: Create project persists to database - - [ ] Given no project "local/test" exists - - [ ] When I create project "local/test" via ProjectRepository - - [ ] Then querying by name returns the project - - [ ] And project_id is a valid ULID - - [ ] Scenario: Update project persists changes - - [ ] Given project "local/test" exists - - [ ] When I update description to "New description" - - [ ] Then re-querying shows updated description - - [ ] And updated_at has changed - - [ ] Commit: "test(behave): add project persistence scenarios" - - [ ] **B5.7b** [Rui] Resource persistence scenarios: - - [ ] Scenario: Add resource persists and links to project - - [ ] Given project "local/test" exists - - [ ] When I add resource "source" to project - - [ ] Then ResourceRepository.get_by_project() returns the resource - - [ ] And resource.project_id matches the project - - [ ] Scenario: Get project includes all resources - - [ ] Given project "local/test" with 3 resources - - [ ] When I call ProjectRepository.get_with_resources() - - [ ] Then project.resources has 3 items - - [ ] And each resource has correct fields - - [ ] Commit: "test(behave): add resource persistence scenarios" - - [ ] **B5.7c** [Rui] Cascade scenarios: - - [ ] Scenario: Delete project cascades to resources - - [ ] Given project "local/test" with 2 resources - - [ ] When I delete the project - - [ ] Then ResourceRepository.get_by_project() returns empty list - - [ ] And the resource records no longer exist in database - - [ ] Commit: "test(behave): add cascade delete scenarios" - - [ ] **B5.7d** [Rui] Uniqueness scenarios: - - [ ] Scenario: Duplicate project name raises error - - [ ] Given project "local/test" exists - - [ ] When I try to create another "local/test" - - [ ] Then DuplicateProjectError is raised - - [ ] Scenario: Duplicate resource name within project raises error - - [ ] Given project "local/test" has resource "source" - - [ ] When I try to add another resource named "source" - - [ ] Then DuplicateResourceError is raised - - [ ] Scenario: Same resource name in different projects is allowed - - [ ] Given project "local/proj1" has resource "source" - - [ ] When I add resource "source" to project "local/proj2" - - [ ] Then it succeeds without error - - [ ] Commit: "test(behave): add uniqueness constraint scenarios" - - [ ] Method `get_by_project(project_id: str) -> list[Resource]` - - [ ] **B5.5** [Hamza] Create database model classes: - - [ ] `ProjectModel(Base)` with `to_domain()` and `from_domain()` - - [ ] `ResourceModel(Base)` with `to_domain()` and `from_domain()` - - [ ] Tests: Integration tests for persistence - - [ ] **B5.6** [Rui] Write Behave scenarios in `features/project_persistence.feature`: - - [ ] Scenario: Create project persists to database - - [ ] Scenario: Add resource persists and links to project - - [ ] Scenario: Get project includes all resources - - [ ] Scenario: Delete project cascades to resources + ```python + def upgrade(): + op.create_table( + 'resources', + sa.Column('resource_id', sa.Text(), nullable=False), + sa.Column('project_id', sa.Text(), nullable=False), + sa.Column('name', sa.Text(), nullable=False), + sa.Column('type', sa.Text(), nullable=False), + sa.Column('location', sa.Text(), nullable=False), + sa.Column('is_remote', sa.Boolean(), nullable=False, server_default='false'), + sa.Column('sandbox_strategy', sa.Text(), nullable=False), + sa.Column('read_only', sa.Boolean(), nullable=False, server_default='false'), + sa.Column('metadata', sa.JSON(), nullable=False, server_default='{}'), + sa.Column('created_at', sa.Text(), nullable=False), + sa.PrimaryKeyConstraint('resource_id'), + sa.ForeignKeyConstraint(['project_id'], ['projects.project_id'], ondelete='CASCADE') + ) + op.create_index('ix_resources_project_id', 'resources', ['project_id']) + op.create_unique_constraint('uq_resources_project_name', 'resources', ['project_id', 'name']) + ``` + + - [ ] Commit: "feat(db): add resources table schema with FK to projects" + + - [ ] **B5.2c** [Hamza] Add downgrade: + - [ ] Drop constraint, index, and table + - [ ] Commit: "feat(db): add resources table downgrade" + + - [ ] **B5.3** [Hamza] Create `ProjectModel` in `src/cleveragents/infrastructure/database/models.py`: + - [ ] **B5.3a** [Hamza] Define SQLAlchemy model: + + ```python + class ProjectModel(Base): + __tablename__ = 'projects' + + project_id = Column(Text, primary_key=True) + name = Column(Text, nullable=False, unique=True) + namespace = Column(Text, nullable=False, index=True) + short_name = Column(Text, nullable=False) + description = Column(Text, nullable=True) + tags = Column(JSON, nullable=False, default=list) + is_remote = Column(Boolean, nullable=False, default=False) + validation_config = Column(JSON, nullable=True) + context_config = Column(JSON, nullable=True) + created_at = Column(Text, nullable=False) + updated_at = Column(Text, nullable=False) + + # Relationship to resources + resources = relationship("ResourceModel", back_populates="project", cascade="all, delete-orphan") + ``` + + - [ ] Commit: "feat(db): add ProjectModel SQLAlchemy class" + + - [ ] **B5.3b** [Hamza] Add domain conversion methods: + + ```python + def to_domain(self) -> Project: + return Project( + project_id=self.project_id, + name=self.name, + namespace=self.namespace, + description=self.description, + tags=self.tags or [], + resources=[r.to_domain() for r in self.resources], + validation_config=ValidationConfig(**self.validation_config) if self.validation_config else None, + context_config=ContextConfig(**self.context_config) if self.context_config else ContextConfig(), + created_at=datetime.fromisoformat(self.created_at), + updated_at=datetime.fromisoformat(self.updated_at), + ) + + @classmethod + def from_domain(cls, project: Project) -> "ProjectModel": + return cls( + project_id=project.project_id, + name=project.namespaced_name, + namespace=project.namespace, + short_name=project.name, + description=project.description, + tags=project.tags, + is_remote=project.is_remote, + validation_config=project.validation_config.model_dump() if project.validation_config else None, + context_config=project.context_config.model_dump(), + created_at=project.created_at.isoformat(), + updated_at=project.updated_at.isoformat(), + ) + ``` + + - [ ] Commit: "feat(db): add ProjectModel domain conversion methods" + + - [ ] **B5.4** [Hamza] Create `ResourceModel` in same file: + - [ ] **B5.4a** [Hamza] Define SQLAlchemy model: + + ```python + class ResourceModel(Base): + __tablename__ = 'resources' + + resource_id = Column(Text, primary_key=True) + project_id = Column(Text, ForeignKey('projects.project_id', ondelete='CASCADE'), nullable=False) + name = Column(Text, nullable=False) + type = Column(Text, nullable=False) + location = Column(Text, nullable=False) + is_remote = Column(Boolean, nullable=False, default=False) + sandbox_strategy = Column(Text, nullable=False) + read_only = Column(Boolean, nullable=False, default=False) + metadata = Column(JSON, nullable=False, default=dict) + created_at = Column(Text, nullable=False) + + # Relationship back to project + project = relationship("ProjectModel", back_populates="resources") + + __table_args__ = ( + UniqueConstraint('project_id', 'name', name='uq_resources_project_name'), + ) + ``` + + - [ ] Commit: "feat(db): add ResourceModel SQLAlchemy class" + + - [ ] **B5.4b** [Hamza] Add domain conversion methods: + - [ ] `to_domain()` - Convert to Resource domain model + - [ ] `from_domain()` - Create from Resource domain model + - [ ] Handle enum conversions (ResourceType, SandboxStrategy) + - [ ] Commit: "feat(db): add ResourceModel domain conversion methods" + + - [ ] **B5.5** [Hamza] Implement `ProjectRepository` in `src/cleveragents/infrastructure/database/repositories.py`: + - [ ] **B5.5a** [Hamza] Define class with session factory: + + ```python + class ProjectRepository: + def __init__(self, session_factory: Callable[[], Session]): + self._session_factory = session_factory + ``` + + - [ ] Commit: "feat(repo): add ProjectRepository scaffold" + + - [ ] **B5.5b** [Hamza] Implement `create(project: Project) -> Project`: + - [ ] Create ProjectModel from domain + - [ ] Add to session + - [ ] Handle duplicate name: raise `DuplicateProjectError` + - [ ] Commit transaction + - [ ] Return created project + - [ ] Commit: "feat(repo): implement ProjectRepository.create()" + - [ ] **B5.5c** [Hamza] Implement `get_by_id(project_id: str) -> Project | None`: + - [ ] Query by primary key with eager load of resources + - [ ] Convert to domain or return None + - [ ] Commit: "feat(repo): implement ProjectRepository.get_by_id()" + - [ ] **B5.5d** [Hamza] Implement `get_by_name(name: str) -> Project | None`: + - [ ] Query by namespaced name (unique index) + - [ ] Eager load resources + - [ ] Convert to domain + - [ ] Commit: "feat(repo): implement ProjectRepository.get_by_name()" + - [ ] **B5.5e** [Hamza] Implement `get_with_resources(project_id: str) -> Project | None`: + - [ ] Same as get_by_id but ensures resources are loaded + - [ ] Use `options(joinedload(ProjectModel.resources))` + - [ ] Commit: "feat(repo): implement ProjectRepository.get_with_resources()" + - [ ] **B5.5f** [Hamza] Implement `list_all(namespace: str | None = None) -> list[Project]`: + - [ ] Query all projects + - [ ] Filter by namespace if provided + - [ ] Order by namespace ASC, short_name ASC + - [ ] Convert all to domain + - [ ] Commit: "feat(repo): implement ProjectRepository.list_all()" + - [ ] **B5.5g** [Hamza] Implement `update(project: Project) -> Project`: + - [ ] Fetch existing by project_id + - [ ] Update all fields from domain + - [ ] Update `updated_at` to now + - [ ] Commit and return + - [ ] Commit: "feat(repo): implement ProjectRepository.update()" + - [ ] **B5.5h** [Hamza] Implement `delete(project_id: str) -> bool`: + - [ ] Delete project (cascade deletes resources via FK) + - [ ] Return True if deleted + - [ ] Commit: "feat(repo): implement ProjectRepository.delete()" + + - [ ] **B5.6** [Hamza] Implement `ResourceRepository` in same file: + - [ ] **B5.6a** [Hamza] Define class: + - [ ] Same pattern as ProjectRepository + - [ ] Commit: "feat(repo): add ResourceRepository scaffold" + - [ ] **B5.6b** [Hamza] Implement `create(resource: Resource, project_id: str) -> Resource`: + - [ ] Create ResourceModel with project_id link + - [ ] Handle duplicate name within project: raise `DuplicateResourceError` + - [ ] Commit: "feat(repo): implement ResourceRepository.create()" + - [ ] **B5.6c** [Hamza] Implement `get_by_project(project_id: str) -> list[Resource]`: + - [ ] Query all resources with given project_id + - [ ] Order by name ASC + - [ ] Commit: "feat(repo): implement ResourceRepository.get_by_project()" + - [ ] **B5.6d** [Hamza] Implement `get_by_name(project_id: str, name: str) -> Resource | None`: + - [ ] Query by unique (project_id, name) pair + - [ ] Commit: "feat(repo): implement ResourceRepository.get_by_name()" + - [ ] **B5.6e** [Hamza] Implement `delete(resource_id: str) -> bool`: + - [ ] Delete resource by ID + - [ ] Commit: "feat(repo): implement ResourceRepository.delete()" + + - [ ] Tests: Integration tests for persistence + - [ ] **B5.7** [Rui] Write Behave scenarios in `features/project_persistence.feature`: + - [ ] **B5.7a** [Rui] Project persistence scenarios: + - [ ] Scenario: Create project persists to database + - [ ] Given no project "local/test" exists + - [ ] When I create project "local/test" via ProjectRepository + - [ ] Then querying by name returns the project + - [ ] And project_id is a valid ULID + - [ ] Scenario: Update project persists changes + - [ ] Given project "local/test" exists + - [ ] When I update description to "New description" + - [ ] Then re-querying shows updated description + - [ ] And updated_at has changed + - [ ] Commit: "test(behave): add project persistence scenarios" + - [ ] **B5.7b** [Rui] Resource persistence scenarios: + - [ ] Scenario: Add resource persists and links to project + - [ ] Given project "local/test" exists + - [ ] When I add resource "source" to project + - [ ] Then ResourceRepository.get_by_project() returns the resource + - [ ] And resource.project_id matches the project + - [ ] Scenario: Get project includes all resources + - [ ] Given project "local/test" with 3 resources + - [ ] When I call ProjectRepository.get_with_resources() + - [ ] Then project.resources has 3 items + - [ ] And each resource has correct fields + - [ ] Commit: "test(behave): add resource persistence scenarios" + - [ ] **B5.7c** [Rui] Cascade scenarios: + - [ ] Scenario: Delete project cascades to resources + - [ ] Given project "local/test" with 2 resources + - [ ] When I delete the project + - [ ] Then ResourceRepository.get_by_project() returns empty list + - [ ] And the resource records no longer exist in database + - [ ] Commit: "test(behave): add cascade delete scenarios" + - [ ] **B5.7d** [Rui] Uniqueness scenarios: + - [ ] Scenario: Duplicate project name raises error + - [ ] Given project "local/test" exists + - [ ] When I try to create another "local/test" + - [ ] Then DuplicateProjectError is raised + - [ ] Scenario: Duplicate resource name within project raises error + - [ ] Given project "local/test" has resource "source" + - [ ] When I try to add another resource named "source" + - [ ] Then DuplicateResourceError is raised + - [ ] Scenario: Same resource name in different projects is allowed + - [ ] Given project "local/proj1" has resource "source" + - [ ] When I add resource "source" to project "local/proj2" + - [ ] Then it succeeds without error + - [ ] Commit: "test(behave): add uniqueness constraint scenarios" + - [ ] Method `get_by_project(project_id: str) -> list[Resource]` + - [ ] **B5.5** [Hamza] Create database model classes: + - [ ] `ProjectModel(Base)` with `to_domain()` and `from_domain()` + - [ ] `ResourceModel(Base)` with `to_domain()` and `from_domain()` + - [ ] Tests: Integration tests for persistence + - [ ] **B5.6** [Rui] Write Behave scenarios in `features/project_persistence.feature`: + - [ ] Scenario: Create project persists to database + - [ ] Scenario: Add resource persists and links to project + - [ ] Scenario: Get project includes all resources + - [ ] Scenario: Delete project cascades to resources **M2 SUCCESS CRITERIA**: + - [ ] Can create a project with resources via CLI - [ ] Git repository resources can be sandboxed with worktrees - [ ] Filesystem resources can be sandboxed with copy-on-write @@ -3476,1343 +3593,1436 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation - [ ] **Stage C1: Actor YAML Schema Formalization** (Day 5-6) **[Aditya - Domain Expert]** - **SEQUENTIAL ORDER**: C1.1 (Enums) → C1.2 (Tool/Route models) → C1.3 (Context model) → C1.4 (ActorConfigSchema) → C1.5 (Examples) → C1.6 (Docs) → C1.7 (Tests) + **SEQUENTIAL ORDER**: C1.1 (Enums) → C1.2 (Tool/Route models) → C1.3 (Context model) → C1.4 (ActorConfigSchema) → C1.5 (Examples) → C1.6 (Docs) → C1.7 (Tests) + - [ ] Code: Formalize actor YAML schema + - [ ] **C1.1** [Aditya] Define core enums in `src/cleveragents/actor/schema.py`: + - [ ] **C1.1a** [Aditya] Create file with `ActorType` enum: - - [ ] Code: Formalize actor YAML schema - - [ ] **C1.1** [Aditya] Define core enums in `src/cleveragents/actor/schema.py`: - - [ ] **C1.1a** [Aditya] Create file with `ActorType` enum: - ```python - class ActorType(str, Enum): - """Type of actor determining execution behavior.""" - LLM = "llm" # Single LLM with system prompt - TOOL = "tool" # Collection of callable tools - GRAPH = "graph" # Multi-node StateGraph with routing - ``` - - [ ] Commit: "feat(actor): define ActorType enum" - - [ ] **C1.1b** [Aditya] Define `NodeType` enum: - ```python - class NodeType(str, Enum): - """Type of node in a graph actor.""" - AGENT = "agent" # LLM agent node - TOOL = "tool" # Tool execution node - CONDITIONAL = "conditional" # Routing/conditional node - SUBGRAPH = "subgraph" # Nested actor reference - ``` - - [ ] Commit: "feat(actor): define NodeType enum" - - [ ] **C1.1c** [Aditya] Define `ContextView` enum: - ```python - class ContextView(str, Enum): - """Role-based context filtering for actors.""" - STRATEGIST = "strategist" # High-level architecture, READMEs - EXECUTOR = "executor" # Precise code sections for edits - REVIEWER = "reviewer" # Diffs, tests, style guides - FULL = "full" # Complete context (default) - ``` - - [ ] Commit: "feat(actor): define ContextView enum" - - [ ] **C1.2** [Aditya] Define tool and route models: - - [ ] **C1.2a** [Aditya] Define `ToolParameter` model: - ```python - class ToolParameter(BaseModel): - """Parameter definition for inline tool.""" - name: str = Field(..., description="Parameter name") - type: str = Field(..., description="JSON Schema type (string, integer, object, etc.)") - description: str = Field(..., description="What this parameter is for") - required: bool = Field(default=True) - default: Any = Field(default=None) - enum: list[str] | None = Field(default=None, description="Allowed values") - ``` - - [ ] Commit: "feat(actor): define ToolParameter model" - - [ ] **C1.2b** [Aditya] Define `ToolDefinition` model: - ```python - class ToolDefinition(BaseModel): - """Inline tool/skill definition in actor YAML.""" - name: str = Field(..., min_length=1, max_length=100, description="Tool identifier") - description: str = Field(..., description="What the tool does (shown to LLM)") - parameters: list[ToolParameter] = Field(default_factory=list) - returns: str = Field(default="Any", description="Return type documentation") - code: str = Field(..., description="Python code to execute") - timeout_seconds: int = Field(default=30, ge=1, le=300) + ```python + class ActorType(str, Enum): + """Type of actor determining execution behavior.""" + LLM = "llm" # Single LLM with system prompt + TOOL = "tool" # Collection of callable tools + GRAPH = "graph" # Multi-node StateGraph with routing + ``` - @field_validator('code') - @classmethod - def validate_code_syntax(cls, v: str) -> str: - """Validate Python syntax without executing.""" - try: - compile(v, '', 'exec') - except SyntaxError as e: - raise ValueError(f"Invalid Python syntax: {e}") - return v - ``` - - [ ] Commit: "feat(actor): define ToolDefinition model with code validation" - - [ ] **C1.2c** [Aditya] Define `EdgeDefinition` model: - ```python - class EdgeDefinition(BaseModel): - """Edge in actor graph topology.""" - source: str = Field(..., description="Source node name") - target: str = Field(..., description="Target node name") - condition: str | None = Field(default=None, description="Python expression for conditional routing") - label: str | None = Field(default=None, description="Edge label for visualization") - ``` - - [ ] Commit: "feat(actor): define EdgeDefinition model" - - [ ] **C1.2d** [Aditya] Define `NodeDefinition` model: - ```python - class NodeDefinition(BaseModel): - """Node in actor graph.""" - name: str = Field(..., description="Unique node identifier") - type: NodeType = Field(..., description="Type of node") - # For agent nodes: - model: str | None = Field(default=None, description="Model name for agent nodes") - system_prompt: str | None = Field(default=None) - tools: list[str] | None = Field(default=None, description="Tool names available to this agent") - # For tool nodes: - tool: str | None = Field(default=None, description="Tool name to execute") - # For subgraph nodes: - actor: str | None = Field(default=None, description="Actor reference (e.g., local/other-actor)") - ``` - - [ ] Commit: "feat(actor): define NodeDefinition model" - - [ ] **C1.2e** [Aditya] Define `RouteDefinition` model: - ```python - class RouteDefinition(BaseModel): - """Complete graph topology definition.""" - nodes: list[NodeDefinition] = Field(..., min_length=1) - edges: list[EdgeDefinition] = Field(default_factory=list) - entry_point: str = Field(..., description="Name of starting node") + - [ ] Commit: "feat(actor): define ActorType enum" - @model_validator(mode='after') - def validate_topology(self) -> Self: - """Validate graph is well-formed.""" - node_names = {n.name for n in self.nodes} - if self.entry_point not in node_names: - raise ValueError(f"entry_point '{self.entry_point}' not in nodes") - for edge in self.edges: - if edge.source not in node_names: - raise ValueError(f"Edge source '{edge.source}' not in nodes") - if edge.target not in node_names: - raise ValueError(f"Edge target '{edge.target}' not in nodes") - return self - ``` - - [ ] Commit: "feat(actor): define RouteDefinition with topology validation" - - [ ] **C1.3** [Aditya] Define context/memory configuration: - - [ ] **C1.3a** [Aditya] Define `MemoryConfig` model: - ```python - class MemoryConfig(BaseModel): - """Memory/conversation history settings.""" - enabled: bool = Field(default=True, description="Whether to maintain history") - max_turns: int = Field(default=20, ge=1, le=100, description="Max conversation turns") - summarization_threshold: int = Field(default=15, description="Turns before summarizing") - include_system_messages: bool = Field(default=True) - ``` - - [ ] Commit: "feat(actor): define MemoryConfig model" - - [ ] **C1.3b** [Aditya] Define `ContextConfigSchema` model: - ```python - class ContextConfigSchema(BaseModel): - """Context window configuration for actor.""" - context_window_fraction: float = Field(default=0.8, ge=0.1, le=1.0, - description="Fraction of model's context window to use") - context_view: ContextView = Field(default=ContextView.FULL, - description="Role-based context filtering") - include_file_patterns: list[str] = Field(default_factory=list, - description="Glob patterns for files to always include") - exclude_file_patterns: list[str] = Field(default_factory=list, - description="Glob patterns for files to never include") - max_file_size_kb: int = Field(default=100, description="Max file size to include") - ``` - - [ ] Commit: "feat(actor): define ContextConfigSchema model" - - [ ] **C1.4** [Aditya] Define main `ActorConfigSchema`: - - [ ] **C1.4a** [Aditya] Create comprehensive model: - ```python - class ActorConfigSchema(BaseModel): - """Complete actor configuration from YAML.""" - # Identity - version: str = Field(default="3", description="Config schema version") - name: str = Field(..., description="Actor name (without namespace)") - namespace: str = Field(default="local") - description: str | None = Field(default=None) - tags: list[str] = Field(default_factory=list) + - [ ] **C1.1b** [Aditya] Define `NodeType` enum: - # Type and provider - type: ActorType = Field(..., description="Actor type") - model: str | None = Field(default=None, description="LLM model name") - provider: str | None = Field(default=None, description="Provider: openai, anthropic, etc.") + ```python + class NodeType(str, Enum): + """Type of node in a graph actor.""" + AGENT = "agent" # LLM agent node + TOOL = "tool" # Tool execution node + CONDITIONAL = "conditional" # Routing/conditional node + SUBGRAPH = "subgraph" # Nested actor reference + ``` - # LLM configuration - system_prompt: str | None = Field(default=None) - temperature: float = Field(default=0.7, ge=0.0, le=2.0) - max_tokens: int | None = Field(default=None) + - [ ] Commit: "feat(actor): define NodeType enum" - # Tools/skills - tools: list[ToolDefinition] = Field(default_factory=list, - description="Inline tool definitions") - builtin_tools: list[str] = Field(default_factory=list, - description="Names of built-in tools to include") - mcp_servers: list[str] = Field(default_factory=list, - description="MCP server identifiers to connect") + - [ ] **C1.1c** [Aditya] Define `ContextView` enum: - # Graph topology (for type=GRAPH) - routes: RouteDefinition | None = Field(default=None) + ```python + class ContextView(str, Enum): + """Role-based context filtering for actors.""" + STRATEGIST = "strategist" # High-level architecture, READMEs + EXECUTOR = "executor" # Precise code sections for edits + REVIEWER = "reviewer" # Diffs, tests, style guides + FULL = "full" # Complete context (default) + ``` - # Memory and context - memory: MemoryConfig = Field(default_factory=MemoryConfig) - context: ContextConfigSchema = Field(default_factory=ContextConfigSchema) + - [ ] Commit: "feat(actor): define ContextView enum" - # Execution - timeout_seconds: int = Field(default=300, description="Total execution timeout") - max_iterations: int = Field(default=50, description="Max LLM calls per invocation") + - [ ] **C1.2** [Aditya] Define tool and route models: + - [ ] **C1.2a** [Aditya] Define `ToolParameter` model: - @model_validator(mode='after') - def validate_type_requirements(self) -> Self: - """Validate fields based on actor type.""" - if self.type == ActorType.LLM: - if not self.model: - raise ValueError("LLM actors require 'model' field") - if self.type == ActorType.GRAPH: - if not self.routes: - raise ValueError("GRAPH actors require 'routes' field") - return self + ```python + class ToolParameter(BaseModel): + """Parameter definition for inline tool.""" + name: str = Field(..., description="Parameter name") + type: str = Field(..., description="JSON Schema type (string, integer, object, etc.)") + description: str = Field(..., description="What this parameter is for") + required: bool = Field(default=True) + default: Any = Field(default=None) + enum: list[str] | None = Field(default=None, description="Allowed values") + ``` - model_config = ConfigDict(extra='forbid') # Reject unknown fields - ``` - - [ ] Commit: "feat(actor): define ActorConfigSchema with validation" - - [ ] **C1.4b** [Aditya] Add YAML loading helper: - ```python - @classmethod - def from_yaml(cls, path: Path | str) -> "ActorConfigSchema": - """Load and validate actor config from YAML file.""" - import yaml - with open(path) as f: - data = yaml.safe_load(f) - return cls.model_validate(data) + - [ ] Commit: "feat(actor): define ToolParameter model" - def to_yaml(self) -> str: - """Serialize config to YAML string.""" - import yaml - return yaml.dump(self.model_dump(exclude_none=True), sort_keys=False) - ``` - - [ ] Commit: "feat(actor): add YAML serialization helpers" - - [ ] **C1.5** [Aditya] Create comprehensive example actors in `examples/actors/`: - - [ ] **C1.5a** [Aditya] Create `simple_llm_actor.yaml`: - ```yaml - version: "3" - name: simple-assistant - namespace: local - description: Basic LLM assistant with no tools - type: llm - model: gpt-4-turbo - provider: openai - system_prompt: | - You are a helpful coding assistant. Answer questions - concisely and provide code examples when appropriate. - temperature: 0.7 - memory: - enabled: true - max_turns: 10 - ``` - - [ ] Commit: "docs(examples): add simple_llm_actor.yaml" - - [ ] **C1.5b** [Aditya] Create `tool_actor.yaml`: - ```yaml - version: "3" - name: file-reader - namespace: local - description: Actor that can read and search files - type: llm - model: gpt-4-turbo - system_prompt: | - You can read and search files to answer questions. - tools: - - name: read_file - description: Read contents of a file - parameters: - - name: path - type: string - description: Path to file - code: | - result = context.get_file(input_data["path"]) - - name: search_files - description: Search for pattern in files - parameters: - - name: pattern - type: string - description: Regex pattern to search - code: | - result = context.search_files("**/*", input_data["pattern"]) - builtin_tools: - - list_directory - ``` - - [ ] Commit: "docs(examples): add tool_actor.yaml" - - [ ] **C1.5c** [Aditya] Create `graph_actor.yaml`: - ```yaml - version: "3" - name: research-writer - namespace: local - description: Multi-step research and writing workflow - type: graph - routes: - entry_point: planner - nodes: - - name: planner - type: agent - model: gpt-4-turbo - system_prompt: Break down the writing task into research topics. - - name: researcher - type: agent - model: gpt-4-turbo - system_prompt: Research the assigned topic thoroughly. - tools: [search_files, read_file] - - name: writer - type: agent - model: gpt-4-turbo - system_prompt: Write content based on research findings. - - name: router - type: conditional - edges: - - source: planner - target: router - - source: router - target: researcher - condition: "state.needs_research" - - source: router - target: writer - condition: "not state.needs_research" - - source: researcher - target: router - ``` - - [ ] Commit: "docs(examples): add graph_actor.yaml" - - [ ] **C1.5d** [Aditya] Create `hierarchical_actor.yaml`: - ```yaml - version: "3" - name: code-reviewer - namespace: local - description: Hierarchical actor that delegates to specialists - type: graph - routes: - entry_point: coordinator - nodes: - - name: coordinator - type: agent - model: gpt-4-turbo - system_prompt: | - Coordinate code review by delegating to specialists. - - name: security-check - type: subgraph - actor: local/security-analyzer - - name: style-check - type: subgraph - actor: local/style-checker - - name: aggregator - type: agent - model: gpt-4-turbo - system_prompt: Combine specialist feedback into final review. - edges: - - source: coordinator - target: security-check - - source: coordinator - target: style-check - - source: security-check - target: aggregator - - source: style-check - target: aggregator - ``` - - [ ] Commit: "docs(examples): add hierarchical_actor.yaml" - - [ ] **C1.5e** [Aditya] Create `strategy_actor.yaml`: - ```yaml - version: "3" - name: default-strategist - namespace: cleveragents - description: Default strategist for Strategize phase - type: llm - model: gpt-4-turbo - temperature: 0.3 # Lower for more consistent strategy - system_prompt: | - You are a technical strategist. Given a task description and codebase context, - create a detailed plan with specific steps. + - [ ] **C1.2b** [Aditya] Define `ToolDefinition` model: - Your output must include: - 1. High-level approach explanation - 2. Ordered list of steps with file paths - 3. Dependencies between steps - 4. Risk assessment - 5. Decisions with alternatives considered + ```python + class ToolDefinition(BaseModel): + """Inline tool/skill definition in actor YAML.""" + name: str = Field(..., min_length=1, max_length=100, description="Tool identifier") + description: str = Field(..., description="What the tool does (shown to LLM)") + parameters: list[ToolParameter] = Field(default_factory=list) + returns: str = Field(default="Any", description="Return type documentation") + code: str = Field(..., description="Python code to execute") + timeout_seconds: int = Field(default=30, ge=1, le=300) - Format decisions as: - DECISION: - CHOSEN: - ALTERNATIVES: - RATIONALE: - context: - context_view: strategist - include_file_patterns: - - "README.md" - - "**/README.md" - - "docs/**/*.md" - ``` - - [ ] Commit: "docs(examples): add strategy_actor.yaml" - - [ ] **C1.5f** [Aditya] Create `execution_actor.yaml`: - ```yaml - version: "3" - name: default-executor - namespace: cleveragents - description: Default executor for Execute phase - type: llm - model: gpt-4-turbo - temperature: 0.2 # Low for precise code generation - system_prompt: | - You are a code executor. Given a strategy and file context, - implement the required changes using the provided tools. + @field_validator('code') + @classmethod + def validate_code_syntax(cls, v: str) -> str: + """Validate Python syntax without executing.""" + try: + compile(v, '', 'exec') + except SyntaxError as e: + raise ValueError(f"Invalid Python syntax: {e}") + return v + ``` - RULES: - - Use tools to read files before editing - - Use edit_file for targeted changes, write_file for new files - - Always verify changes compile/parse correctly - - Document each change with clear commit messages - tools: - - name: edit_file - description: Make targeted edits to an existing file - parameters: - - name: path - type: string - - name: edits - type: array - code: | - result = context.edit_file(input_data["path"], input_data["edits"]) - builtin_tools: - - read_file - - write_file - - delete_file - - list_directory - - search_files - context: - context_view: executor - max_iterations: 100 # Allow more iterations for complex tasks - ``` - - [ ] Commit: "docs(examples): add execution_actor.yaml" - - [ ] **C1.6** [Aditya] Write comprehensive documentation: - - [ ] **C1.6a** [Aditya] Create `docs/reference/actor_configuration.md`: - - [ ] Full YAML schema reference with all fields - - [ ] Type-specific requirements (LLM vs GRAPH) - - [ ] Tool definition syntax and examples - - [ ] Memory and context configuration - - [ ] Commit: "docs: add actor configuration reference" - - [ ] **C1.6b** [Aditya] Add examples section: - - [ ] Example for each actor type - - [ ] Common patterns (research, review, generation) - - [ ] Anti-patterns to avoid - - [ ] Commit: "docs: add actor configuration examples" - - [ ] **C1.6c** [Aditya] Add migration guide: - - [ ] Changes from v2 format - - [ ] Automated migration script (if needed) - - [ ] Commit: "docs: add actor config migration guide" - - [ ] Tests: Behave scenarios for actor schema validation - - [ ] **C1.7** [Rui] Write Behave scenarios in `features/actor_schema.feature`: - - [ ] **C1.7a** [Rui] Valid config scenarios: - - [ ] Scenario: Simple LLM actor config validates successfully - - [ ] Scenario: Tool actor with inline code validates - - [ ] Scenario: Graph actor with complete topology validates - - [ ] Scenario: Actor with MCP servers configured validates - - [ ] Commit: "test(behave): add valid actor config scenarios" - - [ ] **C1.7b** [Rui] Invalid config scenarios: - - [ ] Scenario: LLM actor without model field fails - - [ ] Scenario: Graph actor without routes fails - - [ ] Scenario: Tool with invalid Python syntax fails - - [ ] Scenario: Graph with missing entry_point fails - - [ ] Scenario: Edge referencing non-existent node fails - - [ ] Commit: "test(behave): add invalid actor config scenarios" - - [ ] **C1.7c** [Rui] YAML loading scenarios: - - [ ] Scenario: Load actor config from YAML file - - [ ] Scenario: Invalid YAML syntax produces clear error - - [ ] Scenario: Unknown fields in YAML are rejected - - [ ] Commit: "test(behave): add YAML loading scenarios" + - [ ] Commit: "feat(actor): define ToolDefinition model with code validation" + + - [ ] **C1.2c** [Aditya] Define `EdgeDefinition` model: + + ```python + class EdgeDefinition(BaseModel): + """Edge in actor graph topology.""" + source: str = Field(..., description="Source node name") + target: str = Field(..., description="Target node name") + condition: str | None = Field(default=None, description="Python expression for conditional routing") + label: str | None = Field(default=None, description="Edge label for visualization") + ``` + + - [ ] Commit: "feat(actor): define EdgeDefinition model" + + - [ ] **C1.2d** [Aditya] Define `NodeDefinition` model: + + ```python + class NodeDefinition(BaseModel): + """Node in actor graph.""" + name: str = Field(..., description="Unique node identifier") + type: NodeType = Field(..., description="Type of node") + # For agent nodes: + model: str | None = Field(default=None, description="Model name for agent nodes") + system_prompt: str | None = Field(default=None) + tools: list[str] | None = Field(default=None, description="Tool names available to this agent") + # For tool nodes: + tool: str | None = Field(default=None, description="Tool name to execute") + # For subgraph nodes: + actor: str | None = Field(default=None, description="Actor reference (e.g., local/other-actor)") + ``` + + - [ ] Commit: "feat(actor): define NodeDefinition model" + + - [ ] **C1.2e** [Aditya] Define `RouteDefinition` model: + + ```python + class RouteDefinition(BaseModel): + """Complete graph topology definition.""" + nodes: list[NodeDefinition] = Field(..., min_length=1) + edges: list[EdgeDefinition] = Field(default_factory=list) + entry_point: str = Field(..., description="Name of starting node") + + @model_validator(mode='after') + def validate_topology(self) -> Self: + """Validate graph is well-formed.""" + node_names = {n.name for n in self.nodes} + if self.entry_point not in node_names: + raise ValueError(f"entry_point '{self.entry_point}' not in nodes") + for edge in self.edges: + if edge.source not in node_names: + raise ValueError(f"Edge source '{edge.source}' not in nodes") + if edge.target not in node_names: + raise ValueError(f"Edge target '{edge.target}' not in nodes") + return self + ``` + + - [ ] Commit: "feat(actor): define RouteDefinition with topology validation" + + - [ ] **C1.3** [Aditya] Define context/memory configuration: + - [ ] **C1.3a** [Aditya] Define `MemoryConfig` model: + + ```python + class MemoryConfig(BaseModel): + """Memory/conversation history settings.""" + enabled: bool = Field(default=True, description="Whether to maintain history") + max_turns: int = Field(default=20, ge=1, le=100, description="Max conversation turns") + summarization_threshold: int = Field(default=15, description="Turns before summarizing") + include_system_messages: bool = Field(default=True) + ``` + + - [ ] Commit: "feat(actor): define MemoryConfig model" + + - [ ] **C1.3b** [Aditya] Define `ContextConfigSchema` model: + + ```python + class ContextConfigSchema(BaseModel): + """Context window configuration for actor.""" + context_window_fraction: float = Field(default=0.8, ge=0.1, le=1.0, + description="Fraction of model's context window to use") + context_view: ContextView = Field(default=ContextView.FULL, + description="Role-based context filtering") + include_file_patterns: list[str] = Field(default_factory=list, + description="Glob patterns for files to always include") + exclude_file_patterns: list[str] = Field(default_factory=list, + description="Glob patterns for files to never include") + max_file_size_kb: int = Field(default=100, description="Max file size to include") + ``` + + - [ ] Commit: "feat(actor): define ContextConfigSchema model" + + - [ ] **C1.4** [Aditya] Define main `ActorConfigSchema`: + - [ ] **C1.4a** [Aditya] Create comprehensive model: + + ```python + class ActorConfigSchema(BaseModel): + """Complete actor configuration from YAML.""" + # Identity + version: str = Field(default="3", description="Config schema version") + name: str = Field(..., description="Actor name (without namespace)") + namespace: str = Field(default="local") + description: str | None = Field(default=None) + tags: list[str] = Field(default_factory=list) + + # Type and provider + type: ActorType = Field(..., description="Actor type") + model: str | None = Field(default=None, description="LLM model name") + provider: str | None = Field(default=None, description="Provider: openai, anthropic, etc.") + + # LLM configuration + system_prompt: str | None = Field(default=None) + temperature: float = Field(default=0.7, ge=0.0, le=2.0) + max_tokens: int | None = Field(default=None) + + # Tools/skills + tools: list[ToolDefinition] = Field(default_factory=list, + description="Inline tool definitions") + builtin_tools: list[str] = Field(default_factory=list, + description="Names of built-in tools to include") + mcp_servers: list[str] = Field(default_factory=list, + description="MCP server identifiers to connect") + + # Graph topology (for type=GRAPH) + routes: RouteDefinition | None = Field(default=None) + + # Memory and context + memory: MemoryConfig = Field(default_factory=MemoryConfig) + context: ContextConfigSchema = Field(default_factory=ContextConfigSchema) + + # Execution + timeout_seconds: int = Field(default=300, description="Total execution timeout") + max_iterations: int = Field(default=50, description="Max LLM calls per invocation") + + @model_validator(mode='after') + def validate_type_requirements(self) -> Self: + """Validate fields based on actor type.""" + if self.type == ActorType.LLM: + if not self.model: + raise ValueError("LLM actors require 'model' field") + if self.type == ActorType.GRAPH: + if not self.routes: + raise ValueError("GRAPH actors require 'routes' field") + return self + + model_config = ConfigDict(extra='forbid') # Reject unknown fields + ``` + + - [ ] Commit: "feat(actor): define ActorConfigSchema with validation" + + - [ ] **C1.4b** [Aditya] Add YAML loading helper: + + ```python + @classmethod + def from_yaml(cls, path: Path | str) -> "ActorConfigSchema": + """Load and validate actor config from YAML file.""" + import yaml + with open(path) as f: + data = yaml.safe_load(f) + return cls.model_validate(data) + + def to_yaml(self) -> str: + """Serialize config to YAML string.""" + import yaml + return yaml.dump(self.model_dump(exclude_none=True), sort_keys=False) + ``` + + - [ ] Commit: "feat(actor): add YAML serialization helpers" + + - [ ] **C1.5** [Aditya] Create comprehensive example actors in `examples/actors/`: + - [ ] **C1.5a** [Aditya] Create `simple_llm_actor.yaml`: + + ```yaml + version: "3" + name: simple-assistant + namespace: local + description: Basic LLM assistant with no tools + type: llm + model: gpt-4-turbo + provider: openai + system_prompt: | + You are a helpful coding assistant. Answer questions + concisely and provide code examples when appropriate. + temperature: 0.7 + memory: + enabled: true + max_turns: 10 + ``` + + - [ ] Commit: "docs(examples): add simple_llm_actor.yaml" + + - [ ] **C1.5b** [Aditya] Create `tool_actor.yaml`: + + ```yaml + version: "3" + name: file-reader + namespace: local + description: Actor that can read and search files + type: llm + model: gpt-4-turbo + system_prompt: | + You can read and search files to answer questions. + tools: + - name: read_file + description: Read contents of a file + parameters: + - name: path + type: string + description: Path to file + code: | + result = context.get_file(input_data["path"]) + - name: search_files + description: Search for pattern in files + parameters: + - name: pattern + type: string + description: Regex pattern to search + code: | + result = context.search_files("**/*", input_data["pattern"]) + builtin_tools: + - list_directory + ``` + + - [ ] Commit: "docs(examples): add tool_actor.yaml" + + - [ ] **C1.5c** [Aditya] Create `graph_actor.yaml`: + + ```yaml + version: "3" + name: research-writer + namespace: local + description: Multi-step research and writing workflow + type: graph + routes: + entry_point: planner + nodes: + - name: planner + type: agent + model: gpt-4-turbo + system_prompt: Break down the writing task into research topics. + - name: researcher + type: agent + model: gpt-4-turbo + system_prompt: Research the assigned topic thoroughly. + tools: [search_files, read_file] + - name: writer + type: agent + model: gpt-4-turbo + system_prompt: Write content based on research findings. + - name: router + type: conditional + edges: + - source: planner + target: router + - source: router + target: researcher + condition: "state.needs_research" + - source: router + target: writer + condition: "not state.needs_research" + - source: researcher + target: router + ``` + + - [ ] Commit: "docs(examples): add graph_actor.yaml" + + - [ ] **C1.5d** [Aditya] Create `hierarchical_actor.yaml`: + + ```yaml + version: "3" + name: code-reviewer + namespace: local + description: Hierarchical actor that delegates to specialists + type: graph + routes: + entry_point: coordinator + nodes: + - name: coordinator + type: agent + model: gpt-4-turbo + system_prompt: | + Coordinate code review by delegating to specialists. + - name: security-check + type: subgraph + actor: local/security-analyzer + - name: style-check + type: subgraph + actor: local/style-checker + - name: aggregator + type: agent + model: gpt-4-turbo + system_prompt: Combine specialist feedback into final review. + edges: + - source: coordinator + target: security-check + - source: coordinator + target: style-check + - source: security-check + target: aggregator + - source: style-check + target: aggregator + ``` + + - [ ] Commit: "docs(examples): add hierarchical_actor.yaml" + + - [ ] **C1.5e** [Aditya] Create `strategy_actor.yaml`: + + ```yaml + version: "3" + name: default-strategist + namespace: cleveragents + description: Default strategist for Strategize phase + type: llm + model: gpt-4-turbo + temperature: 0.3 # Lower for more consistent strategy + system_prompt: | + You are a technical strategist. Given a task description and codebase context, + create a detailed plan with specific steps. + + Your output must include: + 1. High-level approach explanation + 2. Ordered list of steps with file paths + 3. Dependencies between steps + 4. Risk assessment + 5. Decisions with alternatives considered + + Format decisions as: + DECISION: + CHOSEN: + ALTERNATIVES: + RATIONALE: + context: + context_view: strategist + include_file_patterns: + - "README.md" + - "**/README.md" + - "docs/**/*.md" + ``` + + - [ ] Commit: "docs(examples): add strategy_actor.yaml" + + - [ ] **C1.5f** [Aditya] Create `execution_actor.yaml`: + + ```yaml + version: "3" + name: default-executor + namespace: cleveragents + description: Default executor for Execute phase + type: llm + model: gpt-4-turbo + temperature: 0.2 # Low for precise code generation + system_prompt: | + You are a code executor. Given a strategy and file context, + implement the required changes using the provided tools. + + RULES: + - Use tools to read files before editing + - Use edit_file for targeted changes, write_file for new files + - Always verify changes compile/parse correctly + - Document each change with clear commit messages + tools: + - name: edit_file + description: Make targeted edits to an existing file + parameters: + - name: path + type: string + - name: edits + type: array + code: | + result = context.edit_file(input_data["path"], input_data["edits"]) + builtin_tools: + - read_file + - write_file + - delete_file + - list_directory + - search_files + context: + context_view: executor + max_iterations: 100 # Allow more iterations for complex tasks + ``` + + - [ ] Commit: "docs(examples): add execution_actor.yaml" + + - [ ] **C1.6** [Aditya] Write comprehensive documentation: + - [ ] **C1.6a** [Aditya] Create `docs/reference/actor_configuration.md`: + - [ ] Full YAML schema reference with all fields + - [ ] Type-specific requirements (LLM vs GRAPH) + - [ ] Tool definition syntax and examples + - [ ] Memory and context configuration + - [ ] Commit: "docs: add actor configuration reference" + - [ ] **C1.6b** [Aditya] Add examples section: + - [ ] Example for each actor type + - [ ] Common patterns (research, review, generation) + - [ ] Anti-patterns to avoid + - [ ] Commit: "docs: add actor configuration examples" + - [ ] **C1.6c** [Aditya] Add migration guide: + - [ ] Changes from v2 format + - [ ] Automated migration script (if needed) + - [ ] Commit: "docs: add actor config migration guide" + + - [ ] Tests: Behave scenarios for actor schema validation + - [ ] **C1.7** [Rui] Write Behave scenarios in `features/actor_schema.feature`: + - [ ] **C1.7a** [Rui] Valid config scenarios: + - [ ] Scenario: Simple LLM actor config validates successfully + - [ ] Scenario: Tool actor with inline code validates + - [ ] Scenario: Graph actor with complete topology validates + - [ ] Scenario: Actor with MCP servers configured validates + - [ ] Commit: "test(behave): add valid actor config scenarios" + - [ ] **C1.7b** [Rui] Invalid config scenarios: + - [ ] Scenario: LLM actor without model field fails + - [ ] Scenario: Graph actor without routes fails + - [ ] Scenario: Tool with invalid Python syntax fails + - [ ] Scenario: Graph with missing entry_point fails + - [ ] Scenario: Edge referencing non-existent node fails + - [ ] Commit: "test(behave): add invalid actor config scenarios" + - [ ] **C1.7c** [Rui] YAML loading scenarios: + - [ ] Scenario: Load actor config from YAML file + - [ ] Scenario: Invalid YAML syntax produces clear error + - [ ] Scenario: Unknown fields in YAML are rejected + - [ ] Commit: "test(behave): add YAML loading scenarios" - [ ] **Stage C2: Actor Loading & Compilation** (Day 6-8) **[Aditya]** - **SEQUENTIAL ORDER**: C2.1 (Config parser) → C2.2 (CompiledActor) → C2.3 (LLM compiler) → C2.4 (Tool compiler) → C2.5 (Graph compiler) → C2.6 (Reference resolution) → C2.7 (Registry) → C2.8 (Tests) + **SEQUENTIAL ORDER**: C2.1 (Config parser) → C2.2 (CompiledActor) → C2.3 (LLM compiler) → C2.4 (Tool compiler) → C2.5 (Graph compiler) → C2.6 (Reference resolution) → C2.7 (Registry) → C2.8 (Tests) + - [ ] Code: Enhance actor loading and compilation to LangGraph + - [ ] **C2.1** [Aditya] Create config parser in `src/cleveragents/actor/config.py`: + - [ ] **C2.1a** [Aditya] Define `ActorConfigParser` class: - - [ ] Code: Enhance actor loading and compilation to LangGraph - - [ ] **C2.1** [Aditya] Create config parser in `src/cleveragents/actor/config.py`: - - [ ] **C2.1a** [Aditya] Define `ActorConfigParser` class: - ```python - class ActorConfigParser: - """Parse and validate actor configurations.""" + ```python + class ActorConfigParser: + """Parse and validate actor configurations.""" - def __init__(self, registry: "ActorRegistry"): - self._registry = registry + def __init__(self, registry: "ActorRegistry"): + self._registry = registry - def parse_file(self, path: Path) -> ActorConfigSchema: - """Parse actor config from YAML file.""" - return ActorConfigSchema.from_yaml(path) + def parse_file(self, path: Path) -> ActorConfigSchema: + """Parse actor config from YAML file.""" + return ActorConfigSchema.from_yaml(path) - def parse_string(self, content: str) -> ActorConfigSchema: - """Parse actor config from YAML string.""" - import yaml - data = yaml.safe_load(content) - return ActorConfigSchema.model_validate(data) - ``` - - [ ] Commit: "feat(actor): add ActorConfigParser scaffold" - - [ ] **C2.1b** [Aditya] Add tools section validation: - - [ ] Validate each tool has unique name - - [ ] Validate tool code compiles (syntax check) - - [ ] Validate tool parameters have valid JSON Schema types - - [ ] Commit: "feat(actor): add tools section validation" - - [ ] **C2.1c** [Aditya] Add routes section validation: - - [ ] Validate all node names are unique - - [ ] Validate entry_point exists in nodes - - [ ] Validate all edge sources/targets exist - - [ ] Check for unreachable nodes (warning) - - [ ] Commit: "feat(actor): add routes section validation" - - [ ] **C2.1d** [Aditya] Add actor reference validation: - - [ ] For subgraph nodes, validate `actor` field is present - - [ ] Validate referenced actor exists in registry - - [ ] Build dependency graph for circular reference detection - - [ ] Commit: "feat(actor): add actor reference validation" - - [ ] **C2.2** [Aditya] Define `CompiledActor` in `src/cleveragents/actor/compiled.py`: - - [ ] **C2.2a** [Aditya] Create CompiledActor dataclass: - ```python - @dataclass - class CompiledActor: - """A compiled actor ready for execution.""" - config: ActorConfigSchema - graph: StateGraph # LangGraph StateGraph - runnable: CompiledStateGraph # Compiled version for execution - tools: dict[str, Callable] # Name -> callable tool functions - referenced_actors: list[str] # Actor names this depends on - compiled_at: datetime + def parse_string(self, content: str) -> ActorConfigSchema: + """Parse actor config from YAML string.""" + import yaml + data = yaml.safe_load(content) + return ActorConfigSchema.model_validate(data) + ``` - def invoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: - """Execute the actor graph with input.""" - return self.runnable.invoke(input_data, config) + - [ ] Commit: "feat(actor): add ActorConfigParser scaffold" - async def ainvoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: - """Execute the actor graph asynchronously.""" - return await self.runnable.ainvoke(input_data, config) - ``` - - [ ] Commit: "feat(actor): define CompiledActor dataclass" - - [ ] **C2.3** [Aditya] Create `ActorCompiler` in `src/cleveragents/actor/compiler.py`: - - [ ] **C2.3a** [Aditya] Define compiler class scaffold: - ```python - class ActorCompiler: - """Compile actor configs into executable LangGraph graphs.""" + - [ ] **C2.1b** [Aditya] Add tools section validation: + - [ ] Validate each tool has unique name + - [ ] Validate tool code compiles (syntax check) + - [ ] Validate tool parameters have valid JSON Schema types + - [ ] Commit: "feat(actor): add tools section validation" + - [ ] **C2.1c** [Aditya] Add routes section validation: + - [ ] Validate all node names are unique + - [ ] Validate entry_point exists in nodes + - [ ] Validate all edge sources/targets exist + - [ ] Check for unreachable nodes (warning) + - [ ] Commit: "feat(actor): add routes section validation" + - [ ] **C2.1d** [Aditya] Add actor reference validation: + - [ ] For subgraph nodes, validate `actor` field is present + - [ ] Validate referenced actor exists in registry + - [ ] Build dependency graph for circular reference detection + - [ ] Commit: "feat(actor): add actor reference validation" - def __init__( - self, - model_factory: ModelFactory, - skill_registry: SkillRegistry, - mcp_manager: MCPServerManager | None = None - ): - self._model_factory = model_factory - self._skill_registry = skill_registry - self._mcp_manager = mcp_manager - self._compiled_cache: dict[str, CompiledActor] = {} - ``` - - [ ] Commit: "feat(actor): add ActorCompiler scaffold" - - [ ] **C2.3b** [Aditya] Implement main `compile()` method: - ```python - def compile(self, config: ActorConfigSchema) -> CompiledActor: - """Compile actor config into executable graph.""" - # Check cache - cache_key = f"{config.namespace}/{config.name}" - if cache_key in self._compiled_cache: - return self._compiled_cache[cache_key] + - [ ] **C2.2** [Aditya] Define `CompiledActor` in `src/cleveragents/actor/compiled.py`: + - [ ] **C2.2a** [Aditya] Create CompiledActor dataclass: - # Compile based on type - match config.type: - case ActorType.LLM: - graph = self._compile_llm_actor(config) - case ActorType.TOOL: - graph = self._compile_tool_actor(config) - case ActorType.GRAPH: - graph = self._compile_graph_actor(config) + ```python + @dataclass + class CompiledActor: + """A compiled actor ready for execution.""" + config: ActorConfigSchema + graph: StateGraph # LangGraph StateGraph + runnable: CompiledStateGraph # Compiled version for execution + tools: dict[str, Callable] # Name -> callable tool functions + referenced_actors: list[str] # Actor names this depends on + compiled_at: datetime - # Build tools dict - tools = self._build_tools(config) + def invoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: + """Execute the actor graph with input.""" + return self.runnable.invoke(input_data, config) - # Create CompiledActor - compiled = CompiledActor( - config=config, - graph=graph, - runnable=graph.compile(), - tools=tools, - referenced_actors=self._get_referenced_actors(config), - compiled_at=datetime.utcnow() - ) + async def ainvoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: + """Execute the actor graph asynchronously.""" + return await self.runnable.ainvoke(input_data, config) + ``` - # Cache and return - self._compiled_cache[cache_key] = compiled - return compiled - ``` - - [ ] Commit: "feat(actor): implement ActorCompiler.compile()" - - [ ] **C2.4** [Aditya] Implement LLM actor compilation: - - [ ] **C2.4a** [Aditya] Implement `_compile_llm_actor()`: - ```python - def _compile_llm_actor(self, config: ActorConfigSchema) -> StateGraph: - """Compile simple LLM actor into single-node graph.""" - from langgraph.graph import StateGraph, END - from langchain_core.messages import HumanMessage, SystemMessage + - [ ] Commit: "feat(actor): define CompiledActor dataclass" - # Create model - model = self._model_factory.create( - model_name=config.model, - provider=config.provider, - temperature=config.temperature, - max_tokens=config.max_tokens - ) + - [ ] **C2.3** [Aditya] Create `ActorCompiler` in `src/cleveragents/actor/compiler.py`: + - [ ] **C2.3a** [Aditya] Define compiler class scaffold: - # Bind tools if any - tools = self._build_tools(config) - if tools: - model = model.bind_tools(list(tools.values())) + ```python + class ActorCompiler: + """Compile actor configs into executable LangGraph graphs.""" - # Define state - class State(TypedDict): - messages: list[BaseMessage] - context: dict + def __init__( + self, + model_factory: ModelFactory, + skill_registry: SkillRegistry, + mcp_manager: MCPServerManager | None = None + ): + self._model_factory = model_factory + self._skill_registry = skill_registry + self._mcp_manager = mcp_manager + self._compiled_cache: dict[str, CompiledActor] = {} + ``` - # Define agent node - def agent(state: State) -> State: - messages = state["messages"] - if config.system_prompt: - messages = [SystemMessage(content=config.system_prompt)] + messages - response = model.invoke(messages) - return {"messages": [response]} + - [ ] Commit: "feat(actor): add ActorCompiler scaffold" - # Build graph - graph = StateGraph(State) - graph.add_node("agent", agent) - graph.set_entry_point("agent") - graph.add_edge("agent", END) + - [ ] **C2.3b** [Aditya] Implement main `compile()` method: - return graph - ``` - - [ ] Commit: "feat(actor): implement LLM actor compilation" - - [ ] **C2.4b** [Aditya] Add memory support to LLM actors: - - [ ] If config.memory.enabled, wrap with memory checkpointer - - [ ] Configure message trimming based on max_turns - - [ ] Commit: "feat(actor): add memory support to LLM actors" - - [ ] **C2.5** [Aditya] Implement tool actor compilation: - - [ ] **C2.5a** [Aditya] Implement `_compile_tool_actor()`: - - [ ] Create ReAct-style agent with tools - - [ ] Configure tool calling loop - - [ ] Add tool nodes for each defined tool - - [ ] Commit: "feat(actor): implement tool actor compilation" - - [ ] **C2.5b** [Aditya] Implement tool node generation: - ```python - def _build_tools(self, config: ActorConfigSchema) -> dict[str, Callable]: - """Build callable tools from config.""" - tools = {} + ```python + def compile(self, config: ActorConfigSchema) -> CompiledActor: + """Compile actor config into executable graph.""" + # Check cache + cache_key = f"{config.namespace}/{config.name}" + if cache_key in self._compiled_cache: + return self._compiled_cache[cache_key] - # Inline tools from YAML - for tool_def in config.tools: - tools[tool_def.name] = self._create_inline_tool(tool_def) + # Compile based on type + match config.type: + case ActorType.LLM: + graph = self._compile_llm_actor(config) + case ActorType.TOOL: + graph = self._compile_tool_actor(config) + case ActorType.GRAPH: + graph = self._compile_graph_actor(config) - # Built-in tools - for tool_name in config.builtin_tools: - tool = self._skill_registry.get_skill(tool_name) - if tool: - tools[tool_name] = tool.to_langchain_tool() + # Build tools dict + tools = self._build_tools(config) - # MCP tools - if config.mcp_servers and self._mcp_manager: - for server_id in config.mcp_servers: - mcp_tools = self._mcp_manager.get_tools(server_id) - tools.update(mcp_tools) - - return tools - ``` - - [ ] Commit: "feat(actor): implement tool building from config" - - [ ] **C2.6** [Aditya] Implement graph actor compilation: - - [ ] **C2.6a** [Aditya] Implement `_compile_graph_actor()`: - ```python - def _compile_graph_actor(self, config: ActorConfigSchema) -> StateGraph: - """Compile multi-node graph actor.""" - routes = config.routes - - # Define state type dynamically based on nodes - State = self._build_state_type(routes) - - # Create graph - graph = StateGraph(State) - - # Add nodes - for node_def in routes.nodes: - node_func = self._create_node(node_def, config) - graph.add_node(node_def.name, node_func) - - # Set entry point - graph.set_entry_point(routes.entry_point) - - # Add edges - for edge in routes.edges: - if edge.condition: - # Conditional edge - condition_func = self._parse_condition(edge.condition) - graph.add_conditional_edges( - edge.source, - condition_func, - {True: edge.target} + # Create CompiledActor + compiled = CompiledActor( + config=config, + graph=graph, + runnable=graph.compile(), + tools=tools, + referenced_actors=self._get_referenced_actors(config), + compiled_at=datetime.utcnow() ) - else: - # Direct edge - graph.add_edge(edge.source, edge.target) - return graph - ``` - - [ ] Commit: "feat(actor): implement graph actor compilation" - - [ ] **C2.6b** [Aditya] Implement node creation by type: - - [ ] Agent nodes: create LLM with optional tools - - [ ] Tool nodes: create tool execution wrapper - - [ ] Conditional nodes: create routing logic - - [ ] Subgraph nodes: compile and embed referenced actor - - [ ] Commit: "feat(actor): implement node creation by type" - - [ ] **C2.7** [Aditya] Implement actor reference resolution: - - [ ] **C2.7a** [Aditya] Add circular reference detection: - ```python - def _check_circular_references( - self, - config: ActorConfigSchema, - visited: set[str] | None = None - ) -> None: - """Detect circular actor references.""" - visited = visited or set() - actor_name = f"{config.namespace}/{config.name}" + # Cache and return + self._compiled_cache[cache_key] = compiled + return compiled + ``` - if actor_name in visited: - raise CircularReferenceError( - f"Circular reference detected: {' -> '.join(visited)} -> {actor_name}" - ) + - [ ] Commit: "feat(actor): implement ActorCompiler.compile()" - visited.add(actor_name) + - [ ] **C2.4** [Aditya] Implement LLM actor compilation: + - [ ] **C2.4a** [Aditya] Implement `_compile_llm_actor()`: - for ref in self._get_referenced_actors(config): - ref_config = self._registry.get_config(ref) - if ref_config: - self._check_circular_references(ref_config, visited.copy()) - ``` - - [ ] Commit: "feat(actor): add circular reference detection" - - [ ] **C2.7b** [Aditya] Implement recursive compilation: - - [ ] When compiling subgraph node, recursively compile referenced actor - - [ ] Cache compiled actors to avoid recompilation - - [ ] Pass context appropriately to subgraphs - - [ ] Commit: "feat(actor): implement recursive actor compilation" - - [ ] **C2.8** [Aditya] Update `ActorRegistry` to support compilation: - - [ ] **C2.8a** [Aditya] Add `get_compiled()` method: - ```python - def get_compiled(self, name: str) -> CompiledActor: - """Get compiled actor by name, compiling if needed.""" - # Check compiled cache - if name in self._compiled_cache: - cached = self._compiled_cache[name] - # Check if config changed - current_config = self.get_config(name) - if current_config and self._config_unchanged(name, current_config): - return cached + ```python + def _compile_llm_actor(self, config: ActorConfigSchema) -> StateGraph: + """Compile simple LLM actor into single-node graph.""" + from langgraph.graph import StateGraph, END + from langchain_core.messages import HumanMessage, SystemMessage - # Load config - config = self.get_config(name) - if not config: - raise ActorNotFoundError(f"Actor '{name}' not found") + # Create model + model = self._model_factory.create( + model_name=config.model, + provider=config.provider, + temperature=config.temperature, + max_tokens=config.max_tokens + ) - # Compile - compiled = self._compiler.compile(config) - self._compiled_cache[name] = compiled + # Bind tools if any + tools = self._build_tools(config) + if tools: + model = model.bind_tools(list(tools.values())) - return compiled - ``` - - [ ] Commit: "feat(actor): add ActorRegistry.get_compiled()" - - [ ] **C2.8b** [Aditya] Add cache invalidation: - - [ ] Monitor actor YAML files for changes - - [ ] Clear cache entry when config file modified - - [ ] Clear dependent actors when base actor changes - - [ ] Commit: "feat(actor): add compiled actor cache invalidation" - - [ ] Tests: Behave scenarios for actor compilation - - [ ] **C2.9** [Rui] Write Behave scenarios in `features/actor_compilation.feature`: - - [ ] **C2.9a** [Rui] LLM actor compilation scenarios: - - [ ] Scenario: Compile simple LLM actor creates valid graph - - [ ] Given actor config with type=llm and model=gpt-4 - - [ ] When I compile the actor - - [ ] Then CompiledActor is returned - - [ ] And graph has single agent node - - [ ] And runnable can be invoked - - [ ] Scenario: LLM actor with tools binds tools correctly - - [ ] Commit: "test(behave): add LLM actor compilation scenarios" - - [ ] **C2.9b** [Rui] Tool actor compilation scenarios: - - [ ] Scenario: Compile tool actor with inline code works - - [ ] Given actor config with inline tool definitions - - [ ] When I compile the actor - - [ ] Then tools dict contains the defined tools - - [ ] And tools are callable - - [ ] Scenario: Built-in tools are included - - [ ] Commit: "test(behave): add tool actor compilation scenarios" - - [ ] **C2.9c** [Rui] Graph actor compilation scenarios: - - [ ] Scenario: Compile graph actor creates correct topology - - [ ] Given actor config with routes defining 3 nodes - - [ ] When I compile the actor - - [ ] Then graph has 3 nodes - - [ ] And edges match route definition - - [ ] And entry_point is set correctly - - [ ] Scenario: Conditional edges work correctly - - [ ] Commit: "test(behave): add graph actor compilation scenarios" - - [ ] **C2.9d** [Rui] Reference resolution scenarios: - - [ ] Scenario: Actor referencing other actor compiles recursively - - [ ] Given actor A references actor B as subgraph - - [ ] When I compile actor A - - [ ] Then actor B is also compiled - - [ ] And actor B graph is embedded in actor A - - [ ] Scenario: Circular reference detected and errors - - [ ] Given actor A references B and B references A - - [ ] When I try to compile actor A - - [ ] Then CircularReferenceError is raised - - [ ] And error message shows the cycle - - [ ] Commit: "test(behave): add reference resolution scenarios" - - [ ] **C2.9e** [Rui] Error scenarios: - - [ ] Scenario: Invalid actor config produces clear error - - [ ] Scenario: Missing referenced actor produces clear error - - [ ] Scenario: Invalid tool code produces clear error - - [ ] Commit: "test(behave): add compilation error scenarios" + # Define state + class State(TypedDict): + messages: list[BaseMessage] + context: dict + + # Define agent node + def agent(state: State) -> State: + messages = state["messages"] + if config.system_prompt: + messages = [SystemMessage(content=config.system_prompt)] + messages + response = model.invoke(messages) + return {"messages": [response]} + + # Build graph + graph = StateGraph(State) + graph.add_node("agent", agent) + graph.set_entry_point("agent") + graph.add_edge("agent", END) + + return graph + ``` + + - [ ] Commit: "feat(actor): implement LLM actor compilation" + + - [ ] **C2.4b** [Aditya] Add memory support to LLM actors: + - [ ] If config.memory.enabled, wrap with memory checkpointer + - [ ] Configure message trimming based on max_turns + - [ ] Commit: "feat(actor): add memory support to LLM actors" + + - [ ] **C2.5** [Aditya] Implement tool actor compilation: + - [ ] **C2.5a** [Aditya] Implement `_compile_tool_actor()`: + - [ ] Create ReAct-style agent with tools + - [ ] Configure tool calling loop + - [ ] Add tool nodes for each defined tool + - [ ] Commit: "feat(actor): implement tool actor compilation" + - [ ] **C2.5b** [Aditya] Implement tool node generation: + + ```python + def _build_tools(self, config: ActorConfigSchema) -> dict[str, Callable]: + """Build callable tools from config.""" + tools = {} + + # Inline tools from YAML + for tool_def in config.tools: + tools[tool_def.name] = self._create_inline_tool(tool_def) + + # Built-in tools + for tool_name in config.builtin_tools: + tool = self._skill_registry.get_skill(tool_name) + if tool: + tools[tool_name] = tool.to_langchain_tool() + + # MCP tools + if config.mcp_servers and self._mcp_manager: + for server_id in config.mcp_servers: + mcp_tools = self._mcp_manager.get_tools(server_id) + tools.update(mcp_tools) + + return tools + ``` + + - [ ] Commit: "feat(actor): implement tool building from config" + + - [ ] **C2.6** [Aditya] Implement graph actor compilation: + - [ ] **C2.6a** [Aditya] Implement `_compile_graph_actor()`: + + ```python + def _compile_graph_actor(self, config: ActorConfigSchema) -> StateGraph: + """Compile multi-node graph actor.""" + routes = config.routes + + # Define state type dynamically based on nodes + State = self._build_state_type(routes) + + # Create graph + graph = StateGraph(State) + + # Add nodes + for node_def in routes.nodes: + node_func = self._create_node(node_def, config) + graph.add_node(node_def.name, node_func) + + # Set entry point + graph.set_entry_point(routes.entry_point) + + # Add edges + for edge in routes.edges: + if edge.condition: + # Conditional edge + condition_func = self._parse_condition(edge.condition) + graph.add_conditional_edges( + edge.source, + condition_func, + {True: edge.target} + ) + else: + # Direct edge + graph.add_edge(edge.source, edge.target) + + return graph + ``` + + - [ ] Commit: "feat(actor): implement graph actor compilation" + + - [ ] **C2.6b** [Aditya] Implement node creation by type: + - [ ] Agent nodes: create LLM with optional tools + - [ ] Tool nodes: create tool execution wrapper + - [ ] Conditional nodes: create routing logic + - [ ] Subgraph nodes: compile and embed referenced actor + - [ ] Commit: "feat(actor): implement node creation by type" + + - [ ] **C2.7** [Aditya] Implement actor reference resolution: + - [ ] **C2.7a** [Aditya] Add circular reference detection: + + ```python + def _check_circular_references( + self, + config: ActorConfigSchema, + visited: set[str] | None = None + ) -> None: + """Detect circular actor references.""" + visited = visited or set() + actor_name = f"{config.namespace}/{config.name}" + + if actor_name in visited: + raise CircularReferenceError( + f"Circular reference detected: {' -> '.join(visited)} -> {actor_name}" + ) + + visited.add(actor_name) + + for ref in self._get_referenced_actors(config): + ref_config = self._registry.get_config(ref) + if ref_config: + self._check_circular_references(ref_config, visited.copy()) + ``` + + - [ ] Commit: "feat(actor): add circular reference detection" + + - [ ] **C2.7b** [Aditya] Implement recursive compilation: + - [ ] When compiling subgraph node, recursively compile referenced actor + - [ ] Cache compiled actors to avoid recompilation + - [ ] Pass context appropriately to subgraphs + - [ ] Commit: "feat(actor): implement recursive actor compilation" + + - [ ] **C2.8** [Aditya] Update `ActorRegistry` to support compilation: + - [ ] **C2.8a** [Aditya] Add `get_compiled()` method: + + ```python + def get_compiled(self, name: str) -> CompiledActor: + """Get compiled actor by name, compiling if needed.""" + # Check compiled cache + if name in self._compiled_cache: + cached = self._compiled_cache[name] + # Check if config changed + current_config = self.get_config(name) + if current_config and self._config_unchanged(name, current_config): + return cached + + # Load config + config = self.get_config(name) + if not config: + raise ActorNotFoundError(f"Actor '{name}' not found") + + # Compile + compiled = self._compiler.compile(config) + self._compiled_cache[name] = compiled + + return compiled + ``` + + - [ ] Commit: "feat(actor): add ActorRegistry.get_compiled()" + + - [ ] **C2.8b** [Aditya] Add cache invalidation: + - [ ] Monitor actor YAML files for changes + - [ ] Clear cache entry when config file modified + - [ ] Clear dependent actors when base actor changes + - [ ] Commit: "feat(actor): add compiled actor cache invalidation" + + - [ ] Tests: Behave scenarios for actor compilation + - [ ] **C2.9** [Rui] Write Behave scenarios in `features/actor_compilation.feature`: + - [ ] **C2.9a** [Rui] LLM actor compilation scenarios: + - [ ] Scenario: Compile simple LLM actor creates valid graph + - [ ] Given actor config with type=llm and model=gpt-4 + - [ ] When I compile the actor + - [ ] Then CompiledActor is returned + - [ ] And graph has single agent node + - [ ] And runnable can be invoked + - [ ] Scenario: LLM actor with tools binds tools correctly + - [ ] Commit: "test(behave): add LLM actor compilation scenarios" + - [ ] **C2.9b** [Rui] Tool actor compilation scenarios: + - [ ] Scenario: Compile tool actor with inline code works + - [ ] Given actor config with inline tool definitions + - [ ] When I compile the actor + - [ ] Then tools dict contains the defined tools + - [ ] And tools are callable + - [ ] Scenario: Built-in tools are included + - [ ] Commit: "test(behave): add tool actor compilation scenarios" + - [ ] **C2.9c** [Rui] Graph actor compilation scenarios: + - [ ] Scenario: Compile graph actor creates correct topology + - [ ] Given actor config with routes defining 3 nodes + - [ ] When I compile the actor + - [ ] Then graph has 3 nodes + - [ ] And edges match route definition + - [ ] And entry_point is set correctly + - [ ] Scenario: Conditional edges work correctly + - [ ] Commit: "test(behave): add graph actor compilation scenarios" + - [ ] **C2.9d** [Rui] Reference resolution scenarios: + - [ ] Scenario: Actor referencing other actor compiles recursively + - [ ] Given actor A references actor B as subgraph + - [ ] When I compile actor A + - [ ] Then actor B is also compiled + - [ ] And actor B graph is embedded in actor A + - [ ] Scenario: Circular reference detected and errors + - [ ] Given actor A references B and B references A + - [ ] When I try to compile actor A + - [ ] Then CircularReferenceError is raised + - [ ] And error message shows the cycle + - [ ] Commit: "test(behave): add reference resolution scenarios" + - [ ] **C2.9e** [Rui] Error scenarios: + - [ ] Scenario: Invalid actor config produces clear error + - [ ] Scenario: Missing referenced actor produces clear error + - [ ] Scenario: Invalid tool code produces clear error + - [ ] Commit: "test(behave): add compilation error scenarios" - [ ] **Stage C3: Skill Execution Framework** (Day 5-6) **[Jeff + Aditya - Critical Path]** - - [ ] Code: Implement skill execution framework - - [ ] **C3.1** [Jeff] Define `Skill` protocol in `src/cleveragents/actor/skills/protocol.py`: - - [ ] **C3.1a** Create abstract base using `typing.Protocol`: - ```python - class Skill(Protocol): - @property - def name(self) -> str: ... - @property - def description(self) -> str: ... - @property - def parameters(self) -> dict[str, Any]: ... # JSON Schema - @property - def metadata(self) -> SkillMetadata: ... + - [ ] Code: Implement skill execution framework + - [ ] **C3.1** [Jeff] Define `Skill` protocol in `src/cleveragents/actor/skills/protocol.py`: + - [ ] **C3.1a** Create abstract base using `typing.Protocol`: - async def execute( - self, - input_data: dict[str, Any], - context: SkillContext - ) -> SkillResult: ... - ``` - - [ ] **C3.1b** Define `SkillResult` dataclass: - - [ ] Field `success: bool` - whether execution succeeded - - [ ] Field `result: Any` - return value if successful - - [ ] Field `error: str | None` - error message if failed - - [ ] Field `changes: list[Change]` - changes made to resources - - [ ] Field `duration_ms: int` - execution time - - [ ] **C3.1c** Add docstrings explaining contract for skill implementers - - [ ] **C3.2** [Jeff] Define `SkillMetadata` Pydantic model in `src/cleveragents/actor/skills/metadata.py`: - - [ ] **C3.2a** Core capability fields: - - [ ] Field `read_only: bool = False` - only performs read operations - - [ ] Field `writes: bool = False` - can modify resources - - [ ] Field `write_scope: list[str] = []` - glob patterns for writable paths - - [ ] Field `idempotent: bool = False` - repeated calls produce same result - - [ ] Field `checkpointable: bool = False` - supports checkpoint/rollback - - [ ] Field `side_effects: list[str] = []` - external side effects (e.g., "network", "subprocess") - - [ ] **C3.2b** Safety and control fields: - - [ ] Field `human_approval_required: bool = False` - requires user confirmation - - [ ] Field `rate_limit: RateLimit | None = None` - calls per minute/hour - - [ ] Field `cost_profile: CostProfile | None = None` - estimated cost per call - - [ ] Field `timeout_seconds: int = 30` - maximum execution time - - [ ] **C3.2c** Define `RateLimit` and `CostProfile` models - - [ ] **C3.2d** Add validation to ensure `writes=True` if `write_scope` is non-empty - - [ ] **C3.3** [Jeff] Create `SkillContext` in `src/cleveragents/actor/skills/context.py`: - - [ ] **C3.3a** Define context fields: - - [ ] Field `plan_id: str` - current plan ULID - - [ ] Field `plan: Plan` - full plan object for reference - - [ ] Field `project: Project` - target project - - [ ] Field `resources: list[Resource]` - available resources - - [ ] Field `sandbox_manager: SandboxManager` - for sandbox access - - [ ] Field `changeset: ChangeSet` - accumulating changes - - [ ] Field `invocation_tracker: SkillInvocationTracker` - tracking calls - - [ ] Field `logger: logging.Logger` - skill-specific logger - - [ ] **C3.3b** Implement convenience methods: - - [ ] Method `get_file(path: str, resource: str | None = None) -> str`: - - [ ] Resolve path to sandboxed location - - [ ] Read and return file contents - - [ ] Raise FileNotFoundError if not exists - - [ ] Method `write_file(path: str, content: str, resource: str | None = None) -> Change`: - - [ ] Resolve path to sandboxed location - - [ ] Validate path against deny-list - - [ ] Create parent directories if needed - - [ ] Write content - - [ ] Create and record Change - - [ ] Return Change for tracking - - [ ] Method `edit_file(path: str, edits: list[Edit], resource: str | None = None) -> Change`: - - [ ] Read current content - - [ ] Apply edits sequentially - - [ ] Write modified content - - [ ] Record Change with edits - - [ ] Method `delete_file(path: str, resource: str | None = None) -> Change`: - - [ ] Validate file exists - - [ ] Delete file - - [ ] Record Change - - [ ] Method `list_files(pattern: str, resource: str | None = None) -> list[str]`: - - [ ] Resolve pattern to sandbox - - [ ] Return matching paths - - [ ] Method `search_files(pattern: str, content_pattern: str, resource: str | None = None) -> list[SearchResult]`: - - [ ] Search file contents with regex - - [ ] Return matches with file, line, context - - [ ] **C3.3c** Implement subplan spawning: - - [ ] Method `spawn_subplan(action: str, target_resources: list[str] | None = None, arguments: dict | None = None) -> str`: - - [ ] Validate action exists - - [ ] Create child plan with parent_plan_id = self.plan_id - - [ ] Queue subplan for execution - - [ ] Return subplan_id for tracking - - [ ] Record as `subplan_spawn` decision type - - [ ] **C3.3d** Implement read-only check enforcement: - - [ ] If plan.action.read_only is True, block all write operations - - [ ] Raise `ReadOnlyViolationError` if write attempted - - [ ] **C3.4** [Aditya] Implement `InlineSkillExecutor` in `src/cleveragents/actor/skills/inline_executor.py`: - - [ ] **C3.4a** Create class for executing inline Python code from actor YAML: - ```python - class InlineSkillExecutor: - def __init__(self, code: str, timeout: int = 30): - self.code = code - self.timeout = timeout - ``` - - [ ] **C3.4b** Create sandboxed execution environment: - - [ ] Restricted `__builtins__`: - - [ ] ALLOWED: `len`, `range`, `str`, `int`, `float`, `list`, `dict`, `set`, `tuple`, `bool`, `None`, `True`, `False`, `print`, `isinstance`, `hasattr`, `getattr`, `enumerate`, `zip`, `map`, `filter`, `sorted`, `reversed`, `any`, `all`, `min`, `max`, `sum`, `abs`, `round` - - [ ] BLOCKED: `open`, `exec`, `eval`, `compile`, `__import__`, `globals`, `locals`, `vars`, `dir`, `input` - - [ ] Inject `context: SkillContext` variable - - [ ] Inject `input_data: dict` variable - - [ ] Inject standard library modules: `re`, `json`, `datetime`, `collections`, `itertools`, `functools` - - [ ] **C3.4c** Execute code and capture result: - - [ ] Use `exec()` with restricted globals/locals - - [ ] Capture `result` variable as return value - - [ ] If no `result` variable, return None - - [ ] Wrap in asyncio.wait_for for timeout - - [ ] **C3.4d** Handle errors gracefully: - - [ ] Catch all exceptions during execution - - [ ] Convert to SkillResult with error message - - [ ] Include stack trace in error for debugging - - [ ] Log error with skill name and input - - [ ] **C3.4e** Add timeout support: - - [ ] Default 30 seconds - - [ ] Configurable via skill metadata - - [ ] Raise TimeoutError if exceeded - - [ ] **C3.5** [Aditya] Implement subplan spawning in skill context: - - [ ] **C3.5a** Update SkillContext.spawn_subplan to create real subplans: - - [ ] Call PlanLifecycleService.use_action() with parent_plan_id - - [ ] Set subplan's root_plan_id to parent's root_plan_id (or parent's id if root) - - [ ] Set subplan's automation_level from parent - - [ ] Return subplan_id - - [ ] **C3.5b** Add subplan tracking to parent plan: - - [ ] Store spawned subplan IDs in plan's execution_log - - [ ] Support querying all subplans of a plan - - [ ] **C3.5c** Add subplan completion handling: - - [ ] Parent plan can check subplan status - - [ ] Parent plan can collect subplan results - - [ ] Support waiting for subplan completion - - [ ] **C3.6** [Jeff + Luis] Implement built-in resource skills in `src/cleveragents/actor/skills/builtin/`: - - [ ] **C3.6a** [Jeff] Create base skill class in `__init__.py`: - ```python - class BuiltinSkill(ABC): - @abstractmethod - async def execute(self, input_data: dict, context: SkillContext) -> SkillResult: ... + ```python + class Skill(Protocol): + @property + def name(self) -> str: ... + @property + def description(self) -> str: ... + @property + def parameters(self) -> dict[str, Any]: ... # JSON Schema + @property + def metadata(self) -> SkillMetadata: ... - def _validate_path(self, path: str, context: SkillContext) -> str: - """Resolve and validate path against sandbox.""" - ... - ``` - - [ ] **C3.6b** [Jeff] File operation skills in `file_ops.py`: - - [ ] **ReadFileSkill**: - - [ ] Parameters: `path: str` (required) - - [ ] Metadata: `read_only=True` - - [ ] Implementation: resolve path, read via sandbox, return content - - [ ] Error handling: FileNotFoundError, PermissionError - - [ ] **WriteFileSkill**: - - [ ] Parameters: `path: str`, `content: str` - - [ ] Metadata: `writes=True, write_scope=['**/*']` - - [ ] Implementation: - - [ ] Validate path not in deny-list - - [ ] Create parent directories if needed - - [ ] Determine if create or modify operation - - [ ] Write content to sandbox - - [ ] Create Change record with operation type - - [ ] Record change in context.changeset - - [ ] Return: Change object with path and operation - - [ ] **EditFileSkill** (most complex - critical for coding): - - [ ] Parameters: `path: str`, `edits: list[Edit]` - - [ ] Metadata: `writes=True, idempotent=False` - - [ ] Implementation: - - [ ] Read original file content - - [ ] For each Edit in edits: - - [ ] If type=SEARCH_REPLACE: find `search` text, replace with `replace` - - [ ] If type=LINE_RANGE: replace lines start_line:end_line with content - - [ ] If type=INSERT_AFTER: insert content after matching line - - [ ] If type=INSERT_BEFORE: insert content before matching line - - [ ] If type=DELETE_LINES: remove lines start_line:end_line - - [ ] Track all changes made - - [ ] Write modified content - - [ ] Create Change with edits list - - [ ] Error handling: SearchTextNotFoundError, InvalidLineRangeError - - [ ] **DeleteFileSkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: delete file, create DELETE Change - - [ ] **MoveFileSkill**: - - [ ] Parameters: `source: str`, `destination: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: move file, create MOVE Change with new_path - - [ ] **CopyFileSkill**: - - [ ] Parameters: `source: str`, `destination: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: copy file, create CREATE Change - - [ ] **C3.6c** [Luis] Directory operation skills in `dir_ops.py`: - - [ ] **CreateDirectorySkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: create directory (and parents), record Change - - [ ] **ListDirectorySkill**: - - [ ] Parameters: `path: str`, `pattern: str = "*"`, `recursive: bool = False` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: list matching files/dirs, return paths - - [ ] **DeleteDirectorySkill**: - - [ ] Parameters: `path: str`, `recursive: bool = False` - - [ ] Metadata: `writes=True` - - [ ] Implementation: delete directory, record DELETE Changes for all contents - - [ ] **C3.6d** [Luis] Search skills in `search_ops.py`: - - [ ] **SearchFilesSkill**: - - [ ] Parameters: `pattern: str` (glob), `content_pattern: str` (regex), `max_results: int = 100` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: - - [ ] Find files matching glob pattern - - [ ] Search each file for content_pattern - - [ ] Return list of SearchResult(file, line_number, line_content, context) - - [ ] **FindDefinitionSkill** (uses tree-sitter for AST): - - [ ] Parameters: `symbol: str`, `language: str | None = None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: - - [ ] Parse files with tree-sitter - - [ ] Find function/class/variable definitions - - [ ] Return list of Location(file, line, column, snippet) - - [ ] **FindReferencesSkill**: - - [ ] Parameters: `symbol: str` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: find all usages of symbol - - [ ] **GetFileInfoSkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: return FileInfo(size, mtime, language, line_count) - - [ ] **C3.6e** [Hamza] Git operation skills in `git_ops.py`: - - [ ] **GitStatusSkill**: - - [ ] Parameters: (none) - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git status --porcelain`, parse output - - [ ] **GitDiffSkill**: - - [ ] Parameters: `path: str | None = None`, `staged: bool = False` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git diff [--staged] [path]` - - [ ] **GitLogSkill**: - - [ ] Parameters: `count: int = 10`, `path: str | None = None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git log --oneline -n {count}` - - [ ] **GitBlameSkill**: - - [ ] Parameters: `path: str`, `start_line: int | None`, `end_line: int | None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git blame -L {start},{end} {path}` - - [ ] **C3.6f** [Jeff] All built-in skills must implement: - - [ ] Full SkillMetadata with accurate capability flags - - [ ] Proper error handling with descriptive messages - - [ ] Logging of all operations for debugging - - [ ] Path validation against sandbox and deny-lists - - [ ] Change recording for all write operations - - [ ] Async execution support - - [ ] **C3.6g** [Jeff] Create skill registration in `src/cleveragents/actor/skills/registry.py`: - - [ ] `BuiltinSkillRegistry` singleton with all built-in skills - - [ ] Method `get_skill(name: str) -> Skill | None` - - [ ] Method `list_skills() -> list[Skill]` - - [ ] Method `list_by_capability(read_only: bool = None, writes: bool = None) -> list[Skill]` - - [ ] Register all C3.6 skills on import - - [ ] **C3.7** [Aditya] Implement MCP skill adapter in `src/cleveragents/actor/skills/mcp_adapter.py`: - - [ ] **C3.7a** `MCPServerConnection` class: - - [ ] Connect to MCP server via stdio or SSE transport - - [ ] List available tools from server - - [ ] Call tools with JSON-RPC - - [ ] Handle server lifecycle (start/stop) - - [ ] **C3.7b** `MCPSkillAdapter` class: - - [ ] Wrap MCP tool as CleverAgents Skill - - [ ] Infer SkillMetadata from MCP tool schema - - [ ] Intercept calls for sandbox path rewriting - - [ ] Record changes when MCP tool modifies resources - - [ ] **C3.7c** Actor YAML integration: - - [ ] Parse `mcp_servers` config in actor definition - - [ ] Auto-register MCP tools as skills in actor context - - [ ] Environment variable substitution for secrets - - [ ] Tests: Integration tests for skill execution - - [ ] **C3.8** [Rui] Write Behave scenarios in `features/skill_execution.feature`: - - [ ] Scenario: Execute inline Python skill with context - - [ ] Scenario: Skill can read files from sandbox - - [ ] Scenario: Skill can write files to sandbox - - [ ] Scenario: Skill with invalid code produces error - - [ ] Scenario: Skill timeout prevents infinite loops - - [ ] Scenario: spawn_subplan creates child plan - - [ ] **C3.9** [Rui] Write Behave scenarios in `features/builtin_skills.feature`: - - [ ] Scenario: WriteFileSkill creates file and records Change - - [ ] Scenario: EditFileSkill applies search/replace edit - - [ ] Scenario: DeleteFileSkill removes file and records Change - - [ ] Scenario: MoveFileSkill renames file and records Change - - [ ] Scenario: ListDirectorySkill returns matching files - - [ ] Scenario: SearchFilesSkill finds content matches - - [ ] Scenario: Skill respects deny-list patterns (.git/, node_modules/) - - [ ] Scenario: Skill enforces sandbox boundaries - - [ ] **C3.10** [Rui] Write Behave scenarios in `features/mcp_integration.feature`: - - [ ] Scenario: Connect to MCP server and list tools - - [ ] Scenario: MCP tool becomes available as skill - - [ ] Scenario: MCP tool call is intercepted for sandbox paths - - [ ] Scenario: MCP tool writes are recorded in ChangeSet + async def execute( + self, + input_data: dict[str, Any], + context: SkillContext + ) -> SkillResult: ... + ``` + + - [ ] **C3.1b** Define `SkillResult` dataclass: + - [ ] Field `success: bool` - whether execution succeeded + - [ ] Field `result: Any` - return value if successful + - [ ] Field `error: str | None` - error message if failed + - [ ] Field `changes: list[Change]` - changes made to resources + - [ ] Field `duration_ms: int` - execution time + - [ ] **C3.1c** Add docstrings explaining contract for skill implementers + + - [ ] **C3.2** [Jeff] Define `SkillMetadata` Pydantic model in `src/cleveragents/actor/skills/metadata.py`: + - [ ] **C3.2a** Core capability fields: + - [ ] Field `read_only: bool = False` - only performs read operations + - [ ] Field `writes: bool = False` - can modify resources + - [ ] Field `write_scope: list[str] = []` - glob patterns for writable paths + - [ ] Field `idempotent: bool = False` - repeated calls produce same result + - [ ] Field `checkpointable: bool = False` - supports checkpoint/rollback + - [ ] Field `side_effects: list[str] = []` - external side effects (e.g., "network", "subprocess") + - [ ] **C3.2b** Safety and control fields: + - [ ] Field `human_approval_required: bool = False` - requires user confirmation + - [ ] Field `rate_limit: RateLimit | None = None` - calls per minute/hour + - [ ] Field `cost_profile: CostProfile | None = None` - estimated cost per call + - [ ] Field `timeout_seconds: int = 30` - maximum execution time + - [ ] **C3.2c** Define `RateLimit` and `CostProfile` models + - [ ] **C3.2d** Add validation to ensure `writes=True` if `write_scope` is non-empty + - [ ] **C3.3** [Jeff] Create `SkillContext` in `src/cleveragents/actor/skills/context.py`: + - [ ] **C3.3a** Define context fields: + - [ ] Field `plan_id: str` - current plan ULID + - [ ] Field `plan: Plan` - full plan object for reference + - [ ] Field `project: Project` - target project + - [ ] Field `resources: list[Resource]` - available resources + - [ ] Field `sandbox_manager: SandboxManager` - for sandbox access + - [ ] Field `changeset: ChangeSet` - accumulating changes + - [ ] Field `invocation_tracker: SkillInvocationTracker` - tracking calls + - [ ] Field `logger: logging.Logger` - skill-specific logger + - [ ] **C3.3b** Implement convenience methods: + - [ ] Method `get_file(path: str, resource: str | None = None) -> str`: + - [ ] Resolve path to sandboxed location + - [ ] Read and return file contents + - [ ] Raise FileNotFoundError if not exists + - [ ] Method `write_file(path: str, content: str, resource: str | None = None) -> Change`: + - [ ] Resolve path to sandboxed location + - [ ] Validate path against deny-list + - [ ] Create parent directories if needed + - [ ] Write content + - [ ] Create and record Change + - [ ] Return Change for tracking + - [ ] Method `edit_file(path: str, edits: list[Edit], resource: str | None = None) -> Change`: + - [ ] Read current content + - [ ] Apply edits sequentially + - [ ] Write modified content + - [ ] Record Change with edits + - [ ] Method `delete_file(path: str, resource: str | None = None) -> Change`: + - [ ] Validate file exists + - [ ] Delete file + - [ ] Record Change + - [ ] Method `list_files(pattern: str, resource: str | None = None) -> list[str]`: + - [ ] Resolve pattern to sandbox + - [ ] Return matching paths + - [ ] Method `search_files(pattern: str, content_pattern: str, resource: str | None = None) -> list[SearchResult]`: + - [ ] Search file contents with regex + - [ ] Return matches with file, line, context + - [ ] **C3.3c** Implement subplan spawning: + - [ ] Method `spawn_subplan(action: str, target_resources: list[str] | None = None, arguments: dict | None = None) -> str`: + - [ ] Validate action exists + - [ ] Create child plan with parent_plan_id = self.plan_id + - [ ] Queue subplan for execution + - [ ] Return subplan_id for tracking + - [ ] Record as `subplan_spawn` decision type + - [ ] **C3.3d** Implement read-only check enforcement: + - [ ] If plan.action.read_only is True, block all write operations + - [ ] Raise `ReadOnlyViolationError` if write attempted + - [ ] **C3.4** [Aditya] Implement `InlineSkillExecutor` in `src/cleveragents/actor/skills/inline_executor.py`: + - [ ] **C3.4a** Create class for executing inline Python code from actor YAML: + ```python + class InlineSkillExecutor: + def __init__(self, code: str, timeout: int = 30): + self.code = code + self.timeout = timeout + ``` + - [ ] **C3.4b** Create sandboxed execution environment: + - [ ] Restricted `__builtins__`: + - [ ] ALLOWED: `len`, `range`, `str`, `int`, `float`, `list`, `dict`, `set`, `tuple`, `bool`, `None`, `True`, `False`, `print`, `isinstance`, `hasattr`, `getattr`, `enumerate`, `zip`, `map`, `filter`, `sorted`, `reversed`, `any`, `all`, `min`, `max`, `sum`, `abs`, `round` + - [ ] BLOCKED: `open`, `exec`, `eval`, `compile`, `__import__`, `globals`, `locals`, `vars`, `dir`, `input` + - [ ] Inject `context: SkillContext` variable + - [ ] Inject `input_data: dict` variable + - [ ] Inject standard library modules: `re`, `json`, `datetime`, `collections`, `itertools`, `functools` + - [ ] **C3.4c** Execute code and capture result: + - [ ] Use `exec()` with restricted globals/locals + - [ ] Capture `result` variable as return value + - [ ] If no `result` variable, return None + - [ ] Wrap in asyncio.wait_for for timeout + - [ ] **C3.4d** Handle errors gracefully: + - [ ] Catch all exceptions during execution + - [ ] Convert to SkillResult with error message + - [ ] Include stack trace in error for debugging + - [ ] Log error with skill name and input + - [ ] **C3.4e** Add timeout support: + - [ ] Default 30 seconds + - [ ] Configurable via skill metadata + - [ ] Raise TimeoutError if exceeded + - [ ] **C3.5** [Aditya] Implement subplan spawning in skill context: + - [ ] **C3.5a** Update SkillContext.spawn_subplan to create real subplans: + - [ ] Call PlanLifecycleService.use_action() with parent_plan_id + - [ ] Set subplan's root_plan_id to parent's root_plan_id (or parent's id if root) + - [ ] Set subplan's automation_level from parent + - [ ] Return subplan_id + - [ ] **C3.5b** Add subplan tracking to parent plan: + - [ ] Store spawned subplan IDs in plan's execution_log + - [ ] Support querying all subplans of a plan + - [ ] **C3.5c** Add subplan completion handling: + - [ ] Parent plan can check subplan status + - [ ] Parent plan can collect subplan results + - [ ] Support waiting for subplan completion + - [ ] **C3.6** [Jeff + Luis] Implement built-in resource skills in `src/cleveragents/actor/skills/builtin/`: + - [ ] **C3.6a** [Jeff] Create base skill class in `__init__.py`: + + ```python + class BuiltinSkill(ABC): + @abstractmethod + async def execute(self, input_data: dict, context: SkillContext) -> SkillResult: ... + + def _validate_path(self, path: str, context: SkillContext) -> str: + """Resolve and validate path against sandbox.""" + ... + ``` + + - [ ] **C3.6b** [Jeff] File operation skills in `file_ops.py`: + - [ ] **ReadFileSkill**: + - [ ] Parameters: `path: str` (required) + - [ ] Metadata: `read_only=True` + - [ ] Implementation: resolve path, read via sandbox, return content + - [ ] Error handling: FileNotFoundError, PermissionError + - [ ] **WriteFileSkill**: + - [ ] Parameters: `path: str`, `content: str` + - [ ] Metadata: `writes=True, write_scope=['**/*']` + - [ ] Implementation: + - [ ] Validate path not in deny-list + - [ ] Create parent directories if needed + - [ ] Determine if create or modify operation + - [ ] Write content to sandbox + - [ ] Create Change record with operation type + - [ ] Record change in context.changeset + - [ ] Return: Change object with path and operation + - [ ] **EditFileSkill** (most complex - critical for coding): + - [ ] Parameters: `path: str`, `edits: list[Edit]` + - [ ] Metadata: `writes=True, idempotent=False` + - [ ] Implementation: + - [ ] Read original file content + - [ ] For each Edit in edits: + - [ ] If type=SEARCH_REPLACE: find `search` text, replace with `replace` + - [ ] If type=LINE_RANGE: replace lines start_line:end_line with content + - [ ] If type=INSERT_AFTER: insert content after matching line + - [ ] If type=INSERT_BEFORE: insert content before matching line + - [ ] If type=DELETE_LINES: remove lines start_line:end_line + - [ ] Track all changes made + - [ ] Write modified content + - [ ] Create Change with edits list + - [ ] Error handling: SearchTextNotFoundError, InvalidLineRangeError + - [ ] **DeleteFileSkill**: + - [ ] Parameters: `path: str` + - [ ] Metadata: `writes=True` + - [ ] Implementation: delete file, create DELETE Change + - [ ] **MoveFileSkill**: + - [ ] Parameters: `source: str`, `destination: str` + - [ ] Metadata: `writes=True` + - [ ] Implementation: move file, create MOVE Change with new_path + - [ ] **CopyFileSkill**: + - [ ] Parameters: `source: str`, `destination: str` + - [ ] Metadata: `writes=True` + - [ ] Implementation: copy file, create CREATE Change + - [ ] **C3.6c** [Luis] Directory operation skills in `dir_ops.py`: + - [ ] **CreateDirectorySkill**: + - [ ] Parameters: `path: str` + - [ ] Metadata: `writes=True` + - [ ] Implementation: create directory (and parents), record Change + - [ ] **ListDirectorySkill**: + - [ ] Parameters: `path: str`, `pattern: str = "*"`, `recursive: bool = False` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: list matching files/dirs, return paths + - [ ] **DeleteDirectorySkill**: + - [ ] Parameters: `path: str`, `recursive: bool = False` + - [ ] Metadata: `writes=True` + - [ ] Implementation: delete directory, record DELETE Changes for all contents + - [ ] **C3.6d** [Luis] Search skills in `search_ops.py`: + - [ ] **SearchFilesSkill**: + - [ ] Parameters: `pattern: str` (glob), `content_pattern: str` (regex), `max_results: int = 100` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: + - [ ] Find files matching glob pattern + - [ ] Search each file for content_pattern + - [ ] Return list of SearchResult(file, line_number, line_content, context) + - [ ] **FindDefinitionSkill** (uses tree-sitter for AST): + - [ ] Parameters: `symbol: str`, `language: str | None = None` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: + - [ ] Parse files with tree-sitter + - [ ] Find function/class/variable definitions + - [ ] Return list of Location(file, line, column, snippet) + - [ ] **FindReferencesSkill**: + - [ ] Parameters: `symbol: str` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: find all usages of symbol + - [ ] **GetFileInfoSkill**: + - [ ] Parameters: `path: str` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: return FileInfo(size, mtime, language, line_count) + - [ ] **C3.6e** [Hamza] Git operation skills in `git_ops.py`: + - [ ] **GitStatusSkill**: + - [ ] Parameters: (none) + - [ ] Metadata: `read_only=True` + - [ ] Implementation: run `git status --porcelain`, parse output + - [ ] **GitDiffSkill**: + - [ ] Parameters: `path: str | None = None`, `staged: bool = False` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: run `git diff [--staged] [path]` + - [ ] **GitLogSkill**: + - [ ] Parameters: `count: int = 10`, `path: str | None = None` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: run `git log --oneline -n {count}` + - [ ] **GitBlameSkill**: + - [ ] Parameters: `path: str`, `start_line: int | None`, `end_line: int | None` + - [ ] Metadata: `read_only=True` + - [ ] Implementation: run `git blame -L {start},{end} {path}` + - [ ] **C3.6f** [Jeff] All built-in skills must implement: + - [ ] Full SkillMetadata with accurate capability flags + - [ ] Proper error handling with descriptive messages + - [ ] Logging of all operations for debugging + - [ ] Path validation against sandbox and deny-lists + - [ ] Change recording for all write operations + - [ ] Async execution support + - [ ] **C3.6g** [Jeff] Create skill registration in `src/cleveragents/actor/skills/registry.py`: + - [ ] `BuiltinSkillRegistry` singleton with all built-in skills + - [ ] Method `get_skill(name: str) -> Skill | None` + - [ ] Method `list_skills() -> list[Skill]` + - [ ] Method `list_by_capability(read_only: bool = None, writes: bool = None) -> list[Skill]` + - [ ] Register all C3.6 skills on import + + - [ ] **C3.7** [Aditya] Implement MCP skill adapter in `src/cleveragents/actor/skills/mcp_adapter.py`: + - [ ] **C3.7a** `MCPServerConnection` class: + - [ ] Connect to MCP server via stdio or SSE transport + - [ ] List available tools from server + - [ ] Call tools with JSON-RPC + - [ ] Handle server lifecycle (start/stop) + - [ ] **C3.7b** `MCPSkillAdapter` class: + - [ ] Wrap MCP tool as CleverAgents Skill + - [ ] Infer SkillMetadata from MCP tool schema + - [ ] Intercept calls for sandbox path rewriting + - [ ] Record changes when MCP tool modifies resources + - [ ] **C3.7c** Actor YAML integration: + - [ ] Parse `mcp_servers` config in actor definition + - [ ] Auto-register MCP tools as skills in actor context + - [ ] Environment variable substitution for secrets + + - [ ] Tests: Integration tests for skill execution + - [ ] **C3.8** [Rui] Write Behave scenarios in `features/skill_execution.feature`: + - [ ] Scenario: Execute inline Python skill with context + - [ ] Scenario: Skill can read files from sandbox + - [ ] Scenario: Skill can write files to sandbox + - [ ] Scenario: Skill with invalid code produces error + - [ ] Scenario: Skill timeout prevents infinite loops + - [ ] Scenario: spawn_subplan creates child plan + - [ ] **C3.9** [Rui] Write Behave scenarios in `features/builtin_skills.feature`: + - [ ] Scenario: WriteFileSkill creates file and records Change + - [ ] Scenario: EditFileSkill applies search/replace edit + - [ ] Scenario: DeleteFileSkill removes file and records Change + - [ ] Scenario: MoveFileSkill renames file and records Change + - [ ] Scenario: ListDirectorySkill returns matching files + - [ ] Scenario: SearchFilesSkill finds content matches + - [ ] Scenario: Skill respects deny-list patterns (.git/, node_modules/) + - [ ] Scenario: Skill enforces sandbox boundaries + - [ ] **C3.10** [Rui] Write Behave scenarios in `features/mcp_integration.feature`: + - [ ] Scenario: Connect to MCP server and list tools + - [ ] Scenario: MCP tool becomes available as skill + - [ ] Scenario: MCP tool call is intercepted for sandbox paths + - [ ] Scenario: MCP tool writes are recorded in ChangeSet - [ ] **Stage C4: Tool-Based Change Tracking** (Day 8-10) **[Luis - Architectural]** - - [ ] Code: Implement tool-based change tracking (NOT output parsing) - - [ ] **CRITICAL ARCHITECTURE**: ChangeSet is built from skill/tool invocations, NOT by parsing LLM text output - - [ ] **C4.1** [Luis] Update `src/cleveragents/domain/models/core/change.py`: - - [ ] Enhance `Change` model with: - - [ ] Field `operation: OperationType` - create/modify/delete/move - - [ ] Field `path: str` - target resource path - - [ ] Field `new_path: str | None` - for move operations - - [ ] Field `content: str | None` - full content (for create) - - [ ] Field `edits: list[Edit] | None` - targeted edits (for modify) - - [ ] Field `patch: str | None` - unified diff (generated, not parsed) - - [ ] Field `language: str | None` - detected language - - [ ] Field `skill_invocation_id: str` - which skill call produced this - - [ ] Field `timestamp: datetime` - when change was made - - [ ] Field `validation_result: ValidationResult | None` - - [ ] Define `Edit` model for targeted edits: - - [ ] Field `type: EditType` - search_replace, line_range, insert_after, etc. - - [ ] Field `search: str | None` - text to find (for search_replace) - - [ ] Field `replace: str | None` - replacement text - - [ ] Field `start_line: int | None` - for line-based edits - - [ ] Field `end_line: int | None` - for line-based edits - - [ ] Field `content: str | None` - content to insert - - [ ] Enhance `ChangeSet` model with: - - [ ] Field `changes: list[Change]` - all resource changes - - [ ] Field `warnings: list[str]` - non-blocking issues - - [ ] Field `skill_invocations: list[SkillInvocation]` - full invocation history - - [ ] Field `generated_by_actor: str` - actor that produced this - - [ ] Field `validation: ChangeSetValidation` - overall validation - - [ ] Method `add_change(change: Change)` - append change from skill - - [ ] Method `get_change(path: str) -> Change | None` - find by path - - [ ] Method `get_changes_by_skill(skill_id: str) -> list[Change]` - changes from specific skill - - [ ] Method `file_paths() -> list[str]` - all affected paths - - [ ] Method `to_diff() -> str` - unified diff of all changes - - [ ] Method `rollback_to(change_id: str)` - remove changes after point - - [ ] **C4.2** [Luis] Implement `SkillInvocationTracker` in `src/cleveragents/actor/skills/tracker.py`: - - [ ] **C4.2a** `SkillInvocation` model: - - [ ] Field `id: str` - unique invocation ID - - [ ] Field `skill_name: str` - which skill was called - - [ ] Field `parameters: dict` - input parameters - - [ ] Field `result: Any` - skill return value - - [ ] Field `changes: list[Change]` - resource changes produced - - [ ] Field `timestamp: datetime` - when invoked - - [ ] Field `duration_ms: int` - execution time - - [ ] Field `error: str | None` - if skill failed - - [ ] **C4.2b** `SkillInvocationTracker` class: - - [ ] Method `start_invocation(skill: Skill, params: dict) -> str` - begin tracking - - [ ] Method `record_change(invocation_id: str, change: Change)` - record change - - [ ] Method `complete_invocation(invocation_id: str, result: Any)` - finish tracking - - [ ] Method `fail_invocation(invocation_id: str, error: Exception)` - record failure - - [ ] Method `get_invocations() -> list[SkillInvocation]` - full history - - [ ] Method `build_changeset() -> ChangeSet` - assemble from invocations - - [ ] **C4.3** [Luis] Implement `ToolCallRouter` in `src/cleveragents/actor/skills/router.py`: - - [ ] **C4.3a** Parse LLM tool calls (NOT text output): - - [ ] Handle OpenAI-style tool_calls from response - - [ ] Handle Anthropic-style tool_use blocks - - [ ] Handle LangChain AgentAction format - - [ ] **C4.3b** Route tool calls to skills: - - [ ] Look up skill by name in registry - - [ ] Validate parameters against skill schema - - [ ] Check capability metadata (read_only, writes, etc.) - - [ ] Enforce permission restrictions - - [ ] **C4.3c** Execute with tracking: - - [ ] Start invocation tracking - - [ ] Execute skill in sandbox context - - [ ] Record changes produced by skill - - [ ] Complete invocation tracking - - [ ] Return result to LLM - - [ ] **C4.4** [Luis] Implement resource path validation in `src/cleveragents/actor/skills/path_validator.py`: - - [ ] Validate paths against sandbox boundaries - - [ ] Enforce deny-list patterns (.git/, node_modules/, __pycache__/, etc.) - - [ ] Auto-create parent directories for new file paths - - [ ] Resolve relative paths to absolute sandbox paths - - [ ] Detect path traversal attempts (../) - - [ ] **C4.5** [Luis] Create diff generation in `src/cleveragents/agents/diff_generator.py`: - - [ ] Method `generate_unified_diff(change: Change, sandbox: Sandbox) -> str`: - - [ ] Compare sandbox state with original - - [ ] Generate unified diff format - - [ ] Include file headers with paths - - [ ] Method `generate_changeset_diff(changeset: ChangeSet, sandbox: Sandbox) -> str`: - - [ ] Combine all change diffs - - [ ] Add summary header (files created, modified, deleted) - - [ ] Include statistics (lines added/removed) - - [ ] Method `generate_edit_preview(edit: Edit, original: str) -> str`: - - [ ] Show what an edit will change - - [ ] Highlight search/replace matches - - [ ] Tests: Tool-based change tracking tests - - [ ] **C4.6** [Rui] Write Behave scenarios in `features/change_tracking.feature`: - - [ ] Scenario: Skill invocation creates Change record - - [ ] Scenario: Multiple skill calls accumulate in ChangeSet - - [ ] Scenario: ChangeSet correctly tracks skill invocation history - - [ ] Scenario: Failed skill invocation is recorded with error - - [ ] Scenario: Generate unified diff from ChangeSet - - [ ] Scenario: Rollback to specific change point - - [ ] **C4.7** [Rui] Write Behave scenarios in `features/tool_call_routing.feature`: - - [ ] Scenario: Route OpenAI-style tool call to skill - - [ ] Scenario: Route Anthropic-style tool_use to skill - - [ ] Scenario: Validate parameters against skill schema - - [ ] Scenario: Reject tool call for read_only skill trying to write - - [ ] Scenario: Path validation rejects traversal attempt - - [ ] Scenario: Path validation auto-creates directories + - [ ] Code: Implement tool-based change tracking (NOT output parsing) + - [ ] **CRITICAL ARCHITECTURE**: ChangeSet is built from skill/tool invocations, NOT by parsing LLM text output + - [ ] **C4.1** [Luis] Update `src/cleveragents/domain/models/core/change.py`: + - [ ] Enhance `Change` model with: + - [ ] Field `operation: OperationType` - create/modify/delete/move + - [ ] Field `path: str` - target resource path + - [ ] Field `new_path: str | None` - for move operations + - [ ] Field `content: str | None` - full content (for create) + - [ ] Field `edits: list[Edit] | None` - targeted edits (for modify) + - [ ] Field `patch: str | None` - unified diff (generated, not parsed) + - [ ] Field `language: str | None` - detected language + - [ ] Field `skill_invocation_id: str` - which skill call produced this + - [ ] Field `timestamp: datetime` - when change was made + - [ ] Field `validation_result: ValidationResult | None` + - [ ] Define `Edit` model for targeted edits: + - [ ] Field `type: EditType` - search_replace, line_range, insert_after, etc. + - [ ] Field `search: str | None` - text to find (for search_replace) + - [ ] Field `replace: str | None` - replacement text + - [ ] Field `start_line: int | None` - for line-based edits + - [ ] Field `end_line: int | None` - for line-based edits + - [ ] Field `content: str | None` - content to insert + - [ ] Enhance `ChangeSet` model with: + - [ ] Field `changes: list[Change]` - all resource changes + - [ ] Field `warnings: list[str]` - non-blocking issues + - [ ] Field `skill_invocations: list[SkillInvocation]` - full invocation history + - [ ] Field `generated_by_actor: str` - actor that produced this + - [ ] Field `validation: ChangeSetValidation` - overall validation + - [ ] Method `add_change(change: Change)` - append change from skill + - [ ] Method `get_change(path: str) -> Change | None` - find by path + - [ ] Method `get_changes_by_skill(skill_id: str) -> list[Change]` - changes from specific skill + - [ ] Method `file_paths() -> list[str]` - all affected paths + - [ ] Method `to_diff() -> str` - unified diff of all changes + - [ ] Method `rollback_to(change_id: str)` - remove changes after point + - [ ] **C4.2** [Luis] Implement `SkillInvocationTracker` in `src/cleveragents/actor/skills/tracker.py`: + - [ ] **C4.2a** `SkillInvocation` model: + - [ ] Field `id: str` - unique invocation ID + - [ ] Field `skill_name: str` - which skill was called + - [ ] Field `parameters: dict` - input parameters + - [ ] Field `result: Any` - skill return value + - [ ] Field `changes: list[Change]` - resource changes produced + - [ ] Field `timestamp: datetime` - when invoked + - [ ] Field `duration_ms: int` - execution time + - [ ] Field `error: str | None` - if skill failed + - [ ] **C4.2b** `SkillInvocationTracker` class: + - [ ] Method `start_invocation(skill: Skill, params: dict) -> str` - begin tracking + - [ ] Method `record_change(invocation_id: str, change: Change)` - record change + - [ ] Method `complete_invocation(invocation_id: str, result: Any)` - finish tracking + - [ ] Method `fail_invocation(invocation_id: str, error: Exception)` - record failure + - [ ] Method `get_invocations() -> list[SkillInvocation]` - full history + - [ ] Method `build_changeset() -> ChangeSet` - assemble from invocations + - [ ] **C4.3** [Luis] Implement `ToolCallRouter` in `src/cleveragents/actor/skills/router.py`: + - [ ] **C4.3a** Parse LLM tool calls (NOT text output): + - [ ] Handle OpenAI-style tool_calls from response + - [ ] Handle Anthropic-style tool_use blocks + - [ ] Handle LangChain AgentAction format + - [ ] **C4.3b** Route tool calls to skills: + - [ ] Look up skill by name in registry + - [ ] Validate parameters against skill schema + - [ ] Check capability metadata (read_only, writes, etc.) + - [ ] Enforce permission restrictions + - [ ] **C4.3c** Execute with tracking: + - [ ] Start invocation tracking + - [ ] Execute skill in sandbox context + - [ ] Record changes produced by skill + - [ ] Complete invocation tracking + - [ ] Return result to LLM + - [ ] **C4.4** [Luis] Implement resource path validation in `src/cleveragents/actor/skills/path_validator.py`: + - [ ] Validate paths against sandbox boundaries + - [ ] Enforce deny-list patterns (.git/, node_modules/, **pycache**/, etc.) + - [ ] Auto-create parent directories for new file paths + - [ ] Resolve relative paths to absolute sandbox paths + - [ ] Detect path traversal attempts (../) + - [ ] **C4.5** [Luis] Create diff generation in `src/cleveragents/agents/diff_generator.py`: + - [ ] Method `generate_unified_diff(change: Change, sandbox: Sandbox) -> str`: + - [ ] Compare sandbox state with original + - [ ] Generate unified diff format + - [ ] Include file headers with paths + - [ ] Method `generate_changeset_diff(changeset: ChangeSet, sandbox: Sandbox) -> str`: + - [ ] Combine all change diffs + - [ ] Add summary header (files created, modified, deleted) + - [ ] Include statistics (lines added/removed) + - [ ] Method `generate_edit_preview(edit: Edit, original: str) -> str`: + - [ ] Show what an edit will change + - [ ] Highlight search/replace matches + - [ ] Tests: Tool-based change tracking tests + - [ ] **C4.6** [Rui] Write Behave scenarios in `features/change_tracking.feature`: + - [ ] Scenario: Skill invocation creates Change record + - [ ] Scenario: Multiple skill calls accumulate in ChangeSet + - [ ] Scenario: ChangeSet correctly tracks skill invocation history + - [ ] Scenario: Failed skill invocation is recorded with error + - [ ] Scenario: Generate unified diff from ChangeSet + - [ ] Scenario: Rollback to specific change point + - [ ] **C4.7** [Rui] Write Behave scenarios in `features/tool_call_routing.feature`: + - [ ] Scenario: Route OpenAI-style tool call to skill + - [ ] Scenario: Route Anthropic-style tool_use to skill + - [ ] Scenario: Validate parameters against skill schema + - [ ] Scenario: Reject tool call for read_only skill trying to write + - [ ] Scenario: Path validation rejects traversal attempt + - [ ] Scenario: Path validation auto-creates directories - [ ] **Stage C5: Validation Pipeline** (Day 9-11) **[Luis]** - - [ ] Code: Implement real validation gates - - [ ] **C5.1** [Luis] Create `ValidationPipeline` in `src/cleveragents/application/services/validation_service.py`: - - [ ] Method `validate_changeset(changeset: ChangeSet, project: Project) -> ValidationResult`: - - [ ] Run all applicable validators - - [ ] Aggregate results - - [ ] Return pass/fail with details - - [ ] Method `validate_syntax(change: Change) -> ValidationResult`: - - [ ] Detect language from extension - - [ ] Run language-specific syntax check - - [ ] Method `validate_lint(change: Change, config: ValidationConfig) -> ValidationResult`: - - [ ] Run lint command from project config - - [ ] Parse lint output for errors - - [ ] Method `validate_tests(project: Project, config: ValidationConfig) -> ValidationResult`: - - [ ] Run test command from project config - - [ ] Parse test results - - [ ] Method `validate_build(project: Project, config: ValidationConfig) -> ValidationResult`: - - [ ] Run build command from project config - - [ ] Check for build errors - - [ ] **C5.2** [Luis] Implement language-specific validators: - - [ ] Python: `python -m py_compile ` - - [ ] JavaScript/TypeScript: `node --check ` or syntax parse - - [ ] JSON: `json.loads()` validation - - [ ] YAML: `yaml.safe_load()` validation - - [ ] **C5.3** [Luis] Implement validation failure handling: - - [ ] If validation fails, attempt repair loop: - - [ ] Send errors to LLM with request to fix - - [ ] Parse fixed output - - [ ] Re-validate - - [ ] Max 3 repair attempts - - [ ] If repair fails, mark changeset as errored - - [ ] Preserve original output for debugging - - [ ] **C5.4** [Luis] Remove stub validation from existing code: - - [ ] Replace "output length > 10" check with real validation - - [ ] Remove "PASS" stub validation - - [ ] Tests: Validation tests - - [ ] **C5.5** [Rui] Write Behave scenarios in `features/validation_pipeline.feature`: - - [ ] Scenario: Valid Python file passes syntax validation - - [ ] Scenario: Invalid Python file fails with clear error - - [ ] Scenario: Lint errors detected and reported - - [ ] Scenario: Test failures detected and reported - - [ ] Scenario: Validation repair loop fixes simple errors - - [ ] Scenario: Validation repair gives up after max attempts + - [ ] Code: Implement real validation gates + - [ ] **C5.1** [Luis] Create `ValidationPipeline` in `src/cleveragents/application/services/validation_service.py`: + - [ ] Method `validate_changeset(changeset: ChangeSet, project: Project) -> ValidationResult`: + - [ ] Run all applicable validators + - [ ] Aggregate results + - [ ] Return pass/fail with details + - [ ] Method `validate_syntax(change: Change) -> ValidationResult`: + - [ ] Detect language from extension + - [ ] Run language-specific syntax check + - [ ] Method `validate_lint(change: Change, config: ValidationConfig) -> ValidationResult`: + - [ ] Run lint command from project config + - [ ] Parse lint output for errors + - [ ] Method `validate_tests(project: Project, config: ValidationConfig) -> ValidationResult`: + - [ ] Run test command from project config + - [ ] Parse test results + - [ ] Method `validate_build(project: Project, config: ValidationConfig) -> ValidationResult`: + - [ ] Run build command from project config + - [ ] Check for build errors + - [ ] **C5.2** [Luis] Implement language-specific validators: + - [ ] Python: `python -m py_compile ` + - [ ] JavaScript/TypeScript: `node --check ` or syntax parse + - [ ] JSON: `json.loads()` validation + - [ ] YAML: `yaml.safe_load()` validation + - [ ] **C5.3** [Luis] Implement validation failure handling: + - [ ] If validation fails, attempt repair loop: + - [ ] Send errors to LLM with request to fix + - [ ] Parse fixed output + - [ ] Re-validate + - [ ] Max 3 repair attempts + - [ ] If repair fails, mark changeset as errored + - [ ] Preserve original output for debugging + - [ ] **C5.4** [Luis] Remove stub validation from existing code: + - [ ] Replace "output length > 10" check with real validation + - [ ] Remove "PASS" stub validation + - [ ] Tests: Validation tests + - [ ] **C5.5** [Rui] Write Behave scenarios in `features/validation_pipeline.feature`: + - [ ] Scenario: Valid Python file passes syntax validation + - [ ] Scenario: Invalid Python file fails with clear error + - [ ] Scenario: Lint errors detected and reported + - [ ] Scenario: Test failures detected and reported + - [ ] Scenario: Validation repair loop fixes simple errors + - [ ] Scenario: Validation repair gives up after max attempts - [ ] **Stage C6: Built-in Provider Actors** (Day 10-11) **[Aditya]** - - [ ] Code: Create built-in actors for each provider - - [ ] **C6.1** [Aditya] Generate built-in actor configs in `src/cleveragents/actor/builtins.py`: - - [ ] `openai/gpt-4` - GPT-4 wrapper - - [ ] `openai/gpt-4-turbo` - GPT-4 Turbo wrapper - - [ ] `openai/gpt-3.5-turbo` - GPT-3.5 wrapper - - [ ] `anthropic/claude-3-opus` - Claude 3 Opus wrapper - - [ ] `anthropic/claude-3-sonnet` - Claude 3 Sonnet wrapper - - [ ] `anthropic/claude-3-haiku` - Claude 3 Haiku wrapper - - [ ] `google/gemini-pro` - Gemini Pro wrapper - - [ ] `google/gemini-ultra` - Gemini Ultra wrapper - - [ ] **C6.2** [Aditya] Ensure built-in actors work as strategy/execution actors: - - [ ] Add appropriate system prompts for each role - - [ ] Configure temperature defaults (lower for execution) - - [ ] Test with plan lifecycle - - [ ] **C6.3** [Aditya] Implement provider capability detection: - - [ ] Check which API keys are configured - - [ ] Only register actors for available providers - - [ ] Clear error message for unavailable providers - - [ ] Tests: Verify built-in actors work in plan lifecycle - - [ ] **C6.4** [Rui] Write Behave scenarios in `features/builtin_actors.feature`: - - [ ] Scenario: Built-in OpenAI actor loads correctly - - [ ] Scenario: Built-in Anthropic actor loads correctly - - [ ] Scenario: Built-in actor can be used as strategy actor - - [ ] Scenario: Missing API key produces clear error + - [ ] Code: Create built-in actors for each provider + - [ ] **C6.1** [Aditya] Generate built-in actor configs in `src/cleveragents/actor/builtins.py`: + - [ ] `openai/gpt-4` - GPT-4 wrapper + - [ ] `openai/gpt-4-turbo` - GPT-4 Turbo wrapper + - [ ] `openai/gpt-3.5-turbo` - GPT-3.5 wrapper + - [ ] `anthropic/claude-3-opus` - Claude 3 Opus wrapper + - [ ] `anthropic/claude-3-sonnet` - Claude 3 Sonnet wrapper + - [ ] `anthropic/claude-3-haiku` - Claude 3 Haiku wrapper + - [ ] `google/gemini-pro` - Gemini Pro wrapper + - [ ] `google/gemini-ultra` - Gemini Ultra wrapper + - [ ] **C6.2** [Aditya] Ensure built-in actors work as strategy/execution actors: + - [ ] Add appropriate system prompts for each role + - [ ] Configure temperature defaults (lower for execution) + - [ ] Test with plan lifecycle + - [ ] **C6.3** [Aditya] Implement provider capability detection: + - [ ] Check which API keys are configured + - [ ] Only register actors for available providers + - [ ] Clear error message for unavailable providers + - [ ] Tests: Verify built-in actors work in plan lifecycle + - [ ] **C6.4** [Rui] Write Behave scenarios in `features/builtin_actors.feature`: + - [ ] Scenario: Built-in OpenAI actor loads correctly + - [ ] Scenario: Built-in Anthropic actor loads correctly + - [ ] Scenario: Built-in actor can be used as strategy actor + - [ ] Scenario: Missing API key produces clear error - [ ] **Stage C7: Plan-Actor Integration** (Day 11-14) **[Aditya + Luis]** - - [ ] Code: Connect actors to plan lifecycle - - [ ] **C7.1** [Aditya] Update `PlanLifecycleService.execute_strategize()`: - - [ ] Load strategy_actor from action - - [ ] Compile actor to LangGraph - - [ ] Build strategy context: - - [ ] Project resources - - [ ] Plan description and arguments - - [ ] Definition of done - - [ ] Invoke actor graph with context - - [ ] Parse strategy output - - [ ] Store strategy in plan - - [ ] **C7.2** [Aditya] Implement dependency closure computation: - - [ ] Method `compute_closure(target: str, project: Project) -> ResourceClosure`: - - [ ] Find direct imports/includes - - [ ] Find symbol dependencies - - [ ] Find test dependencies - - [ ] Find build references - - [ ] Integrate with strategy actor context - - [ ] **C7.3** [Luis] Update `PlanLifecycleService.execute_execution()`: - - [ ] Load execution_actor from action - - [ ] Compile actor to LangGraph - - [ ] Build execution context: - - [ ] Strategy output - - [ ] Resource service for sandbox access - - [ ] Bounded dependency closure - - [ ] Invoke actor graph with context - - [ ] Parse output as ChangeSet - - [ ] Run validation pipeline - - [ ] Handle subplan spawning - - [ ] **C7.4** [Luis] Update `PlanLifecycleService.apply_plan()`: - - [ ] Verify execution completed successfully - - [ ] Commit all sandboxes - - [ ] Apply ChangeSet to resources - - [ ] Record applied artifacts - - [ ] Clean up sandboxes - - [ ] **C7.4a** [Luis] Implement ATOMIC apply: - - [ ] Apply must be all-or-nothing: - - [ ] Write all changes to temp files first - - [ ] Validate all writes succeeded - - [ ] Atomic rename/swap to final locations - - [ ] If any step fails, rollback all changes - - [ ] In git mode: - - [ ] All changes in single commit - - [ ] If commit fails, no partial changes applied - - [ ] Handle partial failure: - - [ ] Preserve sandbox for inspection - - [ ] Clear error message about what failed - - [ ] Allow retry after manual fix - - [ ] **C7.5** [Aditya] Implement hierarchical task decomposition: - - [ ] Strategy actor can emit subplan decisions - - [ ] Each subplan gets bounded context - - [ ] Parallel or sequential execution modes - - [ ] **C7.6** [Luis] Connect output parser to execution flow: - - [ ] After actor produces output, parse to ChangeSet - - [ ] Validate ChangeSet - - [ ] Store ChangeSet in plan - - [ ] **C7.7** [Luis] Implement diff review artifact storage: - - [ ] Store generated diff in plan metadata - - [ ] Create `DiffArtifact` model: - - [ ] Field `diff_id: str` - ULID - - [ ] Field `plan_id: str` - parent plan - - [ ] Field `unified_diff: str` - full unified diff - - [ ] Field `file_summaries: list[FileSummary]` - per-file summary - - [ ] Field `risk_markers: list[str]` - touched auth code, migrations, etc. - - [ ] Field `created_at: datetime` - - [ ] Display diff in `agents [--data-dir PATH] [--config-path PATH] plan diff` command - - [ ] Display diff before apply in review-before-apply mode - - [ ] Tests: End-to-end tests for full plan lifecycle with actors - - [ ] **C7.7** [Rui] Write Behave scenarios in `features/plan_actor_integration.feature`: - - [ ] Scenario: Full lifecycle with LLM actors (mocked) - - [ ] Scenario: Strategy actor receives correct context - - [ ] Scenario: Execution actor receives strategy output - - [ ] Scenario: ChangeSet applied correctly - - [ ] Scenario: Validation failure triggers repair loop - - [ ] **C7.8** [Rui] Write Robot integration test `robot/plan_actor_integration.robot`: - - [ ] Test: Full lifecycle with real sandbox - - [ ] Test: Multi-file generation and application - - [ ] Tests: Dependency closure computation accuracy - - [ ] **C7.9** [Rui] Write Behave scenarios in `features/dependency_closure.feature`: - - [ ] Scenario: Python imports detected correctly - - [ ] Scenario: Test file dependencies included + - [ ] Code: Connect actors to plan lifecycle + - [ ] **C7.1** [Aditya] Update `PlanLifecycleService.execute_strategize()`: + - [ ] Load strategy_actor from action + - [ ] Compile actor to LangGraph + - [ ] Build strategy context: + - [ ] Project resources + - [ ] Plan description and arguments + - [ ] Definition of done + - [ ] Invoke actor graph with context + - [ ] Parse strategy output + - [ ] Store strategy in plan + - [ ] **C7.2** [Aditya] Implement dependency closure computation: + - [ ] Method `compute_closure(target: str, project: Project) -> ResourceClosure`: + - [ ] Find direct imports/includes + - [ ] Find symbol dependencies + - [ ] Find test dependencies + - [ ] Find build references + - [ ] Integrate with strategy actor context + - [ ] **C7.3** [Luis] Update `PlanLifecycleService.execute_execution()`: + - [ ] Load execution_actor from action + - [ ] Compile actor to LangGraph + - [ ] Build execution context: + - [ ] Strategy output + - [ ] Resource service for sandbox access + - [ ] Bounded dependency closure + - [ ] Invoke actor graph with context + - [ ] Parse output as ChangeSet + - [ ] Run validation pipeline + - [ ] Handle subplan spawning + - [ ] **C7.4** [Luis] Update `PlanLifecycleService.apply_plan()`: + - [ ] Verify execution completed successfully + - [ ] Commit all sandboxes + - [ ] Apply ChangeSet to resources + - [ ] Record applied artifacts + - [ ] Clean up sandboxes + - [ ] **C7.4a** [Luis] Implement ATOMIC apply: + - [ ] Apply must be all-or-nothing: + - [ ] Write all changes to temp files first + - [ ] Validate all writes succeeded + - [ ] Atomic rename/swap to final locations + - [ ] If any step fails, rollback all changes + - [ ] In git mode: + - [ ] All changes in single commit + - [ ] If commit fails, no partial changes applied + - [ ] Handle partial failure: + - [ ] Preserve sandbox for inspection + - [ ] Clear error message about what failed + - [ ] Allow retry after manual fix + - [ ] **C7.5** [Aditya] Implement hierarchical task decomposition: + - [ ] Strategy actor can emit subplan decisions + - [ ] Each subplan gets bounded context + - [ ] Parallel or sequential execution modes + - [ ] **C7.6** [Luis] Connect output parser to execution flow: + - [ ] After actor produces output, parse to ChangeSet + - [ ] Validate ChangeSet + - [ ] Store ChangeSet in plan + - [ ] **C7.7** [Luis] Implement diff review artifact storage: + - [ ] Store generated diff in plan metadata + - [ ] Create `DiffArtifact` model: + - [ ] Field `diff_id: str` - ULID + - [ ] Field `plan_id: str` - parent plan + - [ ] Field `unified_diff: str` - full unified diff + - [ ] Field `file_summaries: list[FileSummary]` - per-file summary + - [ ] Field `risk_markers: list[str]` - touched auth code, migrations, etc. + - [ ] Field `created_at: datetime` + - [ ] Display diff in `agents [--data-dir PATH] [--config-path PATH] plan diff` command + - [ ] Display diff before apply in review-before-apply mode + - [ ] Tests: End-to-end tests for full plan lifecycle with actors + - [ ] **C7.7** [Rui] Write Behave scenarios in `features/plan_actor_integration.feature`: + - [ ] Scenario: Full lifecycle with LLM actors (mocked) + - [ ] Scenario: Strategy actor receives correct context + - [ ] Scenario: Execution actor receives strategy output + - [ ] Scenario: ChangeSet applied correctly + - [ ] Scenario: Validation failure triggers repair loop + - [ ] **C7.8** [Rui] Write Robot integration test `robot/plan_actor_integration.robot`: + - [ ] Test: Full lifecycle with real sandbox + - [ ] Test: Multi-file generation and application + - [ ] Tests: Dependency closure computation accuracy + - [ ] **C7.9** [Rui] Write Behave scenarios in `features/dependency_closure.feature`: + - [ ] Scenario: Python imports detected correctly + - [ ] Scenario: Test file dependencies included **M3 SUCCESS CRITERIA**: + - [ ] Can define actors in YAML with skills - [ ] Actors compile to LangGraph graphs - [ ] Skills can execute inline Python code @@ -4833,1753 +5043,1909 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation - [ ] **Stage D1: Decision Data Model** (Day 15-16) **[Hamza - Well Rounded]** - **SEQUENTIAL ORDER**: D1.1 (Enums) → D1.2 (ContextSnapshot) → D1.3 (Decision model) → D1.4 (Helpers) → D1.5 (Tests) + **SEQUENTIAL ORDER**: D1.1 (Enums) → D1.2 (ContextSnapshot) → D1.3 (Decision model) → D1.4 (Helpers) → D1.5 (Tests) + - [ ] Code: Create Decision domain model + - [ ] **D1.1** [Hamza] Define `DecisionType` enum in `src/cleveragents/domain/models/core/decision.py`: + - [ ] **D1.1a** [Hamza] Create file with DecisionType enum: - - [ ] Code: Create Decision domain model - - [ ] **D1.1** [Hamza] Define `DecisionType` enum in `src/cleveragents/domain/models/core/decision.py`: - - [ ] **D1.1a** [Hamza] Create file with DecisionType enum: - ```python - from enum import Enum + ```python + from enum import Enum - class DecisionType(str, Enum): - """Classification of decision points in plan execution.""" + class DecisionType(str, Enum): + """Classification of decision points in plan execution.""" - # Root decisions - PROMPT_DEFINITION = "prompt_definition" # Initial plan prompt + # Root decisions + PROMPT_DEFINITION = "prompt_definition" # Initial plan prompt - # Strategy phase decisions - STRATEGY_CHOICE = "strategy_choice" # High-level approach - IMPLEMENTATION_CHOICE = "implementation_choice" # How to implement - RESOURCE_SELECTION = "resource_selection" # Which resources to use + # Strategy phase decisions + STRATEGY_CHOICE = "strategy_choice" # High-level approach + IMPLEMENTATION_CHOICE = "implementation_choice" # How to implement + RESOURCE_SELECTION = "resource_selection" # Which resources to use - # Execution phase decisions - SUBPLAN_SPAWN = "subplan_spawn" # Decision to create subplan - TOOL_INVOCATION = "tool_invocation" # Which tool/skill to use + # Execution phase decisions + SUBPLAN_SPAWN = "subplan_spawn" # Decision to create subplan + TOOL_INVOCATION = "tool_invocation" # Which tool/skill to use - # Error handling decisions - ERROR_RECOVERY = "error_recovery" # How to handle failure - VALIDATION_RESPONSE = "validation_response" # Response to validation failure + # Error handling decisions + ERROR_RECOVERY = "error_recovery" # How to handle failure + VALIDATION_RESPONSE = "validation_response" # Response to validation failure - # User interaction decisions - USER_INTERVENTION = "user_intervention" # User provided guidance - ``` - - [ ] Commit: "feat(domain): define DecisionType enum" - - [ ] **D1.1b** [Hamza] Add helper method for decision classification: - ```python - @classmethod - def is_strategy_decision(cls, decision_type: "DecisionType") -> bool: - """Check if this is a strategy phase decision.""" - return decision_type in { - cls.PROMPT_DEFINITION, cls.STRATEGY_CHOICE, - cls.IMPLEMENTATION_CHOICE, cls.RESOURCE_SELECTION - } + # User interaction decisions + USER_INTERVENTION = "user_intervention" # User provided guidance + ``` - @classmethod - def is_execution_decision(cls, decision_type: "DecisionType") -> bool: - """Check if this is an execution phase decision.""" - return decision_type in {cls.SUBPLAN_SPAWN, cls.TOOL_INVOCATION} - ``` - - [ ] Commit: "feat(domain): add DecisionType helper methods" - - [ ] **D1.2** [Hamza] Define `ContextSnapshot` model: - - [ ] **D1.2a** [Hamza] Create ContextSnapshot dataclass: - ```python - @dataclass(frozen=True) - class ContextSnapshot: - """Snapshot of context at decision point for replay.""" + - [ ] Commit: "feat(domain): define DecisionType enum" - snapshot_id: str # ULID - hot_context_hash: str # SHA-256 hash of hot context content - hot_context_ref: str # Storage reference (file path or blob ID) - relevant_resources: tuple[str, ...] # Resource IDs in scope - actor_state_ref: str | None # LangGraph checkpoint ID - file_versions: dict[str, str] # path -> git commit or hash - created_at: datetime - ``` - - [ ] Commit: "feat(domain): define ContextSnapshot dataclass" - - [ ] **D1.2b** [Hamza] Add factory method: - ```python - @classmethod - def capture( - cls, - hot_context: str, - resources: list[str], - actor_state: str | None = None, - file_versions: dict[str, str] | None = None - ) -> "ContextSnapshot": - """Capture a snapshot of current context.""" - import hashlib - import ulid + - [ ] **D1.1b** [Hamza] Add helper method for decision classification: - return cls( - snapshot_id=ulid.new().str, - hot_context_hash=hashlib.sha256(hot_context.encode()).hexdigest(), - hot_context_ref="", # Set by storage layer - relevant_resources=tuple(resources), - actor_state_ref=actor_state, - file_versions=file_versions or {}, - created_at=datetime.utcnow() - ) - ``` - - [ ] Commit: "feat(domain): add ContextSnapshot.capture() factory" - - [ ] **D1.3** [Hamza] Define `Decision` Pydantic model: - - [ ] **D1.3a** [Hamza] Create Decision class with identity fields: - ```python - class Decision(BaseModel): - """A recorded decision point in plan execution.""" + ```python + @classmethod + def is_strategy_decision(cls, decision_type: "DecisionType") -> bool: + """Check if this is a strategy phase decision.""" + return decision_type in { + cls.PROMPT_DEFINITION, cls.STRATEGY_CHOICE, + cls.IMPLEMENTATION_CHOICE, cls.RESOURCE_SELECTION + } - model_config = ConfigDict(frozen=True) + @classmethod + def is_execution_decision(cls, decision_type: "DecisionType") -> bool: + """Check if this is an execution phase decision.""" + return decision_type in {cls.SUBPLAN_SPAWN, cls.TOOL_INVOCATION} + ``` - # Identity - decision_id: str = Field(..., description="ULID identifier") - plan_id: str = Field(..., description="Parent plan ULID") + - [ ] Commit: "feat(domain): add DecisionType helper methods" - # Tree structure - parent_decision_id: str | None = Field( - default=None, description="Parent in decision tree" - ) - sequence_number: int = Field( - ..., ge=0, description="Order within plan (0=root)" - ) - ``` - - [ ] Commit: "feat(domain): add Decision model identity fields" - - [ ] **D1.3b** [Hamza] Add decision content fields: - ```python - # Decision content - decision_type: DecisionType = Field(..., description="Classification") - question: str = Field(..., min_length=1, description="What was decided") - chosen_option: str = Field(..., min_length=1, description="The choice made") - alternatives_considered: list[str] = Field( - default_factory=list, description="Other options evaluated" - ) - confidence_score: float | None = Field( - default=None, ge=0.0, le=1.0, description="AI confidence 0.0-1.0" - ) - rationale: str = Field(default="", description="Why this choice") - actor_reasoning: str | None = Field( - default=None, description="Raw LLM chain-of-thought" - ) - ``` - - [ ] Commit: "feat(domain): add Decision content fields" - - [ ] **D1.3c** [Hamza] Add context and relationship fields: - ```python - # Context for replay - context_snapshot: ContextSnapshot = Field( - ..., description="Snapshot at decision time" - ) - checkpoint_id: str | None = Field( - default=None, description="Sandbox checkpoint for rollback" - ) + - [ ] **D1.2** [Hamza] Define `ContextSnapshot` model: + - [ ] **D1.2a** [Hamza] Create ContextSnapshot dataclass: - # Downstream relationships (populated during execution) - downstream_decision_ids: list[str] = Field( - default_factory=list, description="Decisions that depend on this" - ) - downstream_plan_ids: list[str] = Field( - default_factory=list, description="Subplans spawned from this" - ) - artifacts_produced: list[str] = Field( - default_factory=list, description="Artifact IDs created" - ) - ``` - - [ ] Commit: "feat(domain): add Decision context and relationship fields" - - [ ] **D1.3d** [Hamza] Add correction tracking fields: - ```python - # Correction tracking - is_correction: bool = Field( - default=False, description="Is this a corrected decision" - ) - corrects_decision_id: str | None = Field( - default=None, description="Original decision this corrects" - ) - superseded_by: str | None = Field( - default=None, description="Decision that replaced this one" - ) + ```python + @dataclass(frozen=True) + class ContextSnapshot: + """Snapshot of context at decision point for replay.""" - # Timestamps - created_at: datetime = Field(default_factory=datetime.utcnow) - ``` - - [ ] Commit: "feat(domain): add Decision correction tracking fields" - - [ ] **D1.3e** [Hamza] Add validators: - ```python - @field_validator('decision_id', 'plan_id') - @classmethod - def validate_ulid(cls, v: str) -> str: - """Validate ULID format.""" - if len(v) != 26 or not v.isalnum(): - raise ValueError(f"Invalid ULID format: {v}") - return v + snapshot_id: str # ULID + hot_context_hash: str # SHA-256 hash of hot context content + hot_context_ref: str # Storage reference (file path or blob ID) + relevant_resources: tuple[str, ...] # Resource IDs in scope + actor_state_ref: str | None # LangGraph checkpoint ID + file_versions: dict[str, str] # path -> git commit or hash + created_at: datetime + ``` - @model_validator(mode='after') - def validate_correction_consistency(self) -> Self: - """Ensure correction fields are consistent.""" - if self.is_correction and not self.corrects_decision_id: - raise ValueError("Correction must specify corrects_decision_id") - if self.corrects_decision_id and not self.is_correction: - raise ValueError("corrects_decision_id requires is_correction=True") - return self - ``` - - [ ] Commit: "feat(domain): add Decision validators" - - [ ] **D1.4** [Hamza] Add Decision helper methods: - - [ ] **D1.4a** [Hamza] Add computed properties: - ```python - @property - def is_root(self) -> bool: - """Check if this is the root decision (no parent).""" - return self.parent_decision_id is None + - [ ] Commit: "feat(domain): define ContextSnapshot dataclass" - @property - def is_superseded(self) -> bool: - """Check if this decision has been replaced.""" - return self.superseded_by is not None + - [ ] **D1.2b** [Hamza] Add factory method: - @property - def has_downstream_work(self) -> bool: - """Check if this decision spawned work.""" - return bool(self.downstream_decision_ids or self.downstream_plan_ids) + ```python + @classmethod + def capture( + cls, + hot_context: str, + resources: list[str], + actor_state: str | None = None, + file_versions: dict[str, str] | None = None + ) -> "ContextSnapshot": + """Capture a snapshot of current context.""" + import hashlib + import ulid - @property - def summary(self) -> str: - """Short summary for display.""" - q = self.question[:50] + "..." if len(self.question) > 50 else self.question - return f"[{self.decision_type.value}] {q}" - ``` - - [ ] Commit: "feat(domain): add Decision computed properties" - - [ ] **D1.4b** [Hamza] Add mutation methods (return new instance): - ```python - def with_downstream_decision(self, decision_id: str) -> "Decision": - """Return new Decision with added downstream decision.""" - return self.model_copy(update={ - "downstream_decision_ids": [*self.downstream_decision_ids, decision_id] - }) + return cls( + snapshot_id=ulid.new().str, + hot_context_hash=hashlib.sha256(hot_context.encode()).hexdigest(), + hot_context_ref="", # Set by storage layer + relevant_resources=tuple(resources), + actor_state_ref=actor_state, + file_versions=file_versions or {}, + created_at=datetime.utcnow() + ) + ``` - def with_downstream_plan(self, plan_id: str) -> "Decision": - """Return new Decision with added downstream plan.""" - return self.model_copy(update={ - "downstream_plan_ids": [*self.downstream_plan_ids, plan_id] - }) + - [ ] Commit: "feat(domain): add ContextSnapshot.capture() factory" - def with_artifact(self, artifact_id: str) -> "Decision": - """Return new Decision with added artifact.""" - return self.model_copy(update={ - "artifacts_produced": [*self.artifacts_produced, artifact_id] - }) + - [ ] **D1.3** [Hamza] Define `Decision` Pydantic model: + - [ ] **D1.3a** [Hamza] Create Decision class with identity fields: - def mark_superseded(self, by_decision_id: str) -> "Decision": - """Return new Decision marked as superseded.""" - return self.model_copy(update={"superseded_by": by_decision_id}) - ``` - - [ ] Commit: "feat(domain): add Decision mutation methods" - - [ ] Tests: Behave scenarios for decision model - - [ ] **D1.5** [Rui] Write Behave scenarios in `features/decision_model.feature`: - - [ ] **D1.5a** [Rui] Creation scenarios: - - [ ] Scenario: Create decision with all required fields - - [ ] Given valid decision_id, plan_id, question, chosen_option, context_snapshot - - [ ] When I create a Decision with these fields - - [ ] Then the Decision is created successfully - - [ ] And sequence_number defaults to provided value - - [ ] Scenario: Create root decision (no parent) - - [ ] When I create a Decision with parent_decision_id=None - - [ ] Then is_root property returns True - - [ ] Scenario: Create child decision - - [ ] When I create a Decision with parent_decision_id set - - [ ] Then is_root property returns False - - [ ] Commit: "test(behave): add decision creation scenarios" - - [ ] **D1.5b** [Rui] Validation scenarios: - - [ ] Scenario: Invalid ULID format rejected - - [ ] When I create a Decision with decision_id="invalid" - - [ ] Then validation error is raised - - [ ] Scenario: Confidence score must be 0.0-1.0 - - [ ] When I create a Decision with confidence_score=1.5 - - [ ] Then validation error is raised - - [ ] Scenario: Correction without corrects_decision_id fails - - [ ] When I create a Decision with is_correction=True and corrects_decision_id=None - - [ ] Then validation error mentions correction consistency - - [ ] Commit: "test(behave): add decision validation scenarios" - - [ ] **D1.5c** [Rui] DecisionType scenarios: - - [ ] Scenario: Each decision type validates correctly - - [ ] For each DecisionType enum value - - [ ] When I create a Decision with that type - - [ ] Then decision is created successfully - - [ ] Scenario: is_strategy_decision helper works - - [ ] Given DecisionType.STRATEGY_CHOICE - - [ ] Then DecisionType.is_strategy_decision() returns True - - [ ] Given DecisionType.TOOL_INVOCATION - - [ ] Then DecisionType.is_strategy_decision() returns False - - [ ] Commit: "test(behave): add DecisionType scenarios" - - [ ] **D1.5d** [Rui] Context snapshot scenarios: - - [ ] Scenario: ContextSnapshot.capture() creates valid snapshot - - [ ] Given hot_context string and resource list - - [ ] When I call ContextSnapshot.capture() - - [ ] Then snapshot has valid ULID - - [ ] And hot_context_hash is SHA-256 of content - - [ ] Scenario: ContextSnapshot is immutable - - [ ] Given a ContextSnapshot instance - - [ ] When I try to modify a field - - [ ] Then FrozenInstanceError is raised - - [ ] Commit: "test(behave): add ContextSnapshot scenarios" + ```python + class Decision(BaseModel): + """A recorded decision point in plan execution.""" + + model_config = ConfigDict(frozen=True) + + # Identity + decision_id: str = Field(..., description="ULID identifier") + plan_id: str = Field(..., description="Parent plan ULID") + + # Tree structure + parent_decision_id: str | None = Field( + default=None, description="Parent in decision tree" + ) + sequence_number: int = Field( + ..., ge=0, description="Order within plan (0=root)" + ) + ``` + + - [ ] Commit: "feat(domain): add Decision model identity fields" + + - [ ] **D1.3b** [Hamza] Add decision content fields: + + ```python + # Decision content + decision_type: DecisionType = Field(..., description="Classification") + question: str = Field(..., min_length=1, description="What was decided") + chosen_option: str = Field(..., min_length=1, description="The choice made") + alternatives_considered: list[str] = Field( + default_factory=list, description="Other options evaluated" + ) + confidence_score: float | None = Field( + default=None, ge=0.0, le=1.0, description="AI confidence 0.0-1.0" + ) + rationale: str = Field(default="", description="Why this choice") + actor_reasoning: str | None = Field( + default=None, description="Raw LLM chain-of-thought" + ) + ``` + + - [ ] Commit: "feat(domain): add Decision content fields" + + - [ ] **D1.3c** [Hamza] Add context and relationship fields: + + ```python + # Context for replay + context_snapshot: ContextSnapshot = Field( + ..., description="Snapshot at decision time" + ) + checkpoint_id: str | None = Field( + default=None, description="Sandbox checkpoint for rollback" + ) + + # Downstream relationships (populated during execution) + downstream_decision_ids: list[str] = Field( + default_factory=list, description="Decisions that depend on this" + ) + downstream_plan_ids: list[str] = Field( + default_factory=list, description="Subplans spawned from this" + ) + artifacts_produced: list[str] = Field( + default_factory=list, description="Artifact IDs created" + ) + ``` + + - [ ] Commit: "feat(domain): add Decision context and relationship fields" + + - [ ] **D1.3d** [Hamza] Add correction tracking fields: + + ```python + # Correction tracking + is_correction: bool = Field( + default=False, description="Is this a corrected decision" + ) + corrects_decision_id: str | None = Field( + default=None, description="Original decision this corrects" + ) + superseded_by: str | None = Field( + default=None, description="Decision that replaced this one" + ) + + # Timestamps + created_at: datetime = Field(default_factory=datetime.utcnow) + ``` + + - [ ] Commit: "feat(domain): add Decision correction tracking fields" + + - [ ] **D1.3e** [Hamza] Add validators: + + ```python + @field_validator('decision_id', 'plan_id') + @classmethod + def validate_ulid(cls, v: str) -> str: + """Validate ULID format.""" + if len(v) != 26 or not v.isalnum(): + raise ValueError(f"Invalid ULID format: {v}") + return v + + @model_validator(mode='after') + def validate_correction_consistency(self) -> Self: + """Ensure correction fields are consistent.""" + if self.is_correction and not self.corrects_decision_id: + raise ValueError("Correction must specify corrects_decision_id") + if self.corrects_decision_id and not self.is_correction: + raise ValueError("corrects_decision_id requires is_correction=True") + return self + ``` + + - [ ] Commit: "feat(domain): add Decision validators" + + - [ ] **D1.4** [Hamza] Add Decision helper methods: + - [ ] **D1.4a** [Hamza] Add computed properties: + + ```python + @property + def is_root(self) -> bool: + """Check if this is the root decision (no parent).""" + return self.parent_decision_id is None + + @property + def is_superseded(self) -> bool: + """Check if this decision has been replaced.""" + return self.superseded_by is not None + + @property + def has_downstream_work(self) -> bool: + """Check if this decision spawned work.""" + return bool(self.downstream_decision_ids or self.downstream_plan_ids) + + @property + def summary(self) -> str: + """Short summary for display.""" + q = self.question[:50] + "..." if len(self.question) > 50 else self.question + return f"[{self.decision_type.value}] {q}" + ``` + + - [ ] Commit: "feat(domain): add Decision computed properties" + + - [ ] **D1.4b** [Hamza] Add mutation methods (return new instance): + + ```python + def with_downstream_decision(self, decision_id: str) -> "Decision": + """Return new Decision with added downstream decision.""" + return self.model_copy(update={ + "downstream_decision_ids": [*self.downstream_decision_ids, decision_id] + }) + + def with_downstream_plan(self, plan_id: str) -> "Decision": + """Return new Decision with added downstream plan.""" + return self.model_copy(update={ + "downstream_plan_ids": [*self.downstream_plan_ids, plan_id] + }) + + def with_artifact(self, artifact_id: str) -> "Decision": + """Return new Decision with added artifact.""" + return self.model_copy(update={ + "artifacts_produced": [*self.artifacts_produced, artifact_id] + }) + + def mark_superseded(self, by_decision_id: str) -> "Decision": + """Return new Decision marked as superseded.""" + return self.model_copy(update={"superseded_by": by_decision_id}) + ``` + + - [ ] Commit: "feat(domain): add Decision mutation methods" + + - [ ] Tests: Behave scenarios for decision model + - [ ] **D1.5** [Rui] Write Behave scenarios in `features/decision_model.feature`: + - [ ] **D1.5a** [Rui] Creation scenarios: + - [ ] Scenario: Create decision with all required fields + - [ ] Given valid decision_id, plan_id, question, chosen_option, context_snapshot + - [ ] When I create a Decision with these fields + - [ ] Then the Decision is created successfully + - [ ] And sequence_number defaults to provided value + - [ ] Scenario: Create root decision (no parent) + - [ ] When I create a Decision with parent_decision_id=None + - [ ] Then is_root property returns True + - [ ] Scenario: Create child decision + - [ ] When I create a Decision with parent_decision_id set + - [ ] Then is_root property returns False + - [ ] Commit: "test(behave): add decision creation scenarios" + - [ ] **D1.5b** [Rui] Validation scenarios: + - [ ] Scenario: Invalid ULID format rejected + - [ ] When I create a Decision with decision_id="invalid" + - [ ] Then validation error is raised + - [ ] Scenario: Confidence score must be 0.0-1.0 + - [ ] When I create a Decision with confidence_score=1.5 + - [ ] Then validation error is raised + - [ ] Scenario: Correction without corrects_decision_id fails + - [ ] When I create a Decision with is_correction=True and corrects_decision_id=None + - [ ] Then validation error mentions correction consistency + - [ ] Commit: "test(behave): add decision validation scenarios" + - [ ] **D1.5c** [Rui] DecisionType scenarios: + - [ ] Scenario: Each decision type validates correctly + - [ ] For each DecisionType enum value + - [ ] When I create a Decision with that type + - [ ] Then decision is created successfully + - [ ] Scenario: is_strategy_decision helper works + - [ ] Given DecisionType.STRATEGY_CHOICE + - [ ] Then DecisionType.is_strategy_decision() returns True + - [ ] Given DecisionType.TOOL_INVOCATION + - [ ] Then DecisionType.is_strategy_decision() returns False + - [ ] Commit: "test(behave): add DecisionType scenarios" + - [ ] **D1.5d** [Rui] Context snapshot scenarios: + - [ ] Scenario: ContextSnapshot.capture() creates valid snapshot + - [ ] Given hot_context string and resource list + - [ ] When I call ContextSnapshot.capture() + - [ ] Then snapshot has valid ULID + - [ ] And hot_context_hash is SHA-256 of content + - [ ] Scenario: ContextSnapshot is immutable + - [ ] Given a ContextSnapshot instance + - [ ] When I try to modify a field + - [ ] Then FrozenInstanceError is raised + - [ ] Commit: "test(behave): add ContextSnapshot scenarios" - [ ] **Stage D2: Decision Recording** (Day 16-18) **[Hamza]** - **SEQUENTIAL ORDER**: D2.1 (Service scaffold) → D2.2 (record_decision) → D2.3 (tree queries) → D2.4 (context capture) → D2.5 (strategy integration) → D2.6 (downstream updates) → D2.7 (Tests) + **SEQUENTIAL ORDER**: D2.1 (Service scaffold) → D2.2 (record_decision) → D2.3 (tree queries) → D2.4 (context capture) → D2.5 (strategy integration) → D2.6 (downstream updates) → D2.7 (Tests) + - [ ] Code: Record decisions during Strategize + - [ ] **D2.1** [Hamza] Create `DecisionService` scaffold in `src/cleveragents/application/services/decision_service.py`: + - [ ] **D2.1a** [Hamza] Define service class with dependencies: - - [ ] Code: Record decisions during Strategize - - [ ] **D2.1** [Hamza] Create `DecisionService` scaffold in `src/cleveragents/application/services/decision_service.py`: - - [ ] **D2.1a** [Hamza] Define service class with dependencies: - ```python - class DecisionService: - """Service for recording and querying decisions.""" + ```python + class DecisionService: + """Service for recording and querying decisions.""" - def __init__( - self, - decision_repo: DecisionRepository, - snapshot_store: ContextSnapshotStore, - plan_repo: LifecyclePlanRepository - ): - self._decision_repo = decision_repo - self._snapshot_store = snapshot_store - self._plan_repo = plan_repo - self._sequence_counters: dict[str, int] = {} # plan_id -> next sequence - ``` - - [ ] Commit: "feat(service): add DecisionService scaffold" - - [ ] **D2.1b** [Hamza] Define ContextSnapshotStore protocol: - ```python - class ContextSnapshotStore(Protocol): - """Protocol for storing context snapshots.""" + def __init__( + self, + decision_repo: DecisionRepository, + snapshot_store: ContextSnapshotStore, + plan_repo: LifecyclePlanRepository + ): + self._decision_repo = decision_repo + self._snapshot_store = snapshot_store + self._plan_repo = plan_repo + self._sequence_counters: dict[str, int] = {} # plan_id -> next sequence + ``` - def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: - """Store snapshot content and return with ref set.""" - ... + - [ ] Commit: "feat(service): add DecisionService scaffold" - def retrieve(self, snapshot_id: str) -> tuple[ContextSnapshot, str]: - """Retrieve snapshot and its content.""" - ... + - [ ] **D2.1b** [Hamza] Define ContextSnapshotStore protocol: - def retrieve_by_hash(self, hash: str) -> tuple[ContextSnapshot, str] | None: - """Retrieve by content hash (for deduplication).""" - ... - ``` - - [ ] Commit: "feat(service): define ContextSnapshotStore protocol" - - [ ] **D2.2** [Hamza] Implement `record_decision()` method: - - [ ] **D2.2a** [Hamza] Core implementation: - ```python - def record_decision( - self, - plan_id: str, - decision_type: DecisionType, - question: str, - chosen_option: str, - hot_context: str, - resources: list[str], - parent_decision_id: str | None = None, - alternatives: list[str] | None = None, - confidence: float | None = None, - rationale: str = "", - actor_reasoning: str | None = None, - checkpoint_id: str | None = None - ) -> Decision: - """Record a new decision for a plan.""" - import ulid + ```python + class ContextSnapshotStore(Protocol): + """Protocol for storing context snapshots.""" - # Generate IDs - decision_id = ulid.new().str + def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: + """Store snapshot content and return with ref set.""" + ... - # Get next sequence number for this plan - seq = self._get_next_sequence(plan_id) + def retrieve(self, snapshot_id: str) -> tuple[ContextSnapshot, str]: + """Retrieve snapshot and its content.""" + ... - # Capture context snapshot - snapshot = self._capture_snapshot(hot_context, resources, checkpoint_id) + def retrieve_by_hash(self, hash: str) -> tuple[ContextSnapshot, str] | None: + """Retrieve by content hash (for deduplication).""" + ... + ``` - # Create decision - decision = Decision( - decision_id=decision_id, - plan_id=plan_id, - parent_decision_id=parent_decision_id, - sequence_number=seq, - decision_type=decision_type, - question=question, - chosen_option=chosen_option, - alternatives_considered=alternatives or [], - confidence_score=confidence, - rationale=rationale, - actor_reasoning=actor_reasoning, - context_snapshot=snapshot, - checkpoint_id=checkpoint_id - ) + - [ ] Commit: "feat(service): define ContextSnapshotStore protocol" - # Persist - self._decision_repo.create(decision) + - [ ] **D2.2** [Hamza] Implement `record_decision()` method: + - [ ] **D2.2a** [Hamza] Core implementation: - # Update parent's downstream if applicable - if parent_decision_id: - self._add_downstream_decision(parent_decision_id, decision_id) + ```python + def record_decision( + self, + plan_id: str, + decision_type: DecisionType, + question: str, + chosen_option: str, + hot_context: str, + resources: list[str], + parent_decision_id: str | None = None, + alternatives: list[str] | None = None, + confidence: float | None = None, + rationale: str = "", + actor_reasoning: str | None = None, + checkpoint_id: str | None = None + ) -> Decision: + """Record a new decision for a plan.""" + import ulid - logger.info(f"Recorded decision {decision_id}: {decision.summary}") - return decision - ``` - - [ ] Commit: "feat(service): implement record_decision()" - - [ ] **D2.2b** [Hamza] Add sequence number management: - ```python - def _get_next_sequence(self, plan_id: str) -> int: - """Get next sequence number for a plan.""" - if plan_id not in self._sequence_counters: - # Load max sequence from existing decisions - existing = self._decision_repo.get_max_sequence(plan_id) - self._sequence_counters[plan_id] = (existing or -1) + 1 + # Generate IDs + decision_id = ulid.new().str - seq = self._sequence_counters[plan_id] - self._sequence_counters[plan_id] += 1 - return seq - ``` - - [ ] Commit: "feat(service): add sequence number management" - - [ ] **D2.3** [Hamza] Implement tree query methods: - - [ ] **D2.3a** [Hamza] Implement `get_decision_tree()`: - ```python - def get_decision_tree(self, plan_id: str) -> list[Decision]: - """Get all decisions for a plan in tree order.""" - decisions = self._decision_repo.get_by_plan(plan_id) + # Get next sequence number for this plan + seq = self._get_next_sequence(plan_id) - # Sort by sequence number to get chronological order - return sorted(decisions, key=lambda d: d.sequence_number) + # Capture context snapshot + snapshot = self._capture_snapshot(hot_context, resources, checkpoint_id) - def get_decision_tree_nested(self, plan_id: str) -> DecisionTree: - """Get decisions as nested tree structure.""" - decisions = self.get_decision_tree(plan_id) - return self._build_tree(decisions) + # Create decision + decision = Decision( + decision_id=decision_id, + plan_id=plan_id, + parent_decision_id=parent_decision_id, + sequence_number=seq, + decision_type=decision_type, + question=question, + chosen_option=chosen_option, + alternatives_considered=alternatives or [], + confidence_score=confidence, + rationale=rationale, + actor_reasoning=actor_reasoning, + context_snapshot=snapshot, + checkpoint_id=checkpoint_id + ) - def _build_tree(self, decisions: list[Decision]) -> DecisionTree: - """Build tree from flat list of decisions.""" - by_id = {d.decision_id: d for d in decisions} - roots = [] + # Persist + self._decision_repo.create(decision) - for d in decisions: - if d.parent_decision_id is None: - roots.append(DecisionNode(decision=d, children=[])) - else: - # Find parent and add as child - # Implementation details... + # Update parent's downstream if applicable + if parent_decision_id: + self._add_downstream_decision(parent_decision_id, decision_id) - return DecisionTree(roots=roots, total_count=len(decisions)) - ``` - - [ ] Commit: "feat(service): implement decision tree queries" - - [ ] **D2.3b** [Hamza] Implement `get_decision()` and `get_children()`: - ```python - def get_decision(self, decision_id: str) -> Decision | None: - """Get a single decision by ID.""" - return self._decision_repo.get_by_id(decision_id) + logger.info(f"Recorded decision {decision_id}: {decision.summary}") + return decision + ``` - def get_children(self, decision_id: str) -> list[Decision]: - """Get all direct children of a decision.""" - return self._decision_repo.get_children(decision_id) + - [ ] Commit: "feat(service): implement record_decision()" - def get_ancestors(self, decision_id: str) -> list[Decision]: - """Get all ancestors from decision to root.""" - ancestors = [] - current = self.get_decision(decision_id) + - [ ] **D2.2b** [Hamza] Add sequence number management: - while current and current.parent_decision_id: - parent = self.get_decision(current.parent_decision_id) - if parent: - ancestors.append(parent) - current = parent + ```python + def _get_next_sequence(self, plan_id: str) -> int: + """Get next sequence number for a plan.""" + if plan_id not in self._sequence_counters: + # Load max sequence from existing decisions + existing = self._decision_repo.get_max_sequence(plan_id) + self._sequence_counters[plan_id] = (existing or -1) + 1 - return ancestors - ``` - - [ ] Commit: "feat(service): implement get_decision and get_children" - - [ ] **D2.4** [Hamza] Implement context snapshot capture: - - [ ] **D2.4a** [Hamza] Implement `_capture_snapshot()`: - ```python - def _capture_snapshot( - self, - hot_context: str, - resources: list[str], - checkpoint_id: str | None = None - ) -> ContextSnapshot: - """Capture and store a context snapshot.""" - # Create snapshot object - snapshot = ContextSnapshot.capture( - hot_context=hot_context, - resources=resources, - actor_state=checkpoint_id - ) + seq = self._sequence_counters[plan_id] + self._sequence_counters[plan_id] += 1 + return seq + ``` - # Check for duplicate by hash (deduplication) - existing = self._snapshot_store.retrieve_by_hash(snapshot.hot_context_hash) - if existing: - logger.debug(f"Reusing existing snapshot with hash {snapshot.hot_context_hash[:8]}") - return existing[0] + - [ ] Commit: "feat(service): add sequence number management" - # Store new snapshot - stored = self._snapshot_store.store(snapshot, hot_context) - return stored - ``` - - [ ] Commit: "feat(service): implement context snapshot capture" - - [ ] **D2.4b** [Hamza] Implement FileContextSnapshotStore: - ```python - class FileContextSnapshotStore: - """Store snapshots in filesystem.""" + - [ ] **D2.3** [Hamza] Implement tree query methods: + - [ ] **D2.3a** [Hamza] Implement `get_decision_tree()`: - def __init__(self, base_dir: Path): - self._base_dir = base_dir - self._base_dir.mkdir(parents=True, exist_ok=True) + ```python + def get_decision_tree(self, plan_id: str) -> list[Decision]: + """Get all decisions for a plan in tree order.""" + decisions = self._decision_repo.get_by_plan(plan_id) - def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: - """Store snapshot content to file.""" - file_path = self._base_dir / f"{snapshot.snapshot_id}.json" + # Sort by sequence number to get chronological order + return sorted(decisions, key=lambda d: d.sequence_number) - data = { - "snapshot": snapshot.__dict__, - "content": content - } - file_path.write_text(json.dumps(data)) + def get_decision_tree_nested(self, plan_id: str) -> DecisionTree: + """Get decisions as nested tree structure.""" + decisions = self.get_decision_tree(plan_id) + return self._build_tree(decisions) - # Update snapshot with ref - return dataclasses.replace( - snapshot, - hot_context_ref=str(file_path) - ) - ``` - - [ ] Commit: "feat(service): implement FileContextSnapshotStore" - - [ ] **D2.5** [Hamza] Integrate decision recording into strategy actor: - - [ ] **D2.5a** [Hamza] Create DecisionRecordingCallback: - ```python - class DecisionRecordingCallback: - """LangGraph callback to record decisions during execution.""" + def _build_tree(self, decisions: list[Decision]) -> DecisionTree: + """Build tree from flat list of decisions.""" + by_id = {d.decision_id: d for d in decisions} + roots = [] - def __init__(self, decision_service: DecisionService, plan_id: str): - self._service = decision_service - self._plan_id = plan_id - self._current_parent: str | None = None + for d in decisions: + if d.parent_decision_id is None: + roots.append(DecisionNode(decision=d, children=[])) + else: + # Find parent and add as child + # Implementation details... - def on_strategy_decision( - self, - question: str, - chosen: str, - alternatives: list[str], - confidence: float | None, - rationale: str, - context: str - ) -> Decision: - """Called when strategy actor makes a decision.""" - decision = self._service.record_decision( - plan_id=self._plan_id, - decision_type=DecisionType.STRATEGY_CHOICE, - question=question, - chosen_option=chosen, - hot_context=context, - resources=[], # Populated from plan - parent_decision_id=self._current_parent, - alternatives=alternatives, - confidence=confidence, - rationale=rationale - ) - return decision - ``` - - [ ] Commit: "feat(service): add DecisionRecordingCallback" - - [ ] **D2.5b** [Hamza] Record root PROMPT_DEFINITION decision: - ```python - def record_prompt_definition( - self, - plan_id: str, - prompt: str, - context: str - ) -> Decision: - """Record the initial prompt as root decision.""" - return self.record_decision( - plan_id=plan_id, - decision_type=DecisionType.PROMPT_DEFINITION, - question="What should be done?", - chosen_option=prompt, - hot_context=context, - resources=[], - parent_decision_id=None, - rationale="User provided prompt" - ) - ``` - - [ ] Commit: "feat(service): add record_prompt_definition()" - - [ ] **D2.6** [Hamza] Implement downstream relationship updates: - - [ ] **D2.6a** [Hamza] Add methods to update downstream fields: - ```python - def _add_downstream_decision(self, parent_id: str, child_id: str) -> None: - """Add child to parent's downstream_decision_ids.""" - parent = self._decision_repo.get_by_id(parent_id) - if parent: - updated = parent.with_downstream_decision(child_id) - self._decision_repo.update(updated) + return DecisionTree(roots=roots, total_count=len(decisions)) + ``` - def add_downstream_plan(self, decision_id: str, plan_id: str) -> None: - """Record that a decision spawned a subplan.""" - decision = self._decision_repo.get_by_id(decision_id) - if decision: - updated = decision.with_downstream_plan(plan_id) - self._decision_repo.update(updated) - logger.info(f"Linked subplan {plan_id} to decision {decision_id}") + - [ ] Commit: "feat(service): implement decision tree queries" - def add_artifact(self, decision_id: str, artifact_id: str) -> None: - """Record that a decision produced an artifact.""" - decision = self._decision_repo.get_by_id(decision_id) - if decision: - updated = decision.with_artifact(artifact_id) - self._decision_repo.update(updated) - ``` - - [ ] Commit: "feat(service): implement downstream relationship updates" - - [ ] **D2.6b** [Hamza] Implement `mark_superseded()`: - ```python - def mark_superseded(self, decision_id: str, by_decision_id: str) -> None: - """Mark a decision as superseded by another.""" - decision = self._decision_repo.get_by_id(decision_id) - if not decision: - raise DecisionNotFoundError(decision_id) + - [ ] **D2.3b** [Hamza] Implement `get_decision()` and `get_children()`: - if decision.superseded_by: - raise AlreadySupersededError( - f"Decision {decision_id} already superseded by {decision.superseded_by}" - ) + ```python + def get_decision(self, decision_id: str) -> Decision | None: + """Get a single decision by ID.""" + return self._decision_repo.get_by_id(decision_id) - updated = decision.mark_superseded(by_decision_id) - self._decision_repo.update(updated) - logger.info(f"Marked decision {decision_id} as superseded by {by_decision_id}") - ``` - - [ ] Commit: "feat(service): implement mark_superseded()" - - [ ] Tests: Verify decision tree is built during Strategize - - [ ] **D2.7** [Rui] Write Behave scenarios in `features/decision_recording.feature`: - - [ ] **D2.7a** [Rui] Basic recording scenarios: - - [ ] Scenario: Record first decision creates root - - [ ] Given a plan "plan-123" with no decisions - - [ ] When I call record_decision with decision_type=PROMPT_DEFINITION - - [ ] Then a Decision is created with sequence_number=0 - - [ ] And parent_decision_id is None - - [ ] And is_root returns True - - [ ] Scenario: Record subsequent decisions increment sequence - - [ ] Given a plan with 2 existing decisions - - [ ] When I record another decision - - [ ] Then sequence_number is 2 - - [ ] Commit: "test(behave): add basic decision recording scenarios" - - [ ] **D2.7b** [Rui] Tree structure scenarios: - - [ ] Scenario: Child decision links to parent - - [ ] Given root decision D1 exists - - [ ] When I record decision D2 with parent_decision_id=D1.id - - [ ] Then D1.downstream_decision_ids contains D2.id - - [ ] Scenario: Get decision tree returns correct order - - [ ] Given decisions D1, D2, D3 with sequences 0, 1, 2 - - [ ] When I call get_decision_tree(plan_id) - - [ ] Then decisions are returned in sequence order - - [ ] Scenario: Get children returns direct children only - - [ ] Given D1 -> D2 -> D3 (D2 child of D1, D3 child of D2) - - [ ] When I call get_children(D1.id) - - [ ] Then only D2 is returned (not D3) - - [ ] Commit: "test(behave): add decision tree structure scenarios" - - [ ] **D2.7c** [Rui] Context snapshot scenarios: - - [ ] Scenario: Context snapshot captured with decision - - [ ] Given hot context "file contents..." - - [ ] When I record a decision - - [ ] Then decision.context_snapshot is not None - - [ ] And context_snapshot.hot_context_hash is valid SHA-256 - - [ ] Scenario: Duplicate context reuses existing snapshot - - [ ] Given decision D1 with context hash "abc123" - - [ ] When I record D2 with identical context - - [ ] Then D2.context_snapshot.snapshot_id differs from D1 - - [ ] But content is only stored once (deduplication) - - [ ] Commit: "test(behave): add context snapshot scenarios" - - [ ] **D2.7d** [Rui] Downstream relationship scenarios: - - [ ] Scenario: Subplan spawn updates downstream_plan_ids - - [ ] Given decision D1 of type SUBPLAN_SPAWN - - [ ] When subplan SP1 is created from D1 - - [ ] And add_downstream_plan(D1.id, SP1.id) is called - - [ ] Then D1.downstream_plan_ids contains SP1.id - - [ ] Scenario: Artifact production updates artifacts_produced - - [ ] Given decision D1 produces artifact A1 - - [ ] When add_artifact(D1.id, A1.id) is called - - [ ] Then D1.artifacts_produced contains A1.id - - [ ] Commit: "test(behave): add downstream relationship scenarios" + def get_children(self, decision_id: str) -> list[Decision]: + """Get all direct children of a decision.""" + return self._decision_repo.get_children(decision_id) + + def get_ancestors(self, decision_id: str) -> list[Decision]: + """Get all ancestors from decision to root.""" + ancestors = [] + current = self.get_decision(decision_id) + + while current and current.parent_decision_id: + parent = self.get_decision(current.parent_decision_id) + if parent: + ancestors.append(parent) + current = parent + + return ancestors + ``` + + - [ ] Commit: "feat(service): implement get_decision and get_children" + + - [ ] **D2.4** [Hamza] Implement context snapshot capture: + - [ ] **D2.4a** [Hamza] Implement `_capture_snapshot()`: + + ```python + def _capture_snapshot( + self, + hot_context: str, + resources: list[str], + checkpoint_id: str | None = None + ) -> ContextSnapshot: + """Capture and store a context snapshot.""" + # Create snapshot object + snapshot = ContextSnapshot.capture( + hot_context=hot_context, + resources=resources, + actor_state=checkpoint_id + ) + + # Check for duplicate by hash (deduplication) + existing = self._snapshot_store.retrieve_by_hash(snapshot.hot_context_hash) + if existing: + logger.debug(f"Reusing existing snapshot with hash {snapshot.hot_context_hash[:8]}") + return existing[0] + + # Store new snapshot + stored = self._snapshot_store.store(snapshot, hot_context) + return stored + ``` + + - [ ] Commit: "feat(service): implement context snapshot capture" + + - [ ] **D2.4b** [Hamza] Implement FileContextSnapshotStore: + + ```python + class FileContextSnapshotStore: + """Store snapshots in filesystem.""" + + def __init__(self, base_dir: Path): + self._base_dir = base_dir + self._base_dir.mkdir(parents=True, exist_ok=True) + + def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: + """Store snapshot content to file.""" + file_path = self._base_dir / f"{snapshot.snapshot_id}.json" + + data = { + "snapshot": snapshot.__dict__, + "content": content + } + file_path.write_text(json.dumps(data)) + + # Update snapshot with ref + return dataclasses.replace( + snapshot, + hot_context_ref=str(file_path) + ) + ``` + + - [ ] Commit: "feat(service): implement FileContextSnapshotStore" + + - [ ] **D2.5** [Hamza] Integrate decision recording into strategy actor: + - [ ] **D2.5a** [Hamza] Create DecisionRecordingCallback: + + ```python + class DecisionRecordingCallback: + """LangGraph callback to record decisions during execution.""" + + def __init__(self, decision_service: DecisionService, plan_id: str): + self._service = decision_service + self._plan_id = plan_id + self._current_parent: str | None = None + + def on_strategy_decision( + self, + question: str, + chosen: str, + alternatives: list[str], + confidence: float | None, + rationale: str, + context: str + ) -> Decision: + """Called when strategy actor makes a decision.""" + decision = self._service.record_decision( + plan_id=self._plan_id, + decision_type=DecisionType.STRATEGY_CHOICE, + question=question, + chosen_option=chosen, + hot_context=context, + resources=[], # Populated from plan + parent_decision_id=self._current_parent, + alternatives=alternatives, + confidence=confidence, + rationale=rationale + ) + return decision + ``` + + - [ ] Commit: "feat(service): add DecisionRecordingCallback" + + - [ ] **D2.5b** [Hamza] Record root PROMPT_DEFINITION decision: + + ```python + def record_prompt_definition( + self, + plan_id: str, + prompt: str, + context: str + ) -> Decision: + """Record the initial prompt as root decision.""" + return self.record_decision( + plan_id=plan_id, + decision_type=DecisionType.PROMPT_DEFINITION, + question="What should be done?", + chosen_option=prompt, + hot_context=context, + resources=[], + parent_decision_id=None, + rationale="User provided prompt" + ) + ``` + + - [ ] Commit: "feat(service): add record_prompt_definition()" + + - [ ] **D2.6** [Hamza] Implement downstream relationship updates: + - [ ] **D2.6a** [Hamza] Add methods to update downstream fields: + + ```python + def _add_downstream_decision(self, parent_id: str, child_id: str) -> None: + """Add child to parent's downstream_decision_ids.""" + parent = self._decision_repo.get_by_id(parent_id) + if parent: + updated = parent.with_downstream_decision(child_id) + self._decision_repo.update(updated) + + def add_downstream_plan(self, decision_id: str, plan_id: str) -> None: + """Record that a decision spawned a subplan.""" + decision = self._decision_repo.get_by_id(decision_id) + if decision: + updated = decision.with_downstream_plan(plan_id) + self._decision_repo.update(updated) + logger.info(f"Linked subplan {plan_id} to decision {decision_id}") + + def add_artifact(self, decision_id: str, artifact_id: str) -> None: + """Record that a decision produced an artifact.""" + decision = self._decision_repo.get_by_id(decision_id) + if decision: + updated = decision.with_artifact(artifact_id) + self._decision_repo.update(updated) + ``` + + - [ ] Commit: "feat(service): implement downstream relationship updates" + + - [ ] **D2.6b** [Hamza] Implement `mark_superseded()`: + + ```python + def mark_superseded(self, decision_id: str, by_decision_id: str) -> None: + """Mark a decision as superseded by another.""" + decision = self._decision_repo.get_by_id(decision_id) + if not decision: + raise DecisionNotFoundError(decision_id) + + if decision.superseded_by: + raise AlreadySupersededError( + f"Decision {decision_id} already superseded by {decision.superseded_by}" + ) + + updated = decision.mark_superseded(by_decision_id) + self._decision_repo.update(updated) + logger.info(f"Marked decision {decision_id} as superseded by {by_decision_id}") + ``` + + - [ ] Commit: "feat(service): implement mark_superseded()" + + - [ ] Tests: Verify decision tree is built during Strategize + - [ ] **D2.7** [Rui] Write Behave scenarios in `features/decision_recording.feature`: + - [ ] **D2.7a** [Rui] Basic recording scenarios: + - [ ] Scenario: Record first decision creates root + - [ ] Given a plan "plan-123" with no decisions + - [ ] When I call record_decision with decision_type=PROMPT_DEFINITION + - [ ] Then a Decision is created with sequence_number=0 + - [ ] And parent_decision_id is None + - [ ] And is_root returns True + - [ ] Scenario: Record subsequent decisions increment sequence + - [ ] Given a plan with 2 existing decisions + - [ ] When I record another decision + - [ ] Then sequence_number is 2 + - [ ] Commit: "test(behave): add basic decision recording scenarios" + - [ ] **D2.7b** [Rui] Tree structure scenarios: + - [ ] Scenario: Child decision links to parent + - [ ] Given root decision D1 exists + - [ ] When I record decision D2 with parent_decision_id=D1.id + - [ ] Then D1.downstream_decision_ids contains D2.id + - [ ] Scenario: Get decision tree returns correct order + - [ ] Given decisions D1, D2, D3 with sequences 0, 1, 2 + - [ ] When I call get_decision_tree(plan_id) + - [ ] Then decisions are returned in sequence order + - [ ] Scenario: Get children returns direct children only + - [ ] Given D1 -> D2 -> D3 (D2 child of D1, D3 child of D2) + - [ ] When I call get_children(D1.id) + - [ ] Then only D2 is returned (not D3) + - [ ] Commit: "test(behave): add decision tree structure scenarios" + - [ ] **D2.7c** [Rui] Context snapshot scenarios: + - [ ] Scenario: Context snapshot captured with decision + - [ ] Given hot context "file contents..." + - [ ] When I record a decision + - [ ] Then decision.context_snapshot is not None + - [ ] And context_snapshot.hot_context_hash is valid SHA-256 + - [ ] Scenario: Duplicate context reuses existing snapshot + - [ ] Given decision D1 with context hash "abc123" + - [ ] When I record D2 with identical context + - [ ] Then D2.context_snapshot.snapshot_id differs from D1 + - [ ] But content is only stored once (deduplication) + - [ ] Commit: "test(behave): add context snapshot scenarios" + - [ ] **D2.7d** [Rui] Downstream relationship scenarios: + - [ ] Scenario: Subplan spawn updates downstream_plan_ids + - [ ] Given decision D1 of type SUBPLAN_SPAWN + - [ ] When subplan SP1 is created from D1 + - [ ] And add_downstream_plan(D1.id, SP1.id) is called + - [ ] Then D1.downstream_plan_ids contains SP1.id + - [ ] Scenario: Artifact production updates artifacts_produced + - [ ] Given decision D1 produces artifact A1 + - [ ] When add_artifact(D1.id, A1.id) is called + - [ ] Then D1.artifacts_produced contains A1.id + - [ ] Commit: "test(behave): add downstream relationship scenarios" - [ ] **Stage D3: Decision CLI & Viewing** (Day 16-17) **[Hamza]** - **SEQUENTIAL ORDER**: D3.1 (tree command) → D3.2 (explain command) → D3.3 (JSON output) → D3.4 (guidance-file) → D3.5 (Tests) + **SEQUENTIAL ORDER**: D3.1 (tree command) → D3.2 (explain command) → D3.3 (JSON output) → D3.4 (guidance-file) → D3.5 (Tests) + - [ ] Code: Decision viewing commands + - [ ] **D3.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan tree [plan_id]`: + - [ ] **D3.1a** [Hamza] Create command in `src/cleveragents/cli/commands/plan.py`: - - [ ] Code: Decision viewing commands - - [ ] **D3.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan tree [plan_id]`: - - [ ] **D3.1a** [Hamza] Create command in `src/cleveragents/cli/commands/plan.py`: - ```python - @plan.command("tree") - @click.argument("plan_id", required=False) - @click.option("--format", "output_format", - type=click.Choice(["tree", "json", "flat"]), default="tree") - @click.option("--show-superseded", is_flag=True, - help="Include superseded decisions") - def show_tree(plan_id: str | None, output_format: str, show_superseded: bool): - """Display the decision tree for a plan.""" - ``` - - [ ] Commit: "feat(cli): add plan tree command signature" - - [ ] **D3.1b** [Hamza] Implement plan resolution: - ```python - # If no plan_id, use current/most recent plan - if not plan_id: - plan = plan_service.get_current_plan() - if not plan: - console.print("[red]No active plan. Specify a plan ID.[/red]") - raise SystemExit(1) - plan_id = plan.plan_id + ```python + @plan.command("tree") + @click.argument("plan_id", required=False) + @click.option("--format", "output_format", + type=click.Choice(["tree", "json", "flat"]), default="tree") + @click.option("--show-superseded", is_flag=True, + help="Include superseded decisions") + def show_tree(plan_id: str | None, output_format: str, show_superseded: bool): + """Display the decision tree for a plan.""" + ``` - # Fetch decision tree - decisions = decision_service.get_decision_tree(plan_id) - if not decisions: - console.print(f"[yellow]No decisions recorded for plan {plan_id}[/yellow]") - return - ``` - - [ ] Commit: "feat(cli): implement plan resolution for tree command" - - [ ] **D3.1c** [Hamza] Implement tree rendering with Rich: - ```python - def _render_decision_tree(decisions: list[Decision], show_superseded: bool): - """Render decision tree using Rich Tree.""" - from rich.tree import Tree - from rich.text import Text + - [ ] Commit: "feat(cli): add plan tree command signature" - # Build tree structure - root_decisions = [d for d in decisions if d.is_root] - by_parent: dict[str, list[Decision]] = {} - for d in decisions: - if d.parent_decision_id: - by_parent.setdefault(d.parent_decision_id, []).append(d) + - [ ] **D3.1b** [Hamza] Implement plan resolution: - # Create Rich tree - tree = Tree("[bold]Decision Tree[/bold]") + ```python + # If no plan_id, use current/most recent plan + if not plan_id: + plan = plan_service.get_current_plan() + if not plan: + console.print("[red]No active plan. Specify a plan ID.[/red]") + raise SystemExit(1) + plan_id = plan.plan_id - def add_node(parent_tree, decision: Decision): - # Format decision display - type_color = _get_type_color(decision.decision_type) - label = Text() - label.append(f"[{decision.decision_type.value}] ", style=type_color) - label.append(f'"{decision.question[:40]}..."' if len(decision.question) > 40 else f'"{decision.question}"') - - if decision.confidence_score: - label.append(f" (conf: {decision.confidence_score:.2f})", style="dim") - - # Mark superseded - if decision.superseded_by: - if not show_superseded: + # Fetch decision tree + decisions = decision_service.get_decision_tree(plan_id) + if not decisions: + console.print(f"[yellow]No decisions recorded for plan {plan_id}[/yellow]") return - label.stylize("strike dim") - label.append(" [SUPERSEDED]", style="yellow") + ``` - # Mark corrections - if decision.is_correction: - label.append(" [CORRECTION]", style="green") + - [ ] Commit: "feat(cli): implement plan resolution for tree command" - # Add subplan links - for subplan_id in decision.downstream_plan_ids: - label.append(f" → {subplan_id[:8]}", style="cyan") + - [ ] **D3.1c** [Hamza] Implement tree rendering with Rich: - node = parent_tree.add(label) + ```python + def _render_decision_tree(decisions: list[Decision], show_superseded: bool): + """Render decision tree using Rich Tree.""" + from rich.tree import Tree + from rich.text import Text - # Add children recursively - for child in by_parent.get(decision.decision_id, []): - add_node(node, child) + # Build tree structure + root_decisions = [d for d in decisions if d.is_root] + by_parent: dict[str, list[Decision]] = {} + for d in decisions: + if d.parent_decision_id: + by_parent.setdefault(d.parent_decision_id, []).append(d) - for root in root_decisions: - add_node(tree, root) + # Create Rich tree + tree = Tree("[bold]Decision Tree[/bold]") - console.print(tree) - ``` - - [ ] Commit: "feat(cli): implement tree rendering with Rich" - - [ ] **D3.1d** [Hamza] Add type-specific coloring: - ```python - def _get_type_color(decision_type: DecisionType) -> str: - """Get color for decision type.""" - colors = { - DecisionType.PROMPT_DEFINITION: "bold white", - DecisionType.STRATEGY_CHOICE: "blue", - DecisionType.IMPLEMENTATION_CHOICE: "cyan", - DecisionType.RESOURCE_SELECTION: "magenta", - DecisionType.SUBPLAN_SPAWN: "green", - DecisionType.TOOL_INVOCATION: "yellow", - DecisionType.ERROR_RECOVERY: "red", - DecisionType.VALIDATION_RESPONSE: "orange3", - DecisionType.USER_INTERVENTION: "bold yellow", - } - return colors.get(decision_type, "white") - ``` - - [ ] Commit: "feat(cli): add decision type coloring" - - [ ] **D3.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan explain `: - - [ ] **D3.2a** [Hamza] Create command signature: - ```python - @plan.command("explain") - @click.argument("decision_id") - @click.option("--show-context", is_flag=True, help="Show full context snapshot") - @click.option("--show-reasoning", is_flag=True, help="Show raw LLM reasoning") - def explain_decision(decision_id: str, show_context: bool, show_reasoning: bool): - """Show detailed explanation of a specific decision.""" - ``` - - [ ] Commit: "feat(cli): add plan explain command signature" - - [ ] **D3.2b** [Hamza] Implement detailed display: - ```python - decision = decision_service.get_decision(decision_id) - if not decision: - console.print(f"[red]Decision {decision_id} not found[/red]") - raise SystemExit(1) + def add_node(parent_tree, decision: Decision): + # Format decision display + type_color = _get_type_color(decision.decision_type) + label = Text() + label.append(f"[{decision.decision_type.value}] ", style=type_color) + label.append(f'"{decision.question[:40]}..."' if len(decision.question) > 40 else f'"{decision.question}"') - # Create panels for display - from rich.panel import Panel - from rich.table import Table + if decision.confidence_score: + label.append(f" (conf: {decision.confidence_score:.2f})", style="dim") - # Header panel - header = Panel( - f"[bold]Decision: {decision.decision_id}[/bold]\n" - f"Type: {decision.decision_type.value}\n" - f"Plan: {decision.plan_id}", - title="Decision Details" - ) - console.print(header) + # Mark superseded + if decision.superseded_by: + if not show_superseded: + return + label.stylize("strike dim") + label.append(" [SUPERSEDED]", style="yellow") - # Question and answer - console.print(f"\n[bold]Question:[/bold] {decision.question}") - console.print(f"\n[bold green]Chosen:[/bold green] {decision.chosen_option}") + # Mark corrections + if decision.is_correction: + label.append(" [CORRECTION]", style="green") - # Alternatives - if decision.alternatives_considered: - console.print("\n[bold]Alternatives Considered:[/bold]") - for alt in decision.alternatives_considered: - console.print(f" • {alt}") + # Add subplan links + for subplan_id in decision.downstream_plan_ids: + label.append(f" → {subplan_id[:8]}", style="cyan") - # Confidence and rationale - if decision.confidence_score is not None: - bar = "█" * int(decision.confidence_score * 10) + "░" * (10 - int(decision.confidence_score * 10)) - console.print(f"\n[bold]Confidence:[/bold] {decision.confidence_score:.2f} [{bar}]") + node = parent_tree.add(label) - if decision.rationale: - console.print(f"\n[bold]Rationale:[/bold] {decision.rationale}") - ``` - - [ ] Commit: "feat(cli): implement explain decision display" - - [ ] **D3.2c** [Hamza] Show upstream/downstream relationships: - ```python - # Upstream (ancestors) - ancestors = decision_service.get_ancestors(decision_id) - if ancestors: - console.print("\n[bold]Decision Path (what led here):[/bold]") - for i, anc in enumerate(reversed(ancestors)): - indent = " " * i - console.print(f"{indent}↳ [{anc.decision_type.value}] {anc.question[:50]}") + # Add children recursively + for child in by_parent.get(decision.decision_id, []): + add_node(node, child) - # Downstream impact - children = decision_service.get_children(decision_id) - if children or decision.downstream_plan_ids: - console.print("\n[bold]Downstream Impact:[/bold]") - if children: - console.print(f" • {len(children)} child decisions") - if decision.downstream_plan_ids: - console.print(f" • {len(decision.downstream_plan_ids)} subplans spawned:") - for sp_id in decision.downstream_plan_ids: - console.print(f" - {sp_id}") - if decision.artifacts_produced: - console.print(f" • {len(decision.artifacts_produced)} artifacts produced") - ``` - - [ ] Commit: "feat(cli): add upstream/downstream display" - - [ ] **D3.2d** [Hamza] Add context and reasoning display: - ```python - # Context snapshot - if show_context: - console.print("\n[bold]Context Snapshot:[/bold]") - console.print(f" Hash: {decision.context_snapshot.hot_context_hash[:16]}...") - console.print(f" Resources: {', '.join(decision.context_snapshot.relevant_resources)}") + for root in root_decisions: + add_node(tree, root) - # Optionally show full content - try: - _, content = snapshot_store.retrieve(decision.context_snapshot.snapshot_id) - console.print(Panel(content[:1000] + "..." if len(content) > 1000 else content, - title="Context Content")) - except Exception as e: - console.print(f" [dim]Content not available: {e}[/dim]") + console.print(tree) + ``` - # Raw LLM reasoning - if show_reasoning and decision.actor_reasoning: - console.print(Panel(decision.actor_reasoning, title="LLM Reasoning")) - ``` - - [ ] Commit: "feat(cli): add context and reasoning display" - - [ ] **D3.3** [Hamza] Implement JSON output: - - [ ] **D3.3a** [Hamza] Add JSON format to tree command: - ```python - if output_format == "json": - # Build JSON structure - def decision_to_dict(d: Decision) -> dict: - return { - "decision_id": d.decision_id, - "type": d.decision_type.value, - "question": d.question, - "chosen_option": d.chosen_option, - "alternatives": d.alternatives_considered, - "confidence": d.confidence_score, - "rationale": d.rationale, - "parent_id": d.parent_decision_id, - "sequence": d.sequence_number, - "is_correction": d.is_correction, - "superseded_by": d.superseded_by, - "downstream_decisions": d.downstream_decision_ids, - "downstream_plans": d.downstream_plan_ids, - "created_at": d.created_at.isoformat() + - [ ] Commit: "feat(cli): implement tree rendering with Rich" + + - [ ] **D3.1d** [Hamza] Add type-specific coloring: + + ```python + def _get_type_color(decision_type: DecisionType) -> str: + """Get color for decision type.""" + colors = { + DecisionType.PROMPT_DEFINITION: "bold white", + DecisionType.STRATEGY_CHOICE: "blue", + DecisionType.IMPLEMENTATION_CHOICE: "cyan", + DecisionType.RESOURCE_SELECTION: "magenta", + DecisionType.SUBPLAN_SPAWN: "green", + DecisionType.TOOL_INVOCATION: "yellow", + DecisionType.ERROR_RECOVERY: "red", + DecisionType.VALIDATION_RESPONSE: "orange3", + DecisionType.USER_INTERVENTION: "bold yellow", } + return colors.get(decision_type, "white") + ``` - tree_json = { - "plan_id": plan_id, - "decision_count": len(decisions), - "decisions": [decision_to_dict(d) for d in decisions] - } + - [ ] Commit: "feat(cli): add decision type coloring" - print(json.dumps(tree_json, indent=2)) - ``` - - [ ] Commit: "feat(cli): add JSON output for plan tree" - - [ ] **D3.3b** [Hamza] Add flat format (for scripting): - ```python - if output_format == "flat": - # Tab-separated values for easy parsing - print("ID\tTYPE\tSEQ\tPARENT\tQUESTION") - for d in decisions: - print(f"{d.decision_id}\t{d.decision_type.value}\t{d.sequence_number}\t" - f"{d.parent_decision_id or '-'}\t{d.question[:50]}") - ``` - - [ ] Commit: "feat(cli): add flat output for plan tree" - - [ ] **D3.4** [Hamza] Implement `--guidance-file` option: - - [ ] **D3.4a** [Hamza] Add option to plan correct command: - ```python - @plan.command("correct") - @click.argument("decision_id") - @click.option("--mode", type=click.Choice(["revert", "append"]), required=True) - @click.option("--guidance", "-g", help="Correction guidance text") - @click.option("--guidance-file", "-f", type=click.File('r'), - help="Read guidance from file (use - for stdin)") - @click.option("--dry-run", is_flag=True, help="Show impact without executing") - def correct_decision(decision_id, mode, guidance, guidance_file, dry_run): - """Correct a decision and re-execute affected work.""" - ``` - - [ ] Commit: "feat(cli): add guidance-file option to correct command" - - [ ] **D3.4b** [Hamza] Handle guidance source priority: - ```python - # Get guidance from appropriate source - if guidance_file: - guidance_text = guidance_file.read() - elif guidance: - guidance_text = guidance - else: - console.print("[red]Either --guidance or --guidance-file is required[/red]") - raise SystemExit(1) + - [ ] **D3.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan explain `: + - [ ] **D3.2a** [Hamza] Create command signature: - if not guidance_text.strip(): - console.print("[red]Guidance cannot be empty[/red]") - raise SystemExit(1) - ``` - - [ ] Commit: "feat(cli): implement guidance source handling" - - [ ] Tests: Behave scenarios for decision CLI - - [ ] **D3.5** [Rui] Write Behave scenarios in `features/decision_cli.feature`: - - [ ] **D3.5a** [Rui] Tree command scenarios: - - [ ] Scenario: Display decision tree for plan - - [ ] Given plan with 5 decisions in tree structure - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id}` - - [ ] Then output shows tree with all decisions - - [ ] And decisions are color-coded by type - - [ ] Scenario: Tree command with JSON format - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --format=json` - - [ ] Then output is valid JSON - - [ ] And JSON contains all decision fields - - [ ] Scenario: Tree hides superseded by default - - [ ] Given decision D1 superseded by D1' - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree` - - [ ] Then D1 is not shown - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree --show-superseded` - - [ ] Then D1 is shown with strikethrough - - [ ] Commit: "test(behave): add tree command scenarios" - - [ ] **D3.5b** [Rui] Explain command scenarios: - - [ ] Scenario: Explain shows full decision details - - [ ] Given decision D1 with all fields populated - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` - - [ ] Then output shows question, chosen option, alternatives - - [ ] And output shows confidence and rationale - - [ ] Scenario: Explain shows upstream path - - [ ] Given decision D3 with ancestors D1 -> D2 -> D3 - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D3.id}` - - [ ] Then output shows "Decision Path" section - - [ ] And D1 and D2 are listed as ancestors - - [ ] Scenario: Explain shows downstream impact - - [ ] Given decision D1 with 2 child decisions and 1 subplan - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` - - [ ] Then output shows "Downstream Impact" section - - [ ] And shows "2 child decisions" and "1 subplan" - - [ ] Commit: "test(behave): add explain command scenarios" - - [ ] **D3.5c** [Rui] Guidance file scenarios: - - [ ] Scenario: Read guidance from file - - [ ] Given guidance file with text "Fix the authentication bug" - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file guidance.txt` - - [ ] Then correction uses the file content as guidance - - [ ] Scenario: Read guidance from stdin - - [ ] When I run `echo "Fix bug" | agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file=-` - - [ ] Then correction uses stdin content as guidance - - [ ] Commit: "test(behave): add guidance file scenarios" + ```python + @plan.command("explain") + @click.argument("decision_id") + @click.option("--show-context", is_flag=True, help="Show full context snapshot") + @click.option("--show-reasoning", is_flag=True, help="Show raw LLM reasoning") + def explain_decision(decision_id: str, show_context: bool, show_reasoning: bool): + """Show detailed explanation of a specific decision.""" + ``` + + - [ ] Commit: "feat(cli): add plan explain command signature" + + - [ ] **D3.2b** [Hamza] Implement detailed display: + + ```python + decision = decision_service.get_decision(decision_id) + if not decision: + console.print(f"[red]Decision {decision_id} not found[/red]") + raise SystemExit(1) + + # Create panels for display + from rich.panel import Panel + from rich.table import Table + + # Header panel + header = Panel( + f"[bold]Decision: {decision.decision_id}[/bold]\n" + f"Type: {decision.decision_type.value}\n" + f"Plan: {decision.plan_id}", + title="Decision Details" + ) + console.print(header) + + # Question and answer + console.print(f"\n[bold]Question:[/bold] {decision.question}") + console.print(f"\n[bold green]Chosen:[/bold green] {decision.chosen_option}") + + # Alternatives + if decision.alternatives_considered: + console.print("\n[bold]Alternatives Considered:[/bold]") + for alt in decision.alternatives_considered: + console.print(f" • {alt}") + + # Confidence and rationale + if decision.confidence_score is not None: + bar = "█" * int(decision.confidence_score * 10) + "░" * (10 - int(decision.confidence_score * 10)) + console.print(f"\n[bold]Confidence:[/bold] {decision.confidence_score:.2f} [{bar}]") + + if decision.rationale: + console.print(f"\n[bold]Rationale:[/bold] {decision.rationale}") + ``` + + - [ ] Commit: "feat(cli): implement explain decision display" + + - [ ] **D3.2c** [Hamza] Show upstream/downstream relationships: + + ```python + # Upstream (ancestors) + ancestors = decision_service.get_ancestors(decision_id) + if ancestors: + console.print("\n[bold]Decision Path (what led here):[/bold]") + for i, anc in enumerate(reversed(ancestors)): + indent = " " * i + console.print(f"{indent}↳ [{anc.decision_type.value}] {anc.question[:50]}") + + # Downstream impact + children = decision_service.get_children(decision_id) + if children or decision.downstream_plan_ids: + console.print("\n[bold]Downstream Impact:[/bold]") + if children: + console.print(f" • {len(children)} child decisions") + if decision.downstream_plan_ids: + console.print(f" • {len(decision.downstream_plan_ids)} subplans spawned:") + for sp_id in decision.downstream_plan_ids: + console.print(f" - {sp_id}") + if decision.artifacts_produced: + console.print(f" • {len(decision.artifacts_produced)} artifacts produced") + ``` + + - [ ] Commit: "feat(cli): add upstream/downstream display" + + - [ ] **D3.2d** [Hamza] Add context and reasoning display: + + ```python + # Context snapshot + if show_context: + console.print("\n[bold]Context Snapshot:[/bold]") + console.print(f" Hash: {decision.context_snapshot.hot_context_hash[:16]}...") + console.print(f" Resources: {', '.join(decision.context_snapshot.relevant_resources)}") + + # Optionally show full content + try: + _, content = snapshot_store.retrieve(decision.context_snapshot.snapshot_id) + console.print(Panel(content[:1000] + "..." if len(content) > 1000 else content, + title="Context Content")) + except Exception as e: + console.print(f" [dim]Content not available: {e}[/dim]") + + # Raw LLM reasoning + if show_reasoning and decision.actor_reasoning: + console.print(Panel(decision.actor_reasoning, title="LLM Reasoning")) + ``` + + - [ ] Commit: "feat(cli): add context and reasoning display" + + - [ ] **D3.3** [Hamza] Implement JSON output: + - [ ] **D3.3a** [Hamza] Add JSON format to tree command: + + ```python + if output_format == "json": + # Build JSON structure + def decision_to_dict(d: Decision) -> dict: + return { + "decision_id": d.decision_id, + "type": d.decision_type.value, + "question": d.question, + "chosen_option": d.chosen_option, + "alternatives": d.alternatives_considered, + "confidence": d.confidence_score, + "rationale": d.rationale, + "parent_id": d.parent_decision_id, + "sequence": d.sequence_number, + "is_correction": d.is_correction, + "superseded_by": d.superseded_by, + "downstream_decisions": d.downstream_decision_ids, + "downstream_plans": d.downstream_plan_ids, + "created_at": d.created_at.isoformat() + } + + tree_json = { + "plan_id": plan_id, + "decision_count": len(decisions), + "decisions": [decision_to_dict(d) for d in decisions] + } + + print(json.dumps(tree_json, indent=2)) + ``` + + - [ ] Commit: "feat(cli): add JSON output for plan tree" + + - [ ] **D3.3b** [Hamza] Add flat format (for scripting): + + ```python + if output_format == "flat": + # Tab-separated values for easy parsing + print("ID\tTYPE\tSEQ\tPARENT\tQUESTION") + for d in decisions: + print(f"{d.decision_id}\t{d.decision_type.value}\t{d.sequence_number}\t" + f"{d.parent_decision_id or '-'}\t{d.question[:50]}") + ``` + + - [ ] Commit: "feat(cli): add flat output for plan tree" + + - [ ] **D3.4** [Hamza] Implement `--guidance-file` option: + - [ ] **D3.4a** [Hamza] Add option to plan correct command: + + ```python + @plan.command("correct") + @click.argument("decision_id") + @click.option("--mode", type=click.Choice(["revert", "append"]), required=True) + @click.option("--guidance", "-g", help="Correction guidance text") + @click.option("--guidance-file", "-f", type=click.File('r'), + help="Read guidance from file (use - for stdin)") + @click.option("--dry-run", is_flag=True, help="Show impact without executing") + def correct_decision(decision_id, mode, guidance, guidance_file, dry_run): + """Correct a decision and re-execute affected work.""" + ``` + + - [ ] Commit: "feat(cli): add guidance-file option to correct command" + + - [ ] **D3.4b** [Hamza] Handle guidance source priority: + + ```python + # Get guidance from appropriate source + if guidance_file: + guidance_text = guidance_file.read() + elif guidance: + guidance_text = guidance + else: + console.print("[red]Either --guidance or --guidance-file is required[/red]") + raise SystemExit(1) + + if not guidance_text.strip(): + console.print("[red]Guidance cannot be empty[/red]") + raise SystemExit(1) + ``` + + - [ ] Commit: "feat(cli): implement guidance source handling" + + - [ ] Tests: Behave scenarios for decision CLI + - [ ] **D3.5** [Rui] Write Behave scenarios in `features/decision_cli.feature`: + - [ ] **D3.5a** [Rui] Tree command scenarios: + - [ ] Scenario: Display decision tree for plan + - [ ] Given plan with 5 decisions in tree structure + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id}` + - [ ] Then output shows tree with all decisions + - [ ] And decisions are color-coded by type + - [ ] Scenario: Tree command with JSON format + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --format=json` + - [ ] Then output is valid JSON + - [ ] And JSON contains all decision fields + - [ ] Scenario: Tree hides superseded by default + - [ ] Given decision D1 superseded by D1' + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree` + - [ ] Then D1 is not shown + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree --show-superseded` + - [ ] Then D1 is shown with strikethrough + - [ ] Commit: "test(behave): add tree command scenarios" + - [ ] **D3.5b** [Rui] Explain command scenarios: + - [ ] Scenario: Explain shows full decision details + - [ ] Given decision D1 with all fields populated + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` + - [ ] Then output shows question, chosen option, alternatives + - [ ] And output shows confidence and rationale + - [ ] Scenario: Explain shows upstream path + - [ ] Given decision D3 with ancestors D1 -> D2 -> D3 + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D3.id}` + - [ ] Then output shows "Decision Path" section + - [ ] And D1 and D2 are listed as ancestors + - [ ] Scenario: Explain shows downstream impact + - [ ] Given decision D1 with 2 child decisions and 1 subplan + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` + - [ ] Then output shows "Downstream Impact" section + - [ ] And shows "2 child decisions" and "1 subplan" + - [ ] Commit: "test(behave): add explain command scenarios" + - [ ] **D3.5c** [Rui] Guidance file scenarios: + - [ ] Scenario: Read guidance from file + - [ ] Given guidance file with text "Fix the authentication bug" + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file guidance.txt` + - [ ] Then correction uses the file content as guidance + - [ ] Scenario: Read guidance from stdin + - [ ] When I run `echo "Fix bug" | agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file=-` + - [ ] Then correction uses stdin content as guidance + - [ ] Commit: "test(behave): add guidance file scenarios" - [ ] **Stage D4: Decision Correction Mechanism** (Day 17-19) **[Jeff - CRITICAL FOR 30-DAY GOAL]** - **IMPORTANCE**: This is the core mechanism that enables large project autonomy. Without decision correction, any mistake requires restarting from scratch. With it, users can guide the system to correct specific decisions and only recompute affected work. + **IMPORTANCE**: This is the core mechanism that enables large project autonomy. Without decision correction, any mistake requires restarting from scratch. With it, users can guide the system to correct specific decisions and only recompute affected work. - **SEQUENTIAL ORDER**: D4.1 (Service) → D4.2 (Sandbox Checkpoints) → D4.3 (Re-execution) → D4.4-D4.5 (CLI) - - - [ ] Code: Implement decision correction (core to large project autonomy) - - [ ] **D4.1** [Jeff] Implement correction service in `src/cleveragents/application/services/correction_service.py`: - - [ ] **D4.1a** [Jeff] Create service scaffold and types: - - [ ] Import necessary domain models (Decision, Plan, DecisionType) - - [ ] Import repositories (DecisionRepository, LifecyclePlanRepository) - - [ ] Define `CorrectionResult` dataclass: - - [ ] `success: bool` - Whether correction succeeded - - [ ] `correction_attempt_id: str` - ULID of the correction attempt - - [ ] `new_decision_id: str | None` - ID of new decision (for revert mode) - - [ ] `subplan_id: str | None` - ID of fix subplan (for append mode) - - [ ] `affected_decisions: list[str]` - IDs of invalidated decisions - - [ ] `affected_plans: list[str]` - IDs of invalidated subplans - - [ ] `affected_artifacts: list[str]` - IDs of invalidated artifacts - - [ ] `error: str | None` - Error message if failed - - [ ] Define `ImpactAnalysis` dataclass: - - [ ] `decision_id: str` - Decision being analyzed - - [ ] `downstream_decisions: list[str]` - All affected decisions - - [ ] `downstream_plans: list[str]` - All affected subplans - - [ ] `downstream_artifacts: list[str]` - All affected artifacts - - [ ] `total_tokens_to_recompute: int | None` - Estimated cost - - [ ] Create `CorrectionService` class with DI for repositories - - [ ] Commit: "feat(correction): add CorrectionService scaffold and types" - - [ ] **D4.1b** [Jeff] Implement `correct_decision_revert(decision_id: str, guidance: str) -> CorrectionResult`: - - [ ] **Step 1: Validation** - - [ ] Fetch decision from repository - - [ ] Raise `DecisionNotFoundError` if not exists - - [ ] Fetch parent plan - - [ ] Raise `PlanNotCorrectableError` if plan.phase == APPLIED - - [ ] Raise `PlanNotCorrectableError` if decision.superseded_by is not None (already corrected) - - [ ] **Step 2: Impact Analysis** - - [ ] Call `identify_downstream_impact(decision_id)` to find all affected entities - - [ ] Log: "Correction will affect {n} decisions, {m} subplans, {k} artifacts" - - [ ] **Step 3: Create Correction Attempt Record** - - [ ] Generate new ULID for correction_attempt_id - - [ ] Create `correction_attempts` record with: - - [ ] `attempt_id = correction_attempt_id` - - [ ] `plan_id = decision.plan_id` - - [ ] `original_decision_id = decision_id` - - [ ] `status = 'pending'` - - [ ] `guidance = guidance` - - [ ] `created_at = now()` - - [ ] Persist to database - - [ ] **Step 4: Archive Old Subtree** - - [ ] For each affected decision: - - [ ] Create copy in `archived_decisions` table with original values - - [ ] Store reference to correction_attempt_id - - [ ] For each affected artifact: - - [ ] Move file to archive location - - [ ] Update artifact record with archive path - - [ ] Log: "Archived {n} decisions and {k} artifacts" - - [ ] **Step 5: Invalidate Old Decisions** - - [ ] For each affected decision starting from decision_id: - - [ ] Set `superseded_by = None` (will be filled when new decision created) - - [ ] Mark in execution_log that decision is invalidated - - [ ] For each affected subplan: - - [ ] Set state to CANCELLED - - [ ] Rollback subplan sandboxes - - [ ] **Step 6: Create Correction Decision** - - [ ] Create new Decision with: - - [ ] `decision_id = new ULID` - - [ ] `plan_id = original.plan_id` - - [ ] `parent_decision_id = original.parent_decision_id` (same parent) - - [ ] `sequence_number = original.sequence_number` (replaces in sequence) - - [ ] `decision_type = original.decision_type` - - [ ] `is_correction = True` - - [ ] `corrects_decision_id = decision_id` - - [ ] `question = original.question` - - [ ] `chosen_option = guidance` (user's correction) - - [ ] `rationale = f"User correction: {guidance}"` - - [ ] Update original decision: `superseded_by = new_decision_id` - - [ ] Persist new decision - - [ ] **Step 7: Rollback Sandbox** - - [ ] Get checkpoint_id from original decision's context_snapshot - - [ ] Call `sandbox_manager.rollback_to_checkpoint(plan_id, checkpoint_id)` - - [ ] This restores sandbox state to before the decision was made - - [ ] **Step 8: Re-execute from Decision Point** - - [ ] Build re-execution context: - - [ ] Include all decisions up to (but not including) the corrected one - - [ ] Include the new correction decision - - [ ] Include the guidance as additional context - - [ ] Call appropriate phase handler: - - [ ] If decision was in Strategize: resume strategy actor - - [ ] If decision was in Execute: resume execution actor - - [ ] Let actor generate new downstream decisions - - [ ] Continue normal phase flow - - [ ] **Step 9: Finalize** - - [ ] Update correction_attempt: `status = 'completed'`, `new_decision_id = new_decision.decision_id` - - [ ] Increment plan.attempt counter - - [ ] Return CorrectionResult with all IDs and counts - - [ ] **Error Handling** - - [ ] If any step fails, update correction_attempt: `status = 'failed'`, `error = message` - - [ ] Do NOT rollback the archive (preserve for debugging) - - [ ] Return CorrectionResult with success=False, error message - - [ ] Commit: "feat(correction): implement correct_decision_revert()" - - [ ] **D4.1c** [Jeff] Implement `correct_decision_append(decision_id: str, guidance: str) -> CorrectionResult`: - - [ ] **Step 1: Validation** (same as revert, but less strict) - - [ ] Fetch decision and plan - - [ ] Raise error if plan already Applied (can't modify) - - [ ] **Step 2: Create Fix Subplan** - - [ ] Create new action (or use built-in "fix" action) with: - - [ ] Description based on guidance - - [ ] Target resources from original decision's scope - - [ ] Use PlanLifecycleService.use_action() to create subplan - - [ ] Set subplan.parent_plan_id to original plan - - [ ] Set subplan's prompt to include: - - [ ] Original decision context - - [ ] What went wrong (from guidance) - - [ ] Instructions to fix - - [ ] **Step 3: Link to Decision** - - [ ] Add subplan_id to original decision's downstream_plan_ids - - [ ] Create new decision record of type USER_INTERVENTION - - [ ] Store guidance and fix plan reference - - [ ] **Step 4: Execute Fix Plan** - - [ ] If automation level allows, start subplan execution - - [ ] Otherwise, return subplan_id for manual execution - - [ ] Return CorrectionResult with subplan_id - - [ ] Commit: "feat(correction): implement correct_decision_append()" - - [ ] **D4.1d** [Jeff] Implement `identify_downstream_impact(decision_id: str) -> ImpactAnalysis`: - - [ ] **Recursive Decision Collection** - - [ ] Start with decision_id - - [ ] Query all decisions where parent_decision_id = current - - [ ] Recursively process each child - - [ ] Also check downstream_decision_ids relationship (DAG) - - [ ] Collect all IDs in depth-first order - - [ ] **Subplan Collection** - - [ ] For each decision, check if decision_type == SUBPLAN_SPAWN - - [ ] If so, add downstream_plan_ids to affected plans - - [ ] Recursively get that subplan's decisions too - - [ ] **Artifact Collection** - - [ ] For each affected decision, get artifacts_produced list - - [ ] Deduplicate (same artifact may be referenced multiple times) - - [ ] **Cost Estimation** (optional) - - [ ] Estimate tokens by summing context sizes of affected decisions - - [ ] This helps user decide if correction is worth it - - [ ] Return ImpactAnalysis with all collected data - - [ ] Commit: "feat(correction): implement identify_downstream_impact()" - - [ ] **D4.2** [Jeff] Implement sandbox checkpointing for correction: - - [ ] **D4.2a** [Jeff] Extend SandboxManager to track checkpoints: - - [ ] Add `create_checkpoint(plan_id: str, label: str) -> str`: - - [ ] For git sandboxes: commit current state with checkpoint tag - - [ ] For filesystem sandboxes: snapshot directory (or use git worktree trick) - - [ ] Return checkpoint_id - - [ ] Add `list_checkpoints(plan_id: str) -> list[Checkpoint]`: - - [ ] Return all checkpoints for the plan in chronological order - - [ ] Commit: "feat(sandbox): add checkpoint creation to SandboxManager" - - [ ] **D4.2b** [Jeff] Store checkpoint ID with each decision: - - [ ] Update Decision model: add `checkpoint_id: str | None` field - - [ ] When decision is created, auto-create checkpoint - - [ ] Store checkpoint_id in decision record - - [ ] Commit: "feat(decision): track checkpoint_id per decision" - - [ ] **D4.2c** [Jeff] Implement `rollback_to_checkpoint(plan_id: str, checkpoint_id: str) -> None`: - - [ ] Find all sandboxes for plan_id - - [ ] For each sandbox: - - [ ] If git: `git reset --hard {checkpoint_tag}` - - [ ] If filesystem: restore from snapshot - - [ ] Clear any state beyond checkpoint - - [ ] Update sandbox status - - [ ] Commit: "feat(sandbox): implement rollback_to_checkpoint()" - - [ ] **D4.2d** [Jeff] Handle checkpoint cleanup: - - [ ] After successful apply, old checkpoints can be pruned - - [ ] Keep at least N most recent checkpoints for debugging - - [ ] Implement `prune_checkpoints(plan_id: str, keep_count: int) -> int` - - [ ] Commit: "feat(sandbox): implement checkpoint pruning" - - [ ] **D4.3** [Jeff] Implement re-execution from correction point: - - [ ] **D4.3a** [Jeff] Build context for resumed execution: - - [ ] Create `CorrectionContext` dataclass: - - [ ] `original_decision: Decision` - What was being corrected - - [ ] `correction_decision: Decision` - The new corrected decision - - [ ] `guidance: str` - User's correction text - - [ ] `prior_decisions: list[Decision]` - Decisions that remain valid - - [ ] `invalidated_decisions: list[Decision]` - For reference/diff - - [ ] Commit: "feat(correction): define CorrectionContext" - - [ ] **D4.3b** [Jeff] Inject correction context into actor: - - [ ] Update strategy/execution actor invocation to accept CorrectionContext - - [ ] Actor prompt includes: - - [ ] "You previously decided: {original_decision.chosen_option}" - - [ ] "This decision is being corrected because: {guidance}" - - [ ] "Please reconsider and make a new decision based on this feedback." - - [ ] Actor should acknowledge correction and proceed - - [ ] Commit: "feat(correction): inject correction context into actor" - - [ ] **D4.3c** [Jeff] Resume phase execution from decision point: - - [ ] If correction is in Strategize phase: - - [ ] Resume strategy actor from decision point - - [ ] Actor generates new downstream decisions - - [ ] Continue until Strategize complete - - [ ] If correction is in Execute phase: - - [ ] Resume execution actor - - [ ] Regenerate affected subplans - - [ ] Continue execution - - [ ] Normal Apply phase follows - - [ ] Commit: "feat(correction): implement re-execution from decision point" - - [ ] **D4.3d** [Jeff] Handle correction failures: - - [ ] If actor fails during re-execution: - - [ ] Update correction_attempt status to 'failed' - - [ ] Preserve both old and new state for debugging - - [ ] Allow user to try different guidance - - [ ] Max correction attempts per decision: 3 (configurable) - - [ ] Commit: "feat(correction): handle re-execution failures" - - [ ] **D4.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=revert --guidance ""`: - - [ ] **D4.4a** [Hamza] Create correction command in plan CLI: - - [ ] Add `@plan.command("correct")` with Click - - [ ] Required argument: `decision_id: str` - - [ ] Required option: `--mode: str` (choices: revert, append) - - [ ] Required option: `--guidance: str` or `--guidance-file: Path` - - [ ] Optional: `--dry-run` to show impact without executing - - [ ] Commit: "feat(cli): add plan correct command scaffold" - - [ ] **D4.4b** [Hamza] Implement command logic for revert mode: - - [ ] Validate decision_id format (ULID) - - [ ] If --dry-run: - - [ ] Call CorrectionService.identify_downstream_impact() - - [ ] Display impact analysis - - [ ] Exit without making changes - - [ ] Show confirmation prompt: "This will affect {n} decisions. Continue? [y/N]" - - [ ] Support `--yes` to bypass confirmation - - [ ] If confirmed (or --yes), call CorrectionService.correct_decision_revert() - - [ ] Display progress with Rich console: - - [ ] "[1/5] Analyzing impact..." - - [ ] "[2/5] Archiving old decisions..." - - [ ] "[3/5] Rolling back sandbox..." - - [ ] "[4/5] Re-executing from decision point..." - - [ ] "[5/5] Finalizing correction..." - - [ ] On success, display: - - [ ] New decision ID - - [ ] Count of regenerated decisions - - [ ] Diff summary (files changed old vs new) - - [ ] On failure, display error with recovery suggestions - - [ ] Commit: "feat(cli): implement plan correct revert mode" - - [ ] **D4.4c** [Hamza] Handle guidance-file option: - - [ ] If --guidance-file specified: - - [ ] Read file contents as guidance - - [ ] Support `-` for stdin: `cat guidance.txt | agents [--data-dir PATH] [--config-path PATH] plan correct ... --guidance-file=-` - - [ ] Validate guidance is not empty - - [ ] Commit: "feat(cli): add guidance-file support to plan correct" - - [ ] **D4.5** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=append --guidance ""`: - - [ ] **D4.5a** [Hamza] Implement append mode command: - - [ ] No confirmation needed (additive, not destructive) - - [ ] Call CorrectionService.correct_decision_append() - - [ ] Display created subplan ID - - [ ] If automation allows, show execution progress - - [ ] Otherwise show: "Fix subplan created: {subplan_id}. Run `agents [--data-dir PATH] [--config-path PATH] plan execute {subplan_id}` to apply fix." - - [ ] Commit: "feat(cli): implement plan correct append mode" - - [ ] Tests: Correction mechanism tests (CRITICAL for 30-day goal) - - [ ] **D4.6** [Rui] Write Behave scenarios in `features/decision_correction.feature`: - - [ ] **D4.6a** [Rui] Revert mode basic scenarios: - - [ ] Scenario: Correct early decision re-executes downstream work - - [ ] Given a plan with 3 decisions (D1 → D2 → D3) in sequence - - [ ] And D1 chose "use PostgreSQL" with downstream D2 choosing "use psycopg2" - - [ ] When I correct D1 with guidance "use SQLite instead" - - [ ] Then D1 is marked superseded - - [ ] And a new D1' is created with chosen_option "use SQLite" - - [ ] And D2, D3 are invalidated and regenerated - - [ ] And new D2' reflects SQLite (e.g., "use sqlite3") - - [ ] And the correction_attempt record shows status='completed' - - [ ] Commit: "test(behave): add basic revert correction scenario" - - [ ] **D4.6b** [Rui] Subplan invalidation scenarios: - - [ ] Scenario: Correct decision with subplans invalidates subplans - - [ ] Given a plan where D2 is a SUBPLAN_SPAWN decision - - [ ] And subplan SP1 was created from D2 - - [ ] And SP1 has completed some work - - [ ] When I correct D2 with new guidance - - [ ] Then SP1 is marked as CANCELLED - - [ ] And SP1's sandbox is rolled back - - [ ] And a new subplan SP2 is created based on new guidance - - [ ] And SP2 uses the corrected context - - [ ] Commit: "test(behave): add subplan invalidation correction scenario" - - [ ] **D4.6c** [Rui] Append mode scenarios: - - [ ] Scenario: Append mode creates fix subplan without modifying history - - [ ] Given a plan with D1, D2, D3 all completed - - [ ] And the outcome has a bug due to D2's decision - - [ ] When I correct D2 with mode=append and guidance "add error handling" - - [ ] Then D2 is NOT marked as superseded - - [ ] And a new fix subplan is created - - [ ] And the fix subplan's prompt includes the guidance - - [ ] And the fix subplan targets the same resources as D2 - - [ ] Commit: "test(behave): add append mode correction scenario" - - [ ] **D4.6d** [Rui] Safety and error scenarios: - - [ ] Scenario: Cannot correct decision in Applied plan - - [ ] Given a plan that has been Applied successfully - - [ ] When I try to correct any decision - - [ ] Then I receive error "Cannot correct decisions in an applied plan" - - [ ] And no changes are made - - [ ] Scenario: Cannot correct already-corrected decision - - [ ] Given decision D1 that was already corrected to D1' - - [ ] When I try to correct D1 again - - [ ] Then I receive error "Decision already superseded" - - [ ] And hint "Correct the replacement decision D1' instead" - - [ ] Commit: "test(behave): add correction safety scenarios" - - [ ] **D4.6e** [Rui] History preservation scenarios: - - [ ] Scenario: Correction preserves history for comparison - - [ ] Given I correct decision D1 with new guidance - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --show-superseded` - - [ ] Then I see the original D1 decision details - - [ ] And I see the correction D1' decision details - - [ ] And I can compare the outcomes via `agents [--data-dir PATH] [--config-path PATH] plan diff --correction {plan_id}` - - [ ] Scenario: Archived artifacts are accessible - - [ ] Given correction archived some generated files - - [ ] Then I can retrieve archived files for diff comparison - - [ ] Commit: "test(behave): add history preservation scenarios" - - [ ] **D4.6f** [Rui] Dry-run and impact analysis scenarios: - - [ ] Scenario: Dry-run shows impact without making changes - - [ ] Given a plan with 5 decisions and 2 subplans - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct D2 --mode=revert --guidance "..." --dry-run` - - [ ] Then I see "This will affect:" - - [ ] And I see "- 3 decisions" - - [ ] And I see "- 1 subplan" - - [ ] And I see "- Estimated recomputation: ~5000 tokens" - - [ ] And no changes are made to the plan - - [ ] Commit: "test(behave): add dry-run and impact analysis scenarios" + **SEQUENTIAL ORDER**: D4.1 (Service) → D4.2 (Sandbox Checkpoints) → D4.3 (Re-execution) → D4.4-D4.5 (CLI) + - [ ] Code: Implement decision correction (core to large project autonomy) + - [ ] **D4.1** [Jeff] Implement correction service in `src/cleveragents/application/services/correction_service.py`: + - [ ] **D4.1a** [Jeff] Create service scaffold and types: + - [ ] Import necessary domain models (Decision, Plan, DecisionType) + - [ ] Import repositories (DecisionRepository, LifecyclePlanRepository) + - [ ] Define `CorrectionResult` dataclass: + - [ ] `success: bool` - Whether correction succeeded + - [ ] `correction_attempt_id: str` - ULID of the correction attempt + - [ ] `new_decision_id: str | None` - ID of new decision (for revert mode) + - [ ] `subplan_id: str | None` - ID of fix subplan (for append mode) + - [ ] `affected_decisions: list[str]` - IDs of invalidated decisions + - [ ] `affected_plans: list[str]` - IDs of invalidated subplans + - [ ] `affected_artifacts: list[str]` - IDs of invalidated artifacts + - [ ] `error: str | None` - Error message if failed + - [ ] Define `ImpactAnalysis` dataclass: + - [ ] `decision_id: str` - Decision being analyzed + - [ ] `downstream_decisions: list[str]` - All affected decisions + - [ ] `downstream_plans: list[str]` - All affected subplans + - [ ] `downstream_artifacts: list[str]` - All affected artifacts + - [ ] `total_tokens_to_recompute: int | None` - Estimated cost + - [ ] Create `CorrectionService` class with DI for repositories + - [ ] Commit: "feat(correction): add CorrectionService scaffold and types" + - [ ] **D4.1b** [Jeff] Implement `correct_decision_revert(decision_id: str, guidance: str) -> CorrectionResult`: + - [ ] **Step 1: Validation** + - [ ] Fetch decision from repository + - [ ] Raise `DecisionNotFoundError` if not exists + - [ ] Fetch parent plan + - [ ] Raise `PlanNotCorrectableError` if plan.phase == APPLIED + - [ ] Raise `PlanNotCorrectableError` if decision.superseded_by is not None (already corrected) + - [ ] **Step 2: Impact Analysis** + - [ ] Call `identify_downstream_impact(decision_id)` to find all affected entities + - [ ] Log: "Correction will affect {n} decisions, {m} subplans, {k} artifacts" + - [ ] **Step 3: Create Correction Attempt Record** + - [ ] Generate new ULID for correction_attempt_id + - [ ] Create `correction_attempts` record with: + - [ ] `attempt_id = correction_attempt_id` + - [ ] `plan_id = decision.plan_id` + - [ ] `original_decision_id = decision_id` + - [ ] `status = 'pending'` + - [ ] `guidance = guidance` + - [ ] `created_at = now()` + - [ ] Persist to database + - [ ] **Step 4: Archive Old Subtree** + - [ ] For each affected decision: + - [ ] Create copy in `archived_decisions` table with original values + - [ ] Store reference to correction_attempt_id + - [ ] For each affected artifact: + - [ ] Move file to archive location + - [ ] Update artifact record with archive path + - [ ] Log: "Archived {n} decisions and {k} artifacts" + - [ ] **Step 5: Invalidate Old Decisions** + - [ ] For each affected decision starting from decision_id: + - [ ] Set `superseded_by = None` (will be filled when new decision created) + - [ ] Mark in execution_log that decision is invalidated + - [ ] For each affected subplan: + - [ ] Set state to CANCELLED + - [ ] Rollback subplan sandboxes + - [ ] **Step 6: Create Correction Decision** + - [ ] Create new Decision with: + - [ ] `decision_id = new ULID` + - [ ] `plan_id = original.plan_id` + - [ ] `parent_decision_id = original.parent_decision_id` (same parent) + - [ ] `sequence_number = original.sequence_number` (replaces in sequence) + - [ ] `decision_type = original.decision_type` + - [ ] `is_correction = True` + - [ ] `corrects_decision_id = decision_id` + - [ ] `question = original.question` + - [ ] `chosen_option = guidance` (user's correction) + - [ ] `rationale = f"User correction: {guidance}"` + - [ ] Update original decision: `superseded_by = new_decision_id` + - [ ] Persist new decision + - [ ] **Step 7: Rollback Sandbox** + - [ ] Get checkpoint_id from original decision's context_snapshot + - [ ] Call `sandbox_manager.rollback_to_checkpoint(plan_id, checkpoint_id)` + - [ ] This restores sandbox state to before the decision was made + - [ ] **Step 8: Re-execute from Decision Point** + - [ ] Build re-execution context: + - [ ] Include all decisions up to (but not including) the corrected one + - [ ] Include the new correction decision + - [ ] Include the guidance as additional context + - [ ] Call appropriate phase handler: + - [ ] If decision was in Strategize: resume strategy actor + - [ ] If decision was in Execute: resume execution actor + - [ ] Let actor generate new downstream decisions + - [ ] Continue normal phase flow + - [ ] **Step 9: Finalize** + - [ ] Update correction_attempt: `status = 'completed'`, `new_decision_id = new_decision.decision_id` + - [ ] Increment plan.attempt counter + - [ ] Return CorrectionResult with all IDs and counts + - [ ] **Error Handling** + - [ ] If any step fails, update correction_attempt: `status = 'failed'`, `error = message` + - [ ] Do NOT rollback the archive (preserve for debugging) + - [ ] Return CorrectionResult with success=False, error message + - [ ] Commit: "feat(correction): implement correct_decision_revert()" + - [ ] **D4.1c** [Jeff] Implement `correct_decision_append(decision_id: str, guidance: str) -> CorrectionResult`: + - [ ] **Step 1: Validation** (same as revert, but less strict) + - [ ] Fetch decision and plan + - [ ] Raise error if plan already Applied (can't modify) + - [ ] **Step 2: Create Fix Subplan** + - [ ] Create new action (or use built-in "fix" action) with: + - [ ] Description based on guidance + - [ ] Target resources from original decision's scope + - [ ] Use PlanLifecycleService.use_action() to create subplan + - [ ] Set subplan.parent_plan_id to original plan + - [ ] Set subplan's prompt to include: + - [ ] Original decision context + - [ ] What went wrong (from guidance) + - [ ] Instructions to fix + - [ ] **Step 3: Link to Decision** + - [ ] Add subplan_id to original decision's downstream_plan_ids + - [ ] Create new decision record of type USER_INTERVENTION + - [ ] Store guidance and fix plan reference + - [ ] **Step 4: Execute Fix Plan** + - [ ] If automation level allows, start subplan execution + - [ ] Otherwise, return subplan_id for manual execution + - [ ] Return CorrectionResult with subplan_id + - [ ] Commit: "feat(correction): implement correct_decision_append()" + - [ ] **D4.1d** [Jeff] Implement `identify_downstream_impact(decision_id: str) -> ImpactAnalysis`: + - [ ] **Recursive Decision Collection** + - [ ] Start with decision_id + - [ ] Query all decisions where parent_decision_id = current + - [ ] Recursively process each child + - [ ] Also check downstream_decision_ids relationship (DAG) + - [ ] Collect all IDs in depth-first order + - [ ] **Subplan Collection** + - [ ] For each decision, check if decision_type == SUBPLAN_SPAWN + - [ ] If so, add downstream_plan_ids to affected plans + - [ ] Recursively get that subplan's decisions too + - [ ] **Artifact Collection** + - [ ] For each affected decision, get artifacts_produced list + - [ ] Deduplicate (same artifact may be referenced multiple times) + - [ ] **Cost Estimation** (optional) + - [ ] Estimate tokens by summing context sizes of affected decisions + - [ ] This helps user decide if correction is worth it + - [ ] Return ImpactAnalysis with all collected data + - [ ] Commit: "feat(correction): implement identify_downstream_impact()" + - [ ] **D4.2** [Jeff] Implement sandbox checkpointing for correction: + - [ ] **D4.2a** [Jeff] Extend SandboxManager to track checkpoints: + - [ ] Add `create_checkpoint(plan_id: str, label: str) -> str`: + - [ ] For git sandboxes: commit current state with checkpoint tag + - [ ] For filesystem sandboxes: snapshot directory (or use git worktree trick) + - [ ] Return checkpoint_id + - [ ] Add `list_checkpoints(plan_id: str) -> list[Checkpoint]`: + - [ ] Return all checkpoints for the plan in chronological order + - [ ] Commit: "feat(sandbox): add checkpoint creation to SandboxManager" + - [ ] **D4.2b** [Jeff] Store checkpoint ID with each decision: + - [ ] Update Decision model: add `checkpoint_id: str | None` field + - [ ] When decision is created, auto-create checkpoint + - [ ] Store checkpoint_id in decision record + - [ ] Commit: "feat(decision): track checkpoint_id per decision" + - [ ] **D4.2c** [Jeff] Implement `rollback_to_checkpoint(plan_id: str, checkpoint_id: str) -> None`: + - [ ] Find all sandboxes for plan_id + - [ ] For each sandbox: + - [ ] If git: `git reset --hard {checkpoint_tag}` + - [ ] If filesystem: restore from snapshot + - [ ] Clear any state beyond checkpoint + - [ ] Update sandbox status + - [ ] Commit: "feat(sandbox): implement rollback_to_checkpoint()" + - [ ] **D4.2d** [Jeff] Handle checkpoint cleanup: + - [ ] After successful apply, old checkpoints can be pruned + - [ ] Keep at least N most recent checkpoints for debugging + - [ ] Implement `prune_checkpoints(plan_id: str, keep_count: int) -> int` + - [ ] Commit: "feat(sandbox): implement checkpoint pruning" + - [ ] **D4.3** [Jeff] Implement re-execution from correction point: + - [ ] **D4.3a** [Jeff] Build context for resumed execution: + - [ ] Create `CorrectionContext` dataclass: + - [ ] `original_decision: Decision` - What was being corrected + - [ ] `correction_decision: Decision` - The new corrected decision + - [ ] `guidance: str` - User's correction text + - [ ] `prior_decisions: list[Decision]` - Decisions that remain valid + - [ ] `invalidated_decisions: list[Decision]` - For reference/diff + - [ ] Commit: "feat(correction): define CorrectionContext" + - [ ] **D4.3b** [Jeff] Inject correction context into actor: + - [ ] Update strategy/execution actor invocation to accept CorrectionContext + - [ ] Actor prompt includes: + - [ ] "You previously decided: {original_decision.chosen_option}" + - [ ] "This decision is being corrected because: {guidance}" + - [ ] "Please reconsider and make a new decision based on this feedback." + - [ ] Actor should acknowledge correction and proceed + - [ ] Commit: "feat(correction): inject correction context into actor" + - [ ] **D4.3c** [Jeff] Resume phase execution from decision point: + - [ ] If correction is in Strategize phase: + - [ ] Resume strategy actor from decision point + - [ ] Actor generates new downstream decisions + - [ ] Continue until Strategize complete + - [ ] If correction is in Execute phase: + - [ ] Resume execution actor + - [ ] Regenerate affected subplans + - [ ] Continue execution + - [ ] Normal Apply phase follows + - [ ] Commit: "feat(correction): implement re-execution from decision point" + - [ ] **D4.3d** [Jeff] Handle correction failures: + - [ ] If actor fails during re-execution: + - [ ] Update correction_attempt status to 'failed' + - [ ] Preserve both old and new state for debugging + - [ ] Allow user to try different guidance + - [ ] Max correction attempts per decision: 3 (configurable) + - [ ] Commit: "feat(correction): handle re-execution failures" + - [ ] **D4.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=revert --guidance ""`: + - [ ] **D4.4a** [Hamza] Create correction command in plan CLI: + - [ ] Add `@plan.command("correct")` with Click + - [ ] Required argument: `decision_id: str` + - [ ] Required option: `--mode: str` (choices: revert, append) + - [ ] Required option: `--guidance: str` or `--guidance-file: Path` + - [ ] Optional: `--dry-run` to show impact without executing + - [ ] Commit: "feat(cli): add plan correct command scaffold" + - [ ] **D4.4b** [Hamza] Implement command logic for revert mode: + - [ ] Validate decision_id format (ULID) + - [ ] If --dry-run: + - [ ] Call CorrectionService.identify_downstream_impact() + - [ ] Display impact analysis + - [ ] Exit without making changes + - [ ] Show confirmation prompt: "This will affect {n} decisions. Continue? [y/N]" + - [ ] Support `--yes` to bypass confirmation + - [ ] If confirmed (or --yes), call CorrectionService.correct_decision_revert() + - [ ] Display progress with Rich console: + - [ ] "[1/5] Analyzing impact..." + - [ ] "[2/5] Archiving old decisions..." + - [ ] "[3/5] Rolling back sandbox..." + - [ ] "[4/5] Re-executing from decision point..." + - [ ] "[5/5] Finalizing correction..." + - [ ] On success, display: + - [ ] New decision ID + - [ ] Count of regenerated decisions + - [ ] Diff summary (files changed old vs new) + - [ ] On failure, display error with recovery suggestions + - [ ] Commit: "feat(cli): implement plan correct revert mode" + - [ ] **D4.4c** [Hamza] Handle guidance-file option: + - [ ] If --guidance-file specified: + - [ ] Read file contents as guidance + - [ ] Support `-` for stdin: `cat guidance.txt | agents [--data-dir PATH] [--config-path PATH] plan correct ... --guidance-file=-` + - [ ] Validate guidance is not empty + - [ ] Commit: "feat(cli): add guidance-file support to plan correct" + - [ ] **D4.5** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=append --guidance ""`: + - [ ] **D4.5a** [Hamza] Implement append mode command: + - [ ] No confirmation needed (additive, not destructive) + - [ ] Call CorrectionService.correct_decision_append() + - [ ] Display created subplan ID + - [ ] If automation allows, show execution progress + - [ ] Otherwise show: "Fix subplan created: {subplan_id}. Run `agents [--data-dir PATH] [--config-path PATH] plan execute {subplan_id}` to apply fix." + - [ ] Commit: "feat(cli): implement plan correct append mode" + - [ ] Tests: Correction mechanism tests (CRITICAL for 30-day goal) + - [ ] **D4.6** [Rui] Write Behave scenarios in `features/decision_correction.feature`: + - [ ] **D4.6a** [Rui] Revert mode basic scenarios: + - [ ] Scenario: Correct early decision re-executes downstream work + - [ ] Given a plan with 3 decisions (D1 → D2 → D3) in sequence + - [ ] And D1 chose "use PostgreSQL" with downstream D2 choosing "use psycopg2" + - [ ] When I correct D1 with guidance "use SQLite instead" + - [ ] Then D1 is marked superseded + - [ ] And a new D1' is created with chosen_option "use SQLite" + - [ ] And D2, D3 are invalidated and regenerated + - [ ] And new D2' reflects SQLite (e.g., "use sqlite3") + - [ ] And the correction_attempt record shows status='completed' + - [ ] Commit: "test(behave): add basic revert correction scenario" + - [ ] **D4.6b** [Rui] Subplan invalidation scenarios: + - [ ] Scenario: Correct decision with subplans invalidates subplans + - [ ] Given a plan where D2 is a SUBPLAN_SPAWN decision + - [ ] And subplan SP1 was created from D2 + - [ ] And SP1 has completed some work + - [ ] When I correct D2 with new guidance + - [ ] Then SP1 is marked as CANCELLED + - [ ] And SP1's sandbox is rolled back + - [ ] And a new subplan SP2 is created based on new guidance + - [ ] And SP2 uses the corrected context + - [ ] Commit: "test(behave): add subplan invalidation correction scenario" + - [ ] **D4.6c** [Rui] Append mode scenarios: + - [ ] Scenario: Append mode creates fix subplan without modifying history + - [ ] Given a plan with D1, D2, D3 all completed + - [ ] And the outcome has a bug due to D2's decision + - [ ] When I correct D2 with mode=append and guidance "add error handling" + - [ ] Then D2 is NOT marked as superseded + - [ ] And a new fix subplan is created + - [ ] And the fix subplan's prompt includes the guidance + - [ ] And the fix subplan targets the same resources as D2 + - [ ] Commit: "test(behave): add append mode correction scenario" + - [ ] **D4.6d** [Rui] Safety and error scenarios: + - [ ] Scenario: Cannot correct decision in Applied plan + - [ ] Given a plan that has been Applied successfully + - [ ] When I try to correct any decision + - [ ] Then I receive error "Cannot correct decisions in an applied plan" + - [ ] And no changes are made + - [ ] Scenario: Cannot correct already-corrected decision + - [ ] Given decision D1 that was already corrected to D1' + - [ ] When I try to correct D1 again + - [ ] Then I receive error "Decision already superseded" + - [ ] And hint "Correct the replacement decision D1' instead" + - [ ] Commit: "test(behave): add correction safety scenarios" + - [ ] **D4.6e** [Rui] History preservation scenarios: + - [ ] Scenario: Correction preserves history for comparison + - [ ] Given I correct decision D1 with new guidance + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --show-superseded` + - [ ] Then I see the original D1 decision details + - [ ] And I see the correction D1' decision details + - [ ] And I can compare the outcomes via `agents [--data-dir PATH] [--config-path PATH] plan diff --correction {plan_id}` + - [ ] Scenario: Archived artifacts are accessible + - [ ] Given correction archived some generated files + - [ ] Then I can retrieve archived files for diff comparison + - [ ] Commit: "test(behave): add history preservation scenarios" + - [ ] **D4.6f** [Rui] Dry-run and impact analysis scenarios: + - [ ] Scenario: Dry-run shows impact without making changes + - [ ] Given a plan with 5 decisions and 2 subplans + - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct D2 --mode=revert --guidance "..." --dry-run` + - [ ] Then I see "This will affect:" + - [ ] And I see "- 3 decisions" + - [ ] And I see "- 1 subplan" + - [ ] And I see "- Estimated recomputation: ~5000 tokens" + - [ ] And no changes are made to the plan + - [ ] Commit: "test(behave): add dry-run and impact analysis scenarios" - [ ] **Stage D5: Decision Persistence** (Day 18-19) **[Hamza]** - **SEQUENTIAL ORDER**: D5.1 (decisions table) → D5.2 (dependencies table) → D5.3 (correction_attempts) → D5.4 (context_snapshots) → D5.5 (DecisionModel) → D5.6 (DecisionRepository) → D5.7 (Tests) + **SEQUENTIAL ORDER**: D5.1 (decisions table) → D5.2 (dependencies table) → D5.3 (correction_attempts) → D5.4 (context_snapshots) → D5.5 (DecisionModel) → D5.6 (DecisionRepository) → D5.7 (Tests) + - [ ] Code: Decision database schema + - [ ] **D5.1** [Hamza] Create Alembic migration for `decisions` table: + - [ ] **D5.1a** [Hamza] Generate migration file: + - [ ] Run `alembic revision --autogenerate -m "create_decisions_table"` + - [ ] Commit: "chore(db): generate decisions table migration" + - [ ] **D5.1b** [Hamza] Define schema: - - [ ] Code: Decision database schema - - [ ] **D5.1** [Hamza] Create Alembic migration for `decisions` table: - - [ ] **D5.1a** [Hamza] Generate migration file: - - [ ] Run `alembic revision --autogenerate -m "create_decisions_table"` - - [ ] Commit: "chore(db): generate decisions table migration" - - [ ] **D5.1b** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'decisions', - # Identity - sa.Column('decision_id', sa.Text(), nullable=False), - sa.Column('plan_id', sa.Text(), nullable=False), + ```python + def upgrade(): + op.create_table( + 'decisions', + # Identity + sa.Column('decision_id', sa.Text(), nullable=False), + sa.Column('plan_id', sa.Text(), nullable=False), - # Tree structure - sa.Column('parent_decision_id', sa.Text(), nullable=True), - sa.Column('sequence_number', sa.Integer(), nullable=False), + # Tree structure + sa.Column('parent_decision_id', sa.Text(), nullable=True), + sa.Column('sequence_number', sa.Integer(), nullable=False), - # Decision content - sa.Column('decision_type', sa.Text(), nullable=False), - sa.Column('question', sa.Text(), nullable=False), - sa.Column('chosen_option', sa.Text(), nullable=False), - sa.Column('alternatives_considered', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('confidence_score', sa.Float(), nullable=True), - sa.Column('rationale', sa.Text(), nullable=False, server_default=''), - sa.Column('actor_reasoning', sa.Text(), nullable=True), + # Decision content + sa.Column('decision_type', sa.Text(), nullable=False), + sa.Column('question', sa.Text(), nullable=False), + sa.Column('chosen_option', sa.Text(), nullable=False), + sa.Column('alternatives_considered', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('confidence_score', sa.Float(), nullable=True), + sa.Column('rationale', sa.Text(), nullable=False, server_default=''), + sa.Column('actor_reasoning', sa.Text(), nullable=True), - # Context - sa.Column('context_snapshot_id', sa.Text(), nullable=False), - sa.Column('checkpoint_id', sa.Text(), nullable=True), + # Context + sa.Column('context_snapshot_id', sa.Text(), nullable=False), + sa.Column('checkpoint_id', sa.Text(), nullable=True), - # Downstream relationships (denormalized for query performance) - sa.Column('downstream_decision_ids', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('downstream_plan_ids', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('artifacts_produced', sa.JSON(), nullable=False, server_default='[]'), + # Downstream relationships (denormalized for query performance) + sa.Column('downstream_decision_ids', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('downstream_plan_ids', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('artifacts_produced', sa.JSON(), nullable=False, server_default='[]'), - # Correction tracking - sa.Column('is_correction', sa.Boolean(), nullable=False, server_default='false'), - sa.Column('corrects_decision_id', sa.Text(), nullable=True), - sa.Column('superseded_by', sa.Text(), nullable=True), + # Correction tracking + sa.Column('is_correction', sa.Boolean(), nullable=False, server_default='false'), + sa.Column('corrects_decision_id', sa.Text(), nullable=True), + sa.Column('superseded_by', sa.Text(), nullable=True), - # Timestamp - sa.Column('created_at', sa.Text(), nullable=False), + # Timestamp + sa.Column('created_at', sa.Text(), nullable=False), - # Constraints - sa.PrimaryKeyConstraint('decision_id'), - sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['parent_decision_id'], ['decisions.decision_id'], ondelete='SET NULL'), - sa.ForeignKeyConstraint(['context_snapshot_id'], ['context_snapshots.snapshot_id']), - ) - ``` - - [ ] Commit: "feat(db): add decisions table schema" - - [ ] **D5.1c** [Hamza] Add indices for query optimization: - ```python - # Indices for common queries - op.create_index('ix_decisions_plan_id', 'decisions', ['plan_id']) - op.create_index('ix_decisions_parent_id', 'decisions', ['parent_decision_id']) - op.create_index('ix_decisions_plan_sequence', 'decisions', ['plan_id', 'sequence_number']) - op.create_index('ix_decisions_type', 'decisions', ['decision_type']) - op.create_index('ix_decisions_superseded', 'decisions', ['superseded_by'], - postgresql_where=sa.text('superseded_by IS NOT NULL')) - ``` - - [ ] Commit: "feat(db): add decisions table indices" - - [ ] **D5.1d** [Hamza] Add downgrade: - ```python - def downgrade(): - op.drop_index('ix_decisions_superseded') - op.drop_index('ix_decisions_type') - op.drop_index('ix_decisions_plan_sequence') - op.drop_index('ix_decisions_parent_id') - op.drop_index('ix_decisions_plan_id') - op.drop_table('decisions') - ``` - - [ ] Commit: "feat(db): add decisions table downgrade" - - [ ] **D5.2** [Hamza] Create Alembic migration for `decision_dependencies` table: - - [ ] **D5.2a** [Hamza] Define schema for DAG relationships: - ```python - def upgrade(): - op.create_table( - 'decision_dependencies', - sa.Column('id', sa.Integer(), autoincrement=True, nullable=False), - sa.Column('upstream_decision_id', sa.Text(), nullable=False), - sa.Column('downstream_decision_id', sa.Text(), nullable=False), - sa.Column('dependency_type', sa.Text(), nullable=False), # 'data', 'ordering', 'spawned' - sa.Column('created_at', sa.Text(), nullable=False), + # Constraints + sa.PrimaryKeyConstraint('decision_id'), + sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), + sa.ForeignKeyConstraint(['parent_decision_id'], ['decisions.decision_id'], ondelete='SET NULL'), + sa.ForeignKeyConstraint(['context_snapshot_id'], ['context_snapshots.snapshot_id']), + ) + ``` - sa.PrimaryKeyConstraint('id'), - sa.ForeignKeyConstraint(['upstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['downstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), - sa.UniqueConstraint('upstream_decision_id', 'downstream_decision_id', name='uq_decision_dependency') - ) - op.create_index('ix_dep_upstream', 'decision_dependencies', ['upstream_decision_id']) - op.create_index('ix_dep_downstream', 'decision_dependencies', ['downstream_decision_id']) - ``` - - [ ] Commit: "feat(db): add decision_dependencies table" - - [ ] **D5.2b** [Hamza] Add downgrade: - - [ ] Drop indices and table - - [ ] Commit: "feat(db): add decision_dependencies downgrade" - - [ ] **D5.3** [Hamza] Create Alembic migration for `correction_attempts` table: - - [ ] **D5.3a** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'correction_attempts', - sa.Column('attempt_id', sa.Text(), nullable=False), - sa.Column('plan_id', sa.Text(), nullable=False), - sa.Column('original_decision_id', sa.Text(), nullable=False), - sa.Column('new_decision_id', sa.Text(), nullable=True), # Set when complete - sa.Column('mode', sa.Text(), nullable=False), # 'revert' or 'append' - sa.Column('guidance', sa.Text(), nullable=False), - sa.Column('status', sa.Text(), nullable=False), # pending, completed, failed - sa.Column('error_message', sa.Text(), nullable=True), - sa.Column('affected_decisions', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('affected_plans', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('created_at', sa.Text(), nullable=False), - sa.Column('completed_at', sa.Text(), nullable=True), + - [ ] Commit: "feat(db): add decisions table schema" - sa.PrimaryKeyConstraint('attempt_id'), - sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['original_decision_id'], ['decisions.decision_id']), - sa.ForeignKeyConstraint(['new_decision_id'], ['decisions.decision_id']) - ) - op.create_index('ix_correction_plan', 'correction_attempts', ['plan_id']) - op.create_index('ix_correction_status', 'correction_attempts', ['status']) - ``` - - [ ] Commit: "feat(db): add correction_attempts table" - - [ ] **D5.3b** [Hamza] Add downgrade: - - [ ] Commit: "feat(db): add correction_attempts downgrade" - - [ ] **D5.4** [Hamza] Create Alembic migration for `context_snapshots` table: - - [ ] **D5.4a** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'context_snapshots', - sa.Column('snapshot_id', sa.Text(), nullable=False), - sa.Column('hot_context_hash', sa.Text(), nullable=False), - sa.Column('hot_context_ref', sa.Text(), nullable=False), # File path or blob ID - sa.Column('relevant_resources', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('actor_state_ref', sa.Text(), nullable=True), - sa.Column('file_versions', sa.JSON(), nullable=False, server_default='{}'), - sa.Column('content_size_bytes', sa.Integer(), nullable=False, server_default='0'), - sa.Column('created_at', sa.Text(), nullable=False), + - [ ] **D5.1c** [Hamza] Add indices for query optimization: - sa.PrimaryKeyConstraint('snapshot_id') - ) - # Index for content deduplication - op.create_index('ix_snapshot_hash', 'context_snapshots', ['hot_context_hash']) - ``` - - [ ] Commit: "feat(db): add context_snapshots table" - - [ ] **D5.4b** [Hamza] Add downgrade: - - [ ] Commit: "feat(db): add context_snapshots downgrade" - - [ ] **D5.5** [Hamza] Create `DecisionModel` in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **D5.5a** [Hamza] Define SQLAlchemy model: - ```python - class DecisionModel(Base): - __tablename__ = 'decisions' + ```python + # Indices for common queries + op.create_index('ix_decisions_plan_id', 'decisions', ['plan_id']) + op.create_index('ix_decisions_parent_id', 'decisions', ['parent_decision_id']) + op.create_index('ix_decisions_plan_sequence', 'decisions', ['plan_id', 'sequence_number']) + op.create_index('ix_decisions_type', 'decisions', ['decision_type']) + op.create_index('ix_decisions_superseded', 'decisions', ['superseded_by'], + postgresql_where=sa.text('superseded_by IS NOT NULL')) + ``` - decision_id = Column(Text, primary_key=True) - plan_id = Column(Text, ForeignKey('lifecycle_plans.plan_id', ondelete='CASCADE'), nullable=False) - parent_decision_id = Column(Text, ForeignKey('decisions.decision_id', ondelete='SET NULL'), nullable=True) - sequence_number = Column(Integer, nullable=False) + - [ ] Commit: "feat(db): add decisions table indices" - decision_type = Column(Text, nullable=False) - question = Column(Text, nullable=False) - chosen_option = Column(Text, nullable=False) - alternatives_considered = Column(JSON, nullable=False, default=list) - confidence_score = Column(Float, nullable=True) - rationale = Column(Text, nullable=False, default='') - actor_reasoning = Column(Text, nullable=True) + - [ ] **D5.1d** [Hamza] Add downgrade: - context_snapshot_id = Column(Text, ForeignKey('context_snapshots.snapshot_id'), nullable=False) - checkpoint_id = Column(Text, nullable=True) + ```python + def downgrade(): + op.drop_index('ix_decisions_superseded') + op.drop_index('ix_decisions_type') + op.drop_index('ix_decisions_plan_sequence') + op.drop_index('ix_decisions_parent_id') + op.drop_index('ix_decisions_plan_id') + op.drop_table('decisions') + ``` - downstream_decision_ids = Column(JSON, nullable=False, default=list) - downstream_plan_ids = Column(JSON, nullable=False, default=list) - artifacts_produced = Column(JSON, nullable=False, default=list) + - [ ] Commit: "feat(db): add decisions table downgrade" - is_correction = Column(Boolean, nullable=False, default=False) - corrects_decision_id = Column(Text, nullable=True) - superseded_by = Column(Text, nullable=True) + - [ ] **D5.2** [Hamza] Create Alembic migration for `decision_dependencies` table: + - [ ] **D5.2a** [Hamza] Define schema for DAG relationships: - created_at = Column(Text, nullable=False) + ```python + def upgrade(): + op.create_table( + 'decision_dependencies', + sa.Column('id', sa.Integer(), autoincrement=True, nullable=False), + sa.Column('upstream_decision_id', sa.Text(), nullable=False), + sa.Column('downstream_decision_id', sa.Text(), nullable=False), + sa.Column('dependency_type', sa.Text(), nullable=False), # 'data', 'ordering', 'spawned' + sa.Column('created_at', sa.Text(), nullable=False), - # Relationships - plan = relationship("LifecyclePlanModel", back_populates="decisions") - parent = relationship("DecisionModel", remote_side=[decision_id], backref="children") - context_snapshot = relationship("ContextSnapshotModel") - ``` - - [ ] Commit: "feat(db): add DecisionModel SQLAlchemy class" - - [ ] **D5.5b** [Hamza] Add domain conversion methods: - ```python - def to_domain(self) -> Decision: - """Convert to domain model.""" - return Decision( - decision_id=self.decision_id, - plan_id=self.plan_id, - parent_decision_id=self.parent_decision_id, - sequence_number=self.sequence_number, - decision_type=DecisionType(self.decision_type), - question=self.question, - chosen_option=self.chosen_option, - alternatives_considered=self.alternatives_considered or [], - confidence_score=self.confidence_score, - rationale=self.rationale, - actor_reasoning=self.actor_reasoning, - context_snapshot=self.context_snapshot.to_domain(), - checkpoint_id=self.checkpoint_id, - downstream_decision_ids=self.downstream_decision_ids or [], - downstream_plan_ids=self.downstream_plan_ids or [], - artifacts_produced=self.artifacts_produced or [], - is_correction=self.is_correction, - corrects_decision_id=self.corrects_decision_id, - superseded_by=self.superseded_by, - created_at=datetime.fromisoformat(self.created_at) - ) + sa.PrimaryKeyConstraint('id'), + sa.ForeignKeyConstraint(['upstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), + sa.ForeignKeyConstraint(['downstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), + sa.UniqueConstraint('upstream_decision_id', 'downstream_decision_id', name='uq_decision_dependency') + ) + op.create_index('ix_dep_upstream', 'decision_dependencies', ['upstream_decision_id']) + op.create_index('ix_dep_downstream', 'decision_dependencies', ['downstream_decision_id']) + ``` - @classmethod - def from_domain(cls, decision: Decision) -> "DecisionModel": - """Create from domain model.""" - return cls( - decision_id=decision.decision_id, - plan_id=decision.plan_id, - parent_decision_id=decision.parent_decision_id, - sequence_number=decision.sequence_number, - decision_type=decision.decision_type.value, - question=decision.question, - chosen_option=decision.chosen_option, - alternatives_considered=decision.alternatives_considered, - confidence_score=decision.confidence_score, - rationale=decision.rationale, - actor_reasoning=decision.actor_reasoning, - context_snapshot_id=decision.context_snapshot.snapshot_id, - checkpoint_id=decision.checkpoint_id, - downstream_decision_ids=decision.downstream_decision_ids, - downstream_plan_ids=decision.downstream_plan_ids, - artifacts_produced=decision.artifacts_produced, - is_correction=decision.is_correction, - corrects_decision_id=decision.corrects_decision_id, - superseded_by=decision.superseded_by, - created_at=decision.created_at.isoformat() - ) - ``` - - [ ] Commit: "feat(db): add DecisionModel conversion methods" - - [ ] **D5.6** [Hamza] Implement `DecisionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **D5.6a** [Hamza] Define repository class: - ```python - class DecisionRepository: - """Repository for Decision persistence.""" + - [ ] Commit: "feat(db): add decision_dependencies table" - def __init__(self, session_factory: Callable[[], Session]): - self._session_factory = session_factory - ``` - - [ ] Commit: "feat(repo): add DecisionRepository scaffold" - - [ ] **D5.6b** [Hamza] Implement `create()`: - ```python - def create(self, decision: Decision) -> Decision: - """Persist a new decision.""" - with self._session_factory() as session: - model = DecisionModel.from_domain(decision) - session.add(model) - try: - session.commit() - except IntegrityError as e: - session.rollback() - if "FOREIGN KEY" in str(e): - raise PlanNotFoundError(decision.plan_id) - raise - return decision - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.create()" - - [ ] **D5.6c** [Hamza] Implement `get_by_id()`: - ```python - def get_by_id(self, decision_id: str) -> Decision | None: - """Get decision by ID.""" - with self._session_factory() as session: - model = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(decision_id=decision_id).first() - return model.to_domain() if model else None - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_id()" - - [ ] **D5.6d** [Hamza] Implement `get_by_plan()`: - ```python - def get_by_plan(self, plan_id: str) -> list[Decision]: - """Get all decisions for a plan, ordered by sequence.""" - with self._session_factory() as session: - models = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(plan_id=plan_id).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_plan()" - - [ ] **D5.6e** [Hamza] Implement `get_children()`: - ```python - def get_children(self, decision_id: str) -> list[Decision]: - """Get direct children of a decision.""" - with self._session_factory() as session: - models = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(parent_decision_id=decision_id).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_children()" - - [ ] **D5.6f** [Hamza] Implement `get_tree()` with recursive CTE: - ```python - def get_tree(self, plan_id: str) -> list[Decision]: - """Get full decision tree for a plan using recursive CTE.""" - with self._session_factory() as session: - # Use recursive CTE for efficient tree retrieval - cte = session.query(DecisionModel).filter( - DecisionModel.plan_id == plan_id, - DecisionModel.parent_decision_id.is_(None) - ).cte(name='decision_tree', recursive=True) + - [ ] **D5.2b** [Hamza] Add downgrade: + - [ ] Drop indices and table + - [ ] Commit: "feat(db): add decision_dependencies downgrade" - cte_alias = aliased(DecisionModel, cte) - recursive = session.query(DecisionModel).join( - cte_alias, DecisionModel.parent_decision_id == cte_alias.decision_id - ) - cte = cte.union_all(recursive) + - [ ] **D5.3** [Hamza] Create Alembic migration for `correction_attempts` table: + - [ ] **D5.3a** [Hamza] Define schema: - models = session.query(DecisionModel).select_from(cte).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_tree()" - - [ ] **D5.6g** [Hamza] Implement `get_downstream()`: - ```python - def get_downstream(self, decision_id: str) -> list[Decision]: - """Get all downstream decisions (recursive).""" - with self._session_factory() as session: - # Get the starting decision - start = session.query(DecisionModel).filter_by( - decision_id=decision_id - ).first() - if not start: - return [] + ```python + def upgrade(): + op.create_table( + 'correction_attempts', + sa.Column('attempt_id', sa.Text(), nullable=False), + sa.Column('plan_id', sa.Text(), nullable=False), + sa.Column('original_decision_id', sa.Text(), nullable=False), + sa.Column('new_decision_id', sa.Text(), nullable=True), # Set when complete + sa.Column('mode', sa.Text(), nullable=False), # 'revert' or 'append' + sa.Column('guidance', sa.Text(), nullable=False), + sa.Column('status', sa.Text(), nullable=False), # pending, completed, failed + sa.Column('error_message', sa.Text(), nullable=True), + sa.Column('affected_decisions', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('affected_plans', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('created_at', sa.Text(), nullable=False), + sa.Column('completed_at', sa.Text(), nullable=True), - # Recursively collect all downstream - result = [] - to_process = list(start.downstream_decision_ids) - seen = set() + sa.PrimaryKeyConstraint('attempt_id'), + sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), + sa.ForeignKeyConstraint(['original_decision_id'], ['decisions.decision_id']), + sa.ForeignKeyConstraint(['new_decision_id'], ['decisions.decision_id']) + ) + op.create_index('ix_correction_plan', 'correction_attempts', ['plan_id']) + op.create_index('ix_correction_status', 'correction_attempts', ['status']) + ``` - while to_process: - did = to_process.pop(0) - if did in seen: - continue - seen.add(did) + - [ ] Commit: "feat(db): add correction_attempts table" - d = session.query(DecisionModel).filter_by(decision_id=did).first() - if d: - result.append(d.to_domain()) - to_process.extend(d.downstream_decision_ids) + - [ ] **D5.3b** [Hamza] Add downgrade: + - [ ] Commit: "feat(db): add correction_attempts downgrade" - return result - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_downstream()" - - [ ] **D5.6h** [Hamza] Implement `update()`: - ```python - def update(self, decision: Decision) -> Decision: - """Update an existing decision.""" - with self._session_factory() as session: - model = session.query(DecisionModel).filter_by( - decision_id=decision.decision_id - ).first() - if not model: - raise DecisionNotFoundError(decision.decision_id) + - [ ] **D5.4** [Hamza] Create Alembic migration for `context_snapshots` table: + - [ ] **D5.4a** [Hamza] Define schema: - # Update fields - model.downstream_decision_ids = decision.downstream_decision_ids - model.downstream_plan_ids = decision.downstream_plan_ids - model.artifacts_produced = decision.artifacts_produced - model.superseded_by = decision.superseded_by + ```python + def upgrade(): + op.create_table( + 'context_snapshots', + sa.Column('snapshot_id', sa.Text(), nullable=False), + sa.Column('hot_context_hash', sa.Text(), nullable=False), + sa.Column('hot_context_ref', sa.Text(), nullable=False), # File path or blob ID + sa.Column('relevant_resources', sa.JSON(), nullable=False, server_default='[]'), + sa.Column('actor_state_ref', sa.Text(), nullable=True), + sa.Column('file_versions', sa.JSON(), nullable=False, server_default='{}'), + sa.Column('content_size_bytes', sa.Integer(), nullable=False, server_default='0'), + sa.Column('created_at', sa.Text(), nullable=False), - session.commit() - return decision - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.update()" - - [ ] **D5.6i** [Hamza] Implement `get_max_sequence()`: - ```python - def get_max_sequence(self, plan_id: str) -> int | None: - """Get maximum sequence number for a plan.""" - with self._session_factory() as session: - result = session.query(func.max(DecisionModel.sequence_number)).filter_by( - plan_id=plan_id - ).scalar() - return result - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_max_sequence()" - - [ ] **D5.6j** [Hamza] Add retry decorator to all methods: - - [ ] Same pattern as other repositories - - [ ] Commit: "feat(repo): add retry decorator to DecisionRepository" - - [ ] Tests: Integration tests for decision persistence - - [ ] **D5.7** [Rui] Write Behave scenarios in `features/decision_persistence.feature`: - - [ ] **D5.7a** [Rui] Basic persistence scenarios: - - [ ] Scenario: Decision persists with all fields - - [ ] Given a valid Decision domain object - - [ ] When I call decision_repo.create(decision) - - [ ] Then decision is stored in database - - [ ] And get_by_id returns the decision - - [ ] And all fields match original - - [ ] Scenario: Decision FK to plan enforced - - [ ] Given no plan with ID "nonexistent" - - [ ] When I try to create decision with that plan_id - - [ ] Then PlanNotFoundError is raised - - [ ] Commit: "test(behave): add basic decision persistence scenarios" - - [ ] **D5.7b** [Rui] Tree query scenarios: - - [ ] Scenario: get_by_plan returns decisions in sequence order - - [ ] Given plan with decisions at sequences 0, 1, 2 - - [ ] When I call get_by_plan(plan_id) - - [ ] Then decisions are returned in sequence order - - [ ] Scenario: get_tree returns full hierarchy - - [ ] Given plan with 3-level decision tree - - [ ] When I call get_tree(plan_id) - - [ ] Then all decisions are returned - - [ ] And tree structure is preserved - - [ ] Scenario: get_children returns only direct children - - [ ] Given D1 -> D2 -> D3 hierarchy - - [ ] When I call get_children(D1.id) - - [ ] Then only D2 is returned - - [ ] Commit: "test(behave): add decision tree query scenarios" - - [ ] **D5.7c** [Rui] Context snapshot scenarios: - - [ ] Scenario: Context snapshot stored and retrievable - - [ ] Given decision with context_snapshot - - [ ] When decision is persisted - - [ ] Then context_snapshot_id is stored - - [ ] And snapshot can be retrieved by ID - - [ ] Scenario: Snapshot deduplication by hash - - [ ] Given two decisions with identical context content - - [ ] Then only one snapshot is stored - - [ ] And both decisions reference same snapshot - - [ ] Commit: "test(behave): add context snapshot persistence scenarios" - - [ ] **D5.7d** [Rui] Correction tracking scenarios: - - [ ] Scenario: Correction attempt persists - - [ ] Given correction attempt with all fields - - [ ] When persisted via CorrectionAttemptRepository - - [ ] Then can be retrieved by attempt_id - - [ ] And status can be updated - - [ ] Scenario: superseded_by updates correctly - - [ ] Given decision D1 - - [ ] When mark_superseded(D1.id, D2.id) called - - [ ] Then D1.superseded_by equals D2.id - - [ ] Commit: "test(behave): add correction persistence scenarios" + sa.PrimaryKeyConstraint('snapshot_id') + ) + # Index for content deduplication + op.create_index('ix_snapshot_hash', 'context_snapshots', ['hot_context_hash']) + ``` + + - [ ] Commit: "feat(db): add context_snapshots table" + + - [ ] **D5.4b** [Hamza] Add downgrade: + - [ ] Commit: "feat(db): add context_snapshots downgrade" + + - [ ] **D5.5** [Hamza] Create `DecisionModel` in `src/cleveragents/infrastructure/database/models.py`: + - [ ] **D5.5a** [Hamza] Define SQLAlchemy model: + + ```python + class DecisionModel(Base): + __tablename__ = 'decisions' + + decision_id = Column(Text, primary_key=True) + plan_id = Column(Text, ForeignKey('lifecycle_plans.plan_id', ondelete='CASCADE'), nullable=False) + parent_decision_id = Column(Text, ForeignKey('decisions.decision_id', ondelete='SET NULL'), nullable=True) + sequence_number = Column(Integer, nullable=False) + + decision_type = Column(Text, nullable=False) + question = Column(Text, nullable=False) + chosen_option = Column(Text, nullable=False) + alternatives_considered = Column(JSON, nullable=False, default=list) + confidence_score = Column(Float, nullable=True) + rationale = Column(Text, nullable=False, default='') + actor_reasoning = Column(Text, nullable=True) + + context_snapshot_id = Column(Text, ForeignKey('context_snapshots.snapshot_id'), nullable=False) + checkpoint_id = Column(Text, nullable=True) + + downstream_decision_ids = Column(JSON, nullable=False, default=list) + downstream_plan_ids = Column(JSON, nullable=False, default=list) + artifacts_produced = Column(JSON, nullable=False, default=list) + + is_correction = Column(Boolean, nullable=False, default=False) + corrects_decision_id = Column(Text, nullable=True) + superseded_by = Column(Text, nullable=True) + + created_at = Column(Text, nullable=False) + + # Relationships + plan = relationship("LifecyclePlanModel", back_populates="decisions") + parent = relationship("DecisionModel", remote_side=[decision_id], backref="children") + context_snapshot = relationship("ContextSnapshotModel") + ``` + + - [ ] Commit: "feat(db): add DecisionModel SQLAlchemy class" + + - [ ] **D5.5b** [Hamza] Add domain conversion methods: + + ```python + def to_domain(self) -> Decision: + """Convert to domain model.""" + return Decision( + decision_id=self.decision_id, + plan_id=self.plan_id, + parent_decision_id=self.parent_decision_id, + sequence_number=self.sequence_number, + decision_type=DecisionType(self.decision_type), + question=self.question, + chosen_option=self.chosen_option, + alternatives_considered=self.alternatives_considered or [], + confidence_score=self.confidence_score, + rationale=self.rationale, + actor_reasoning=self.actor_reasoning, + context_snapshot=self.context_snapshot.to_domain(), + checkpoint_id=self.checkpoint_id, + downstream_decision_ids=self.downstream_decision_ids or [], + downstream_plan_ids=self.downstream_plan_ids or [], + artifacts_produced=self.artifacts_produced or [], + is_correction=self.is_correction, + corrects_decision_id=self.corrects_decision_id, + superseded_by=self.superseded_by, + created_at=datetime.fromisoformat(self.created_at) + ) + + @classmethod + def from_domain(cls, decision: Decision) -> "DecisionModel": + """Create from domain model.""" + return cls( + decision_id=decision.decision_id, + plan_id=decision.plan_id, + parent_decision_id=decision.parent_decision_id, + sequence_number=decision.sequence_number, + decision_type=decision.decision_type.value, + question=decision.question, + chosen_option=decision.chosen_option, + alternatives_considered=decision.alternatives_considered, + confidence_score=decision.confidence_score, + rationale=decision.rationale, + actor_reasoning=decision.actor_reasoning, + context_snapshot_id=decision.context_snapshot.snapshot_id, + checkpoint_id=decision.checkpoint_id, + downstream_decision_ids=decision.downstream_decision_ids, + downstream_plan_ids=decision.downstream_plan_ids, + artifacts_produced=decision.artifacts_produced, + is_correction=decision.is_correction, + corrects_decision_id=decision.corrects_decision_id, + superseded_by=decision.superseded_by, + created_at=decision.created_at.isoformat() + ) + ``` + + - [ ] Commit: "feat(db): add DecisionModel conversion methods" + + - [ ] **D5.6** [Hamza] Implement `DecisionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: + - [ ] **D5.6a** [Hamza] Define repository class: + + ```python + class DecisionRepository: + """Repository for Decision persistence.""" + + def __init__(self, session_factory: Callable[[], Session]): + self._session_factory = session_factory + ``` + + - [ ] Commit: "feat(repo): add DecisionRepository scaffold" + + - [ ] **D5.6b** [Hamza] Implement `create()`: + + ```python + def create(self, decision: Decision) -> Decision: + """Persist a new decision.""" + with self._session_factory() as session: + model = DecisionModel.from_domain(decision) + session.add(model) + try: + session.commit() + except IntegrityError as e: + session.rollback() + if "FOREIGN KEY" in str(e): + raise PlanNotFoundError(decision.plan_id) + raise + return decision + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.create()" + + - [ ] **D5.6c** [Hamza] Implement `get_by_id()`: + + ```python + def get_by_id(self, decision_id: str) -> Decision | None: + """Get decision by ID.""" + with self._session_factory() as session: + model = session.query(DecisionModel).options( + joinedload(DecisionModel.context_snapshot) + ).filter_by(decision_id=decision_id).first() + return model.to_domain() if model else None + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_id()" + + - [ ] **D5.6d** [Hamza] Implement `get_by_plan()`: + + ```python + def get_by_plan(self, plan_id: str) -> list[Decision]: + """Get all decisions for a plan, ordered by sequence.""" + with self._session_factory() as session: + models = session.query(DecisionModel).options( + joinedload(DecisionModel.context_snapshot) + ).filter_by(plan_id=plan_id).order_by( + DecisionModel.sequence_number + ).all() + return [m.to_domain() for m in models] + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_plan()" + + - [ ] **D5.6e** [Hamza] Implement `get_children()`: + + ```python + def get_children(self, decision_id: str) -> list[Decision]: + """Get direct children of a decision.""" + with self._session_factory() as session: + models = session.query(DecisionModel).options( + joinedload(DecisionModel.context_snapshot) + ).filter_by(parent_decision_id=decision_id).order_by( + DecisionModel.sequence_number + ).all() + return [m.to_domain() for m in models] + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_children()" + + - [ ] **D5.6f** [Hamza] Implement `get_tree()` with recursive CTE: + + ```python + def get_tree(self, plan_id: str) -> list[Decision]: + """Get full decision tree for a plan using recursive CTE.""" + with self._session_factory() as session: + # Use recursive CTE for efficient tree retrieval + cte = session.query(DecisionModel).filter( + DecisionModel.plan_id == plan_id, + DecisionModel.parent_decision_id.is_(None) + ).cte(name='decision_tree', recursive=True) + + cte_alias = aliased(DecisionModel, cte) + recursive = session.query(DecisionModel).join( + cte_alias, DecisionModel.parent_decision_id == cte_alias.decision_id + ) + cte = cte.union_all(recursive) + + models = session.query(DecisionModel).select_from(cte).order_by( + DecisionModel.sequence_number + ).all() + return [m.to_domain() for m in models] + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_tree()" + + - [ ] **D5.6g** [Hamza] Implement `get_downstream()`: + + ```python + def get_downstream(self, decision_id: str) -> list[Decision]: + """Get all downstream decisions (recursive).""" + with self._session_factory() as session: + # Get the starting decision + start = session.query(DecisionModel).filter_by( + decision_id=decision_id + ).first() + if not start: + return [] + + # Recursively collect all downstream + result = [] + to_process = list(start.downstream_decision_ids) + seen = set() + + while to_process: + did = to_process.pop(0) + if did in seen: + continue + seen.add(did) + + d = session.query(DecisionModel).filter_by(decision_id=did).first() + if d: + result.append(d.to_domain()) + to_process.extend(d.downstream_decision_ids) + + return result + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_downstream()" + + - [ ] **D5.6h** [Hamza] Implement `update()`: + + ```python + def update(self, decision: Decision) -> Decision: + """Update an existing decision.""" + with self._session_factory() as session: + model = session.query(DecisionModel).filter_by( + decision_id=decision.decision_id + ).first() + if not model: + raise DecisionNotFoundError(decision.decision_id) + + # Update fields + model.downstream_decision_ids = decision.downstream_decision_ids + model.downstream_plan_ids = decision.downstream_plan_ids + model.artifacts_produced = decision.artifacts_produced + model.superseded_by = decision.superseded_by + + session.commit() + return decision + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.update()" + + - [ ] **D5.6i** [Hamza] Implement `get_max_sequence()`: + + ```python + def get_max_sequence(self, plan_id: str) -> int | None: + """Get maximum sequence number for a plan.""" + with self._session_factory() as session: + result = session.query(func.max(DecisionModel.sequence_number)).filter_by( + plan_id=plan_id + ).scalar() + return result + ``` + + - [ ] Commit: "feat(repo): implement DecisionRepository.get_max_sequence()" + + - [ ] **D5.6j** [Hamza] Add retry decorator to all methods: + - [ ] Same pattern as other repositories + - [ ] Commit: "feat(repo): add retry decorator to DecisionRepository" + + - [ ] Tests: Integration tests for decision persistence + - [ ] **D5.7** [Rui] Write Behave scenarios in `features/decision_persistence.feature`: + - [ ] **D5.7a** [Rui] Basic persistence scenarios: + - [ ] Scenario: Decision persists with all fields + - [ ] Given a valid Decision domain object + - [ ] When I call decision_repo.create(decision) + - [ ] Then decision is stored in database + - [ ] And get_by_id returns the decision + - [ ] And all fields match original + - [ ] Scenario: Decision FK to plan enforced + - [ ] Given no plan with ID "nonexistent" + - [ ] When I try to create decision with that plan_id + - [ ] Then PlanNotFoundError is raised + - [ ] Commit: "test(behave): add basic decision persistence scenarios" + - [ ] **D5.7b** [Rui] Tree query scenarios: + - [ ] Scenario: get_by_plan returns decisions in sequence order + - [ ] Given plan with decisions at sequences 0, 1, 2 + - [ ] When I call get_by_plan(plan_id) + - [ ] Then decisions are returned in sequence order + - [ ] Scenario: get_tree returns full hierarchy + - [ ] Given plan with 3-level decision tree + - [ ] When I call get_tree(plan_id) + - [ ] Then all decisions are returned + - [ ] And tree structure is preserved + - [ ] Scenario: get_children returns only direct children + - [ ] Given D1 -> D2 -> D3 hierarchy + - [ ] When I call get_children(D1.id) + - [ ] Then only D2 is returned + - [ ] Commit: "test(behave): add decision tree query scenarios" + - [ ] **D5.7c** [Rui] Context snapshot scenarios: + - [ ] Scenario: Context snapshot stored and retrievable + - [ ] Given decision with context_snapshot + - [ ] When decision is persisted + - [ ] Then context_snapshot_id is stored + - [ ] And snapshot can be retrieved by ID + - [ ] Scenario: Snapshot deduplication by hash + - [ ] Given two decisions with identical context content + - [ ] Then only one snapshot is stored + - [ ] And both decisions reference same snapshot + - [ ] Commit: "test(behave): add context snapshot persistence scenarios" + - [ ] **D5.7d** [Rui] Correction tracking scenarios: + - [ ] Scenario: Correction attempt persists + - [ ] Given correction attempt with all fields + - [ ] When persisted via CorrectionAttemptRepository + - [ ] Then can be retrieved by attempt_id + - [ ] And status can be updated + - [ ] Scenario: superseded_by updates correctly + - [ ] Given decision D1 + - [ ] When mark_superseded(D1.id, D2.id) called + - [ ] Then D1.superseded_by equals D2.id + - [ ] Commit: "test(behave): add correction persistence scenarios" **M4 SUCCESS CRITERIA** (Day 21): + - [ ] Decisions are recorded during Strategize phase with full context - [ ] Decision tree can be viewed via `agents [--data-dir PATH] [--config-path PATH] plan tree` command - [ ] `agents [--data-dir PATH] [--config-path PATH] plan explain ` shows full decision details @@ -6598,1400 +6964,1528 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation - [ ] **Stage E1: Subplan Model** (Day 12) **[Luis]** - **SEQUENTIAL ORDER**: E1.1 (Enums) → E1.2 (SubplanConfig) → E1.3 (Plan extension) → E1.4 (SubplanStatus) → E1.5 (Failure rules) → E1.6 (Tests) + **SEQUENTIAL ORDER**: E1.1 (Enums) → E1.2 (SubplanConfig) → E1.3 (Plan extension) → E1.4 (SubplanStatus) → E1.5 (Failure rules) → E1.6 (Tests) + - [ ] **E1.1** [Luis] Define execution enums in `src/cleveragents/domain/models/core/plan.py`: + - [ ] **E1.1a** [Luis] Define `ExecutionMode` enum: - - [ ] **E1.1** [Luis] Define execution enums in `src/cleveragents/domain/models/core/plan.py`: - - [ ] **E1.1a** [Luis] Define `ExecutionMode` enum: - ```python - class ExecutionMode(str, Enum): - """How subplans should be executed.""" - SEQUENTIAL = "sequential" # One after another, ordered by sequence - PARALLEL = "parallel" # All at once (up to max_parallel) - DEPENDENCY_ORDERED = "dependency_ordered" # Respect DAG dependencies - ``` - - [ ] Commit: "feat(domain): define ExecutionMode enum" - - [ ] **E1.1b** [Luis] Define `MergeStrategy` enum: - ```python - class MergeStrategy(str, Enum): - """How to merge results from parallel subplans.""" - GIT_THREE_WAY = "git_three_way" # Use git merge-file for code - SEQUENTIAL_APPLY = "sequential_apply" # Apply in completion order - FAIL_ON_CONFLICT = "fail_on_conflict" # Error if any conflicts - LAST_WINS = "last_wins" # Later changes overwrite earlier - ``` - - [ ] Commit: "feat(domain): define MergeStrategy enum" - - [ ] **E1.2** [Luis] Define `SubplanConfig` model: - - [ ] **E1.2a** [Luis] Create SubplanConfig dataclass: - ```python - class SubplanConfig(BaseModel): - """Configuration for subplan execution.""" + ```python + class ExecutionMode(str, Enum): + """How subplans should be executed.""" + SEQUENTIAL = "sequential" # One after another, ordered by sequence + PARALLEL = "parallel" # All at once (up to max_parallel) + DEPENDENCY_ORDERED = "dependency_ordered" # Respect DAG dependencies + ``` - execution_mode: ExecutionMode = Field( - default=ExecutionMode.SEQUENTIAL, - description="How to execute subplans" - ) - merge_strategy: MergeStrategy = Field( - default=MergeStrategy.GIT_THREE_WAY, - description="How to merge subplan results" - ) - max_parallel: int = Field( - default=5, ge=1, le=50, - description="Max concurrent subplans (for PARALLEL mode)" - ) - fail_fast: bool = Field( - default=False, - description="Stop all subplans on first failure" - ) - timeout_per_subplan_seconds: int | None = Field( - default=None, - description="Timeout for each subplan (None=no timeout)" - ) - retry_failed: bool = Field( - default=True, - description="Automatically retry failed subplans" - ) - max_retries: int = Field( - default=2, ge=0, le=5, - description="Max retry attempts per subplan" - ) - ``` - - [ ] Commit: "feat(domain): define SubplanConfig model" - - [ ] **E1.3** [Luis] Extend Plan model for subplan hierarchy: - - [ ] **E1.3a** [Luis] Add parent/root plan fields (verify exist): - ```python - # In Plan model - parent_plan_id: str | None = Field( - default=None, - description="Parent plan ID if this is a subplan" - ) - root_plan_id: str | None = Field( - default=None, - description="Root plan ID (topmost ancestor)" - ) - ``` - - [ ] Commit: "feat(domain): verify parent/root plan fields on Plan" - - [ ] **E1.3b** [Luis] Add subplan configuration field: - ```python - subplan_config: SubplanConfig | None = Field( - default=None, - description="Config for subplan execution (set on parent plans)" - ) - subplan_statuses: list["SubplanStatus"] = Field( - default_factory=list, - description="Status tracking for spawned subplans" - ) - ``` - - [ ] Commit: "feat(domain): add subplan config and status fields to Plan" - - [ ] **E1.3c** [Luis] Add computed properties: - ```python - @property - def is_subplan(self) -> bool: - """Check if this plan is a subplan (has parent).""" - return self.parent_plan_id is not None + - [ ] Commit: "feat(domain): define ExecutionMode enum" - @property - def is_root_plan(self) -> bool: - """Check if this is the root plan.""" - return self.root_plan_id is None or self.root_plan_id == self.plan_id + - [ ] **E1.1b** [Luis] Define `MergeStrategy` enum: - @property - def depth(self) -> int: - """Distance from root plan (0 for root).""" - # Note: This requires parent chain traversal - # For efficiency, may be cached or stored - if self.is_root_plan: - return 0 - # Computed by service layer traversing parent_plan_id chain - return -1 # Placeholder, computed externally + ```python + class MergeStrategy(str, Enum): + """How to merge results from parallel subplans.""" + GIT_THREE_WAY = "git_three_way" # Use git merge-file for code + SEQUENTIAL_APPLY = "sequential_apply" # Apply in completion order + FAIL_ON_CONFLICT = "fail_on_conflict" # Error if any conflicts + LAST_WINS = "last_wins" # Later changes overwrite earlier + ``` - @property - def has_subplans(self) -> bool: - """Check if this plan has spawned subplans.""" - return len(self.subplan_statuses) > 0 - ``` - - [ ] Commit: "feat(domain): add subplan computed properties to Plan" - - [ ] **E1.4** [Luis] Define `SubplanStatus` tracking model: - - [ ] **E1.4a** [Luis] Create SubplanStatus dataclass: - ```python - @dataclass - class SubplanStatus: - """Track status of a spawned subplan.""" + - [ ] Commit: "feat(domain): define MergeStrategy enum" - subplan_id: str # The subplan's plan_id - action_name: str # Action used to create subplan - target_resources: list[str] # Resources subplan works on + - [ ] **E1.2** [Luis] Define `SubplanConfig` model: + - [ ] **E1.2a** [Luis] Create SubplanConfig dataclass: - # Status tracking - status: ProcessingState = ProcessingState.QUEUED - started_at: datetime | None = None - completed_at: datetime | None = None + ```python + class SubplanConfig(BaseModel): + """Configuration for subplan execution.""" - # Results - error: str | None = None - changeset_summary: str | None = None # Brief summary of changes - files_changed: int = 0 + execution_mode: ExecutionMode = Field( + default=ExecutionMode.SEQUENTIAL, + description="How to execute subplans" + ) + merge_strategy: MergeStrategy = Field( + default=MergeStrategy.GIT_THREE_WAY, + description="How to merge subplan results" + ) + max_parallel: int = Field( + default=5, ge=1, le=50, + description="Max concurrent subplans (for PARALLEL mode)" + ) + fail_fast: bool = Field( + default=False, + description="Stop all subplans on first failure" + ) + timeout_per_subplan_seconds: int | None = Field( + default=None, + description="Timeout for each subplan (None=no timeout)" + ) + retry_failed: bool = Field( + default=True, + description="Automatically retry failed subplans" + ) + max_retries: int = Field( + default=2, ge=0, le=5, + description="Max retry attempts per subplan" + ) + ``` - # Retries - attempt_number: int = 1 - previous_attempts: list["SubplanAttempt"] = field(default_factory=list) - ``` - - [ ] Commit: "feat(domain): define SubplanStatus dataclass" - - [ ] **E1.4b** [Luis] Define SubplanAttempt for retry tracking: - ```python - @dataclass - class SubplanAttempt: - """Record of a subplan execution attempt.""" - attempt_number: int - started_at: datetime - completed_at: datetime | None - error: str | None - was_retried: bool - ``` - - [ ] Commit: "feat(domain): define SubplanAttempt dataclass" - - [ ] **E1.5** [Luis] Define subplan failure handling rules: - - [ ] **E1.5a** [Luis] Create `SubplanFailureHandler` class: - ```python - class SubplanFailureHandler: - """Handle subplan failures based on configuration.""" + - [ ] Commit: "feat(domain): define SubplanConfig model" - def should_stop_others( - self, - config: SubplanConfig, - failed_status: SubplanStatus - ) -> bool: - """Determine if other subplans should stop.""" - if config.fail_fast: - return True - if config.execution_mode == ExecutionMode.SEQUENTIAL: - return True # Sequential always stops on failure - return False # Parallel continues others + - [ ] **E1.3** [Luis] Extend Plan model for subplan hierarchy: + - [ ] **E1.3a** [Luis] Add parent/root plan fields (verify exist): - def should_retry( - self, - config: SubplanConfig, - status: SubplanStatus - ) -> bool: - """Determine if failed subplan should be retried.""" - if not config.retry_failed: - return False - if status.attempt_number > config.max_retries: - return False - # Don't retry on certain errors - if status.error and "ValidationError" in status.error: - return True # Validation failures can be retried - if status.error and "TimeoutError" in status.error: - return True # Timeouts can be retried - return False - ``` - - [ ] Commit: "feat(domain): define SubplanFailureHandler" - - [ ] **E1.5b** [Luis] Add failure state constants: - ```python - # Error = application/system bug, likely not recoverable - # Failure = task couldn't complete (tests fail, validation fail), may be retryable + ```python + # In Plan model + parent_plan_id: str | None = Field( + default=None, + description="Parent plan ID if this is a subplan" + ) + root_plan_id: str | None = Field( + default=None, + description="Root plan ID (topmost ancestor)" + ) + ``` - RETRIABLE_FAILURES = { - "ValidationError", - "TimeoutError", - "TemporaryResourceError", - "MergeConflictError" # May succeed with different merge strategy - } + - [ ] Commit: "feat(domain): verify parent/root plan fields on Plan" - NON_RETRIABLE_ERRORS = { - "ConfigurationError", - "AuthenticationError", - "MissingResourceError", - "CircularDependencyError" - } - ``` - - [ ] Commit: "feat(domain): define retriable vs non-retriable failures" - - [ ] **E1.6** [Rui] Write Behave tests for subplan model: - - [ ] **E1.6a** [Rui] Plan hierarchy scenarios: - - [ ] Scenario: Plan with parent_plan_id has is_subplan=True - - [ ] Given Plan with parent_plan_id set - - [ ] Then is_subplan returns True - - [ ] And is_root_plan returns False - - [ ] Scenario: Root plan has is_subplan=False and is_root_plan=True - - [ ] Scenario: SubplanConfig validates max_parallel bounds - - [ ] Commit: "test(behave): add plan hierarchy scenarios" - - [ ] **E1.6b** [Rui] Execution mode scenarios: - - [ ] Scenario: ExecutionMode enum has all required values - - [ ] Scenario: MergeStrategy enum has all required values - - [ ] Scenario: SubplanConfig defaults are applied - - [ ] Commit: "test(behave): add execution mode scenarios" - - [ ] **E1.6c** [Rui] SubplanStatus scenarios: - - [ ] Scenario: SubplanStatus tracks state correctly - - [ ] Scenario: SubplanAttempt records retry history - - [ ] Scenario: Failure handler respects fail_fast setting - - [ ] Commit: "test(behave): add SubplanStatus scenarios" + - [ ] **E1.3b** [Luis] Add subplan configuration field: + + ```python + subplan_config: SubplanConfig | None = Field( + default=None, + description="Config for subplan execution (set on parent plans)" + ) + subplan_statuses: list["SubplanStatus"] = Field( + default_factory=list, + description="Status tracking for spawned subplans" + ) + ``` + + - [ ] Commit: "feat(domain): add subplan config and status fields to Plan" + + - [ ] **E1.3c** [Luis] Add computed properties: + + ```python + @property + def is_subplan(self) -> bool: + """Check if this plan is a subplan (has parent).""" + return self.parent_plan_id is not None + + @property + def is_root_plan(self) -> bool: + """Check if this is the root plan.""" + return self.root_plan_id is None or self.root_plan_id == self.plan_id + + @property + def depth(self) -> int: + """Distance from root plan (0 for root).""" + # Note: This requires parent chain traversal + # For efficiency, may be cached or stored + if self.is_root_plan: + return 0 + # Computed by service layer traversing parent_plan_id chain + return -1 # Placeholder, computed externally + + @property + def has_subplans(self) -> bool: + """Check if this plan has spawned subplans.""" + return len(self.subplan_statuses) > 0 + ``` + + - [ ] Commit: "feat(domain): add subplan computed properties to Plan" + + - [ ] **E1.4** [Luis] Define `SubplanStatus` tracking model: + - [ ] **E1.4a** [Luis] Create SubplanStatus dataclass: + + ```python + @dataclass + class SubplanStatus: + """Track status of a spawned subplan.""" + + subplan_id: str # The subplan's plan_id + action_name: str # Action used to create subplan + target_resources: list[str] # Resources subplan works on + + # Status tracking + status: ProcessingState = ProcessingState.QUEUED + started_at: datetime | None = None + completed_at: datetime | None = None + + # Results + error: str | None = None + changeset_summary: str | None = None # Brief summary of changes + files_changed: int = 0 + + # Retries + attempt_number: int = 1 + previous_attempts: list["SubplanAttempt"] = field(default_factory=list) + ``` + + - [ ] Commit: "feat(domain): define SubplanStatus dataclass" + + - [ ] **E1.4b** [Luis] Define SubplanAttempt for retry tracking: + + ```python + @dataclass + class SubplanAttempt: + """Record of a subplan execution attempt.""" + attempt_number: int + started_at: datetime + completed_at: datetime | None + error: str | None + was_retried: bool + ``` + + - [ ] Commit: "feat(domain): define SubplanAttempt dataclass" + + - [ ] **E1.5** [Luis] Define subplan failure handling rules: + - [ ] **E1.5a** [Luis] Create `SubplanFailureHandler` class: + + ```python + class SubplanFailureHandler: + """Handle subplan failures based on configuration.""" + + def should_stop_others( + self, + config: SubplanConfig, + failed_status: SubplanStatus + ) -> bool: + """Determine if other subplans should stop.""" + if config.fail_fast: + return True + if config.execution_mode == ExecutionMode.SEQUENTIAL: + return True # Sequential always stops on failure + return False # Parallel continues others + + def should_retry( + self, + config: SubplanConfig, + status: SubplanStatus + ) -> bool: + """Determine if failed subplan should be retried.""" + if not config.retry_failed: + return False + if status.attempt_number > config.max_retries: + return False + # Don't retry on certain errors + if status.error and "ValidationError" in status.error: + return True # Validation failures can be retried + if status.error and "TimeoutError" in status.error: + return True # Timeouts can be retried + return False + ``` + + - [ ] Commit: "feat(domain): define SubplanFailureHandler" + + - [ ] **E1.5b** [Luis] Add failure state constants: + + ```python + # Error = application/system bug, likely not recoverable + # Failure = task couldn't complete (tests fail, validation fail), may be retryable + + RETRIABLE_FAILURES = { + "ValidationError", + "TimeoutError", + "TemporaryResourceError", + "MergeConflictError" # May succeed with different merge strategy + } + + NON_RETRIABLE_ERRORS = { + "ConfigurationError", + "AuthenticationError", + "MissingResourceError", + "CircularDependencyError" + } + ``` + + - [ ] Commit: "feat(domain): define retriable vs non-retriable failures" + + - [ ] **E1.6** [Rui] Write Behave tests for subplan model: + - [ ] **E1.6a** [Rui] Plan hierarchy scenarios: + - [ ] Scenario: Plan with parent_plan_id has is_subplan=True + - [ ] Given Plan with parent_plan_id set + - [ ] Then is_subplan returns True + - [ ] And is_root_plan returns False + - [ ] Scenario: Root plan has is_subplan=False and is_root_plan=True + - [ ] Scenario: SubplanConfig validates max_parallel bounds + - [ ] Commit: "test(behave): add plan hierarchy scenarios" + - [ ] **E1.6b** [Rui] Execution mode scenarios: + - [ ] Scenario: ExecutionMode enum has all required values + - [ ] Scenario: MergeStrategy enum has all required values + - [ ] Scenario: SubplanConfig defaults are applied + - [ ] Commit: "test(behave): add execution mode scenarios" + - [ ] **E1.6c** [Rui] SubplanStatus scenarios: + - [ ] Scenario: SubplanStatus tracks state correctly + - [ ] Scenario: SubplanAttempt records retry history + - [ ] Scenario: Failure handler respects fail_fast setting + - [ ] Commit: "test(behave): add SubplanStatus scenarios" - [ ] **Stage E2: Subplan Spawning** (Day 12-13) **[Jeff + Luis]** - **SEQUENTIAL ORDER**: E2.1 (Service scaffold) → E2.2 (spawn_subplan) → E2.3 (spawn_batch) → E2.4 (queries) → E2.5 (strategy actor) → E2.6 (execute phase) → E2.7 (status tracking) → E2.8 (bounded context) → E2.9 (Tests) + **SEQUENTIAL ORDER**: E2.1 (Service scaffold) → E2.2 (spawn_subplan) → E2.3 (spawn_batch) → E2.4 (queries) → E2.5 (strategy actor) → E2.6 (execute phase) → E2.7 (status tracking) → E2.8 (bounded context) → E2.9 (Tests) + - [ ] **E2.1** [Jeff] Create `SubplanService` scaffold in `src/cleveragents/application/services/subplan_service.py`: + - [ ] **E2.1a** [Jeff] Define service class: - - [ ] **E2.1** [Jeff] Create `SubplanService` scaffold in `src/cleveragents/application/services/subplan_service.py`: - - [ ] **E2.1a** [Jeff] Define service class: - ```python - class SubplanService: - """Service for spawning and managing subplans.""" + ```python + class SubplanService: + """Service for spawning and managing subplans.""" - def __init__( - self, - plan_service: PlanLifecycleService, - decision_service: DecisionService, - plan_repo: LifecyclePlanRepository, - context_builder: ContextBuilder - ): - self._plan_service = plan_service - self._decision_service = decision_service - self._plan_repo = plan_repo - self._context_builder = context_builder - ``` - - [ ] Commit: "feat(service): add SubplanService scaffold" - - [ ] **E2.2** [Jeff] Implement `spawn_subplan()` method: - - [ ] **E2.2a** [Jeff] Core implementation: - ```python - def spawn_subplan( - self, - parent_plan: Plan, - decision: Decision, - action_name: str, - target_resources: list[str] | None = None, - arguments: dict | None = None - ) -> Plan: - """Spawn a single subplan from a parent plan.""" - # Validate parent is not already a deep subplan - if parent_plan.depth >= 10: # Max nesting depth - raise MaxSubplanDepthError(f"Cannot spawn subplan at depth {parent_plan.depth + 1}") + def __init__( + self, + plan_service: PlanLifecycleService, + decision_service: DecisionService, + plan_repo: LifecyclePlanRepository, + context_builder: ContextBuilder + ): + self._plan_service = plan_service + self._decision_service = decision_service + self._plan_repo = plan_repo + self._context_builder = context_builder + ``` - # Create subplan via plan service - subplan = self._plan_service.use_action( - action_name=action_name, - project_ids=target_resources or parent_plan.project_ids, - arguments=arguments or {}, - parent_plan_id=parent_plan.plan_id, - root_plan_id=parent_plan.root_plan_id or parent_plan.plan_id, - automation_level=parent_plan.automation_level - ) + - [ ] Commit: "feat(service): add SubplanService scaffold" - # Link to decision - self._decision_service.add_downstream_plan(decision.decision_id, subplan.plan_id) + - [ ] **E2.2** [Jeff] Implement `spawn_subplan()` method: + - [ ] **E2.2a** [Jeff] Core implementation: - # Create status tracking - status = SubplanStatus( - subplan_id=subplan.plan_id, - action_name=action_name, - target_resources=target_resources or [] - ) + ```python + def spawn_subplan( + self, + parent_plan: Plan, + decision: Decision, + action_name: str, + target_resources: list[str] | None = None, + arguments: dict | None = None + ) -> Plan: + """Spawn a single subplan from a parent plan.""" + # Validate parent is not already a deep subplan + if parent_plan.depth >= 10: # Max nesting depth + raise MaxSubplanDepthError(f"Cannot spawn subplan at depth {parent_plan.depth + 1}") - # Update parent with new subplan status - self._update_parent_subplan_status(parent_plan.plan_id, status) + # Create subplan via plan service + subplan = self._plan_service.use_action( + action_name=action_name, + project_ids=target_resources or parent_plan.project_ids, + arguments=arguments or {}, + parent_plan_id=parent_plan.plan_id, + root_plan_id=parent_plan.root_plan_id or parent_plan.plan_id, + automation_level=parent_plan.automation_level + ) - logger.info(f"Spawned subplan {subplan.plan_id} from parent {parent_plan.plan_id}") - return subplan - ``` - - [ ] Commit: "feat(service): implement spawn_subplan()" - - [ ] **E2.2b** [Jeff] Add bounded context calculation: - ```python - def _build_bounded_context( - self, - parent_plan: Plan, - decision: Decision, - target_resources: list[str] - ) -> BoundedContext: - """Build bounded context for subplan from decision scope.""" - return self._context_builder.build_from_decision( - parent_context=parent_plan.context, - decision=decision, - resource_filter=target_resources - ) - ``` - - [ ] Commit: "feat(service): add bounded context for subplans" - - [ ] **E2.3** [Jeff] Implement `spawn_batch()` method: - - [ ] **E2.3a** [Jeff] Batch spawning implementation: - ```python - def spawn_batch( - self, - parent_plan: Plan, - decisions: list[Decision], - execution_mode: ExecutionMode = ExecutionMode.PARALLEL - ) -> list[Plan]: - """Spawn multiple subplans at once.""" - subplans = [] + # Link to decision + self._decision_service.add_downstream_plan(decision.decision_id, subplan.plan_id) - for decision in decisions: - if decision.decision_type != DecisionType.SUBPLAN_SPAWN: - continue + # Create status tracking + status = SubplanStatus( + subplan_id=subplan.plan_id, + action_name=action_name, + target_resources=target_resources or [] + ) - # Extract spawn parameters from decision - spawn_params = self._extract_spawn_params(decision) + # Update parent with new subplan status + self._update_parent_subplan_status(parent_plan.plan_id, status) - subplan = self.spawn_subplan( - parent_plan=parent_plan, - decision=decision, - action_name=spawn_params["action_name"], - target_resources=spawn_params.get("target_resources"), - arguments=spawn_params.get("arguments") - ) - subplans.append(subplan) + logger.info(f"Spawned subplan {subplan.plan_id} from parent {parent_plan.plan_id}") + return subplan + ``` - # Update parent's execution mode - self._update_parent_execution_mode(parent_plan.plan_id, execution_mode) + - [ ] Commit: "feat(service): implement spawn_subplan()" - logger.info(f"Spawned {len(subplans)} subplans from parent {parent_plan.plan_id}") - return subplans + - [ ] **E2.2b** [Jeff] Add bounded context calculation: - def _extract_spawn_params(self, decision: Decision) -> dict: - """Extract spawn parameters from SUBPLAN_SPAWN decision.""" - # Parse chosen_option which contains action name and params - # Format: "action_name:arg1=val1,arg2=val2" - return { - "action_name": decision.chosen_option.split(":")[0], - "target_resources": decision.context_snapshot.relevant_resources, - "arguments": {} # Parsed from decision metadata - } - ``` - - [ ] Commit: "feat(service): implement spawn_batch()" - - [ ] **E2.4** [Jeff] Implement query methods: - - [ ] **E2.4a** [Jeff] Implement `get_subplans()`: - ```python - def get_subplans(self, parent_plan_id: str) -> list[Plan]: - """Get all direct child subplans.""" - return self._plan_repo.get_children(parent_plan_id) + ```python + def _build_bounded_context( + self, + parent_plan: Plan, + decision: Decision, + target_resources: list[str] + ) -> BoundedContext: + """Build bounded context for subplan from decision scope.""" + return self._context_builder.build_from_decision( + parent_context=parent_plan.context, + decision=decision, + resource_filter=target_resources + ) + ``` - def get_subplan_statuses(self, parent_plan_id: str) -> list[SubplanStatus]: - """Get status tracking for all subplans.""" - parent = self._plan_repo.get_by_id(parent_plan_id) - return parent.subplan_statuses if parent else [] - ``` - - [ ] Commit: "feat(service): implement get_subplans()" - - [ ] **E2.4b** [Jeff] Implement `get_full_tree()`: - ```python - def get_full_tree(self, root_plan_id: str) -> PlanTree: - """Get full subplan tree from root.""" - plans = self._plan_repo.get_tree(root_plan_id) - return self._build_plan_tree(plans, root_plan_id) + - [ ] Commit: "feat(service): add bounded context for subplans" - def _build_plan_tree(self, plans: list[Plan], root_id: str) -> PlanTree: - """Build tree structure from flat list.""" - by_parent: dict[str, list[Plan]] = {} - root = None + - [ ] **E2.3** [Jeff] Implement `spawn_batch()` method: + - [ ] **E2.3a** [Jeff] Batch spawning implementation: - for plan in plans: - if plan.plan_id == root_id: - root = plan - elif plan.parent_plan_id: - by_parent.setdefault(plan.parent_plan_id, []).append(plan) + ```python + def spawn_batch( + self, + parent_plan: Plan, + decisions: list[Decision], + execution_mode: ExecutionMode = ExecutionMode.PARALLEL + ) -> list[Plan]: + """Spawn multiple subplans at once.""" + subplans = [] - def build_node(plan: Plan) -> PlanTreeNode: - children = [build_node(c) for c in by_parent.get(plan.plan_id, [])] - return PlanTreeNode(plan=plan, children=children) + for decision in decisions: + if decision.decision_type != DecisionType.SUBPLAN_SPAWN: + continue - return PlanTree(root=build_node(root), total_count=len(plans)) - ``` - - [ ] Commit: "feat(service): implement get_full_tree()" - - [ ] **E2.5** [Aditya] Configure strategy actor to emit subplan_spawn decisions: - - [ ] **E2.5a** [Aditya] Add plan_subplan tool to strategy actor: - ```yaml - # In strategy_actor.yaml - tools: - - name: plan_subplan - description: | - Decompose work into a subplan for parallel or sequential execution. - Use when work can be broken into independent pieces. - parameters: - - name: action_name - type: string - description: Action to use for subplan (e.g., "local/code-fix") - - name: description - type: string - description: What the subplan should accomplish - - name: target_files - type: array - description: Files this subplan should work on - - name: execution_mode - type: string - enum: [sequential, parallel, dependency_ordered] - description: How this relates to other subplans - - name: depends_on - type: array - description: IDs of subplans this depends on (for dependency_ordered) - code: | - result = context.create_subplan_decision( - action_name=input_data["action_name"], - description=input_data["description"], - target_files=input_data.get("target_files", []), - execution_mode=input_data.get("execution_mode", "parallel"), - depends_on=input_data.get("depends_on", []) - ) - ``` - - [ ] Commit: "feat(actor): add plan_subplan tool to strategy actor" - - [ ] **E2.5b** [Aditya] Implement `context.create_subplan_decision()`: - ```python - def create_subplan_decision( - self, - action_name: str, - description: str, - target_files: list[str], - execution_mode: str = "parallel", - depends_on: list[str] | None = None - ) -> str: - """Create a SUBPLAN_SPAWN decision (not actual plan yet).""" - decision = self._decision_service.record_decision( - plan_id=self.plan_id, - decision_type=DecisionType.SUBPLAN_SPAWN, - question=f"Should we create subplan for: {description}", - chosen_option=f"{action_name}:{','.join(target_files)}", - hot_context=self._get_context_for_files(target_files), - resources=target_files, - parent_decision_id=self._current_decision_id, - rationale=description - ) + # Extract spawn parameters from decision + spawn_params = self._extract_spawn_params(decision) - # Store metadata for Execute phase to process - self._pending_subplans.append({ - "decision_id": decision.decision_id, - "action_name": action_name, - "target_files": target_files, - "execution_mode": execution_mode, - "depends_on": depends_on or [] - }) + subplan = self.spawn_subplan( + parent_plan=parent_plan, + decision=decision, + action_name=spawn_params["action_name"], + target_resources=spawn_params.get("target_resources"), + arguments=spawn_params.get("arguments") + ) + subplans.append(subplan) - return decision.decision_id - ``` - - [ ] Commit: "feat(context): implement create_subplan_decision()" - - [ ] **E2.6** [Jeff] Execute phase processes subplan decisions: - - [ ] **E2.6a** [Jeff] Add subplan processing to execute phase: - ```python - # In PlanLifecycleService.execute_execution() + # Update parent's execution mode + self._update_parent_execution_mode(parent_plan.plan_id, execution_mode) - async def _process_subplan_decisions(self, plan: Plan) -> None: - """Process SUBPLAN_SPAWN decisions after strategy.""" - # Get all pending subplan decisions - decisions = self._decision_service.get_by_plan_and_type( - plan.plan_id, DecisionType.SUBPLAN_SPAWN - ) + logger.info(f"Spawned {len(subplans)} subplans from parent {parent_plan.plan_id}") + return subplans - if not decisions: - return + def _extract_spawn_params(self, decision: Decision) -> dict: + """Extract spawn parameters from SUBPLAN_SPAWN decision.""" + # Parse chosen_option which contains action name and params + # Format: "action_name:arg1=val1,arg2=val2" + return { + "action_name": decision.chosen_option.split(":")[0], + "target_resources": decision.context_snapshot.relevant_resources, + "arguments": {} # Parsed from decision metadata + } + ``` - # Group by execution mode - parallel_decisions = [] - sequential_decisions = [] - dependency_decisions = [] + - [ ] Commit: "feat(service): implement spawn_batch()" - for d in decisions: - mode = self._get_execution_mode(d) - if mode == ExecutionMode.PARALLEL: - parallel_decisions.append(d) - elif mode == ExecutionMode.SEQUENTIAL: - sequential_decisions.append(d) - else: - dependency_decisions.append(d) + - [ ] **E2.4** [Jeff] Implement query methods: + - [ ] **E2.4a** [Jeff] Implement `get_subplans()`: - # Execute in appropriate order - if parallel_decisions: - await self._execute_parallel_subplans(plan, parallel_decisions) - if sequential_decisions: - await self._execute_sequential_subplans(plan, sequential_decisions) - if dependency_decisions: - await self._execute_dependency_ordered_subplans(plan, dependency_decisions) - ``` - - [ ] Commit: "feat(service): add subplan decision processing to execute phase" - - [ ] **E2.6b** [Jeff] Implement sequential execution: - ```python - async def _execute_sequential_subplans( - self, - parent: Plan, - decisions: list[Decision] - ) -> None: - """Execute subplans one at a time in order.""" - for decision in decisions: - subplan = self._subplan_service.spawn_subplan( - parent_plan=parent, - decision=decision, - action_name=self._extract_action_name(decision) - ) + ```python + def get_subplans(self, parent_plan_id: str) -> list[Plan]: + """Get all direct child subplans.""" + return self._plan_repo.get_children(parent_plan_id) - # Execute and wait - await self._execute_subplan(subplan) + def get_subplan_statuses(self, parent_plan_id: str) -> list[SubplanStatus]: + """Get status tracking for all subplans.""" + parent = self._plan_repo.get_by_id(parent_plan_id) + return parent.subplan_statuses if parent else [] + ``` - # Check result - status = self._get_subplan_status(parent, subplan.plan_id) - if status.status == ProcessingState.ERRORED: - if parent.subplan_config.fail_fast: - raise SubplanFailedError(subplan.plan_id, status.error) - # Otherwise continue to next - ``` - - [ ] Commit: "feat(service): implement sequential subplan execution" - - [ ] **E2.7** [Luis] Implement subplan status tracking: - - [ ] **E2.7a** [Luis] Create status update mechanism: - ```python - class SubplanStatusTracker: - """Track and update subplan statuses.""" + - [ ] Commit: "feat(service): implement get_subplans()" - def __init__(self, plan_repo: LifecyclePlanRepository): - self._plan_repo = plan_repo - self._listeners: dict[str, list[Callable]] = {} + - [ ] **E2.4b** [Jeff] Implement `get_full_tree()`: - def update_status( - self, - parent_plan_id: str, - subplan_id: str, - new_status: ProcessingState, - error: str | None = None - ) -> None: - """Update status of a subplan.""" - parent = self._plan_repo.get_by_id(parent_plan_id) - if not parent: - return + ```python + def get_full_tree(self, root_plan_id: str) -> PlanTree: + """Get full subplan tree from root.""" + plans = self._plan_repo.get_tree(root_plan_id) + return self._build_plan_tree(plans, root_plan_id) - # Find and update status - for status in parent.subplan_statuses: - if status.subplan_id == subplan_id: - status.status = new_status - if new_status == ProcessingState.PROCESSING: - status.started_at = datetime.utcnow() - elif new_status in (ProcessingState.COMPLETE, ProcessingState.ERRORED): - status.completed_at = datetime.utcnow() - if error: - status.error = error - break + def _build_plan_tree(self, plans: list[Plan], root_id: str) -> PlanTree: + """Build tree structure from flat list.""" + by_parent: dict[str, list[Plan]] = {} + root = None - # Persist - self._plan_repo.update(parent) + for plan in plans: + if plan.plan_id == root_id: + root = plan + elif plan.parent_plan_id: + by_parent.setdefault(plan.parent_plan_id, []).append(plan) - # Notify listeners - self._notify_listeners(parent_plan_id, subplan_id, new_status) + def build_node(plan: Plan) -> PlanTreeNode: + children = [build_node(c) for c in by_parent.get(plan.plan_id, [])] + return PlanTreeNode(plan=plan, children=children) - def subscribe(self, parent_plan_id: str, callback: Callable) -> None: - """Subscribe to status updates for a parent plan.""" - self._listeners.setdefault(parent_plan_id, []).append(callback) - ``` - - [ ] Commit: "feat(service): implement SubplanStatusTracker" - - [ ] **E2.7b** [Luis] Determine parent state from subplan states: - ```python - def compute_parent_state(self, parent: Plan) -> ProcessingState: - """Compute parent state based on subplan states.""" - statuses = parent.subplan_statuses + return PlanTree(root=build_node(root), total_count=len(plans)) + ``` - if not statuses: - return parent.state + - [ ] Commit: "feat(service): implement get_full_tree()" - # Count states - errored = sum(1 for s in statuses if s.status == ProcessingState.ERRORED) - complete = sum(1 for s in statuses if s.status == ProcessingState.COMPLETE) - processing = sum(1 for s in statuses if s.status == ProcessingState.PROCESSING) + - [ ] **E2.5** [Aditya] Configure strategy actor to emit subplan_spawn decisions: + - [ ] **E2.5a** [Aditya] Add plan_subplan tool to strategy actor: - config = parent.subplan_config or SubplanConfig() + ```yaml + # In strategy_actor.yaml + tools: + - name: plan_subplan + description: | + Decompose work into a subplan for parallel or sequential execution. + Use when work can be broken into independent pieces. + parameters: + - name: action_name + type: string + description: Action to use for subplan (e.g., "local/code-fix") + - name: description + type: string + description: What the subplan should accomplish + - name: target_files + type: array + description: Files this subplan should work on + - name: execution_mode + type: string + enum: [sequential, parallel, dependency_ordered] + description: How this relates to other subplans + - name: depends_on + type: array + description: IDs of subplans this depends on (for dependency_ordered) + code: | + result = context.create_subplan_decision( + action_name=input_data["action_name"], + description=input_data["description"], + target_files=input_data.get("target_files", []), + execution_mode=input_data.get("execution_mode", "parallel"), + depends_on=input_data.get("depends_on", []) + ) + ``` - # Determine parent state - if processing > 0: - return ProcessingState.PROCESSING + - [ ] Commit: "feat(actor): add plan_subplan tool to strategy actor" - if errored > 0: - if config.execution_mode == ExecutionMode.PARALLEL: - # Parallel: error only if ALL failed - if errored == len(statuses): - return ProcessingState.ERRORED - else: - # Sequential: error on first failure - return ProcessingState.ERRORED + - [ ] **E2.5b** [Aditya] Implement `context.create_subplan_decision()`: - if complete == len(statuses): - return ProcessingState.COMPLETE + ```python + def create_subplan_decision( + self, + action_name: str, + description: str, + target_files: list[str], + execution_mode: str = "parallel", + depends_on: list[str] | None = None + ) -> str: + """Create a SUBPLAN_SPAWN decision (not actual plan yet).""" + decision = self._decision_service.record_decision( + plan_id=self.plan_id, + decision_type=DecisionType.SUBPLAN_SPAWN, + question=f"Should we create subplan for: {description}", + chosen_option=f"{action_name}:{','.join(target_files)}", + hot_context=self._get_context_for_files(target_files), + resources=target_files, + parent_decision_id=self._current_decision_id, + rationale=description + ) - return ProcessingState.QUEUED # Some still pending - ``` - - [ ] Commit: "feat(service): implement parent state computation" - - [ ] **E2.8** [Luis] Implement bounded context for subplans: - - [ ] **E2.8a** [Luis] Create `ContextBuilder` for bounded contexts: - ```python - class ContextBuilder: - """Build bounded contexts for subplans.""" + # Store metadata for Execute phase to process + self._pending_subplans.append({ + "decision_id": decision.decision_id, + "action_name": action_name, + "target_files": target_files, + "execution_mode": execution_mode, + "depends_on": depends_on or [] + }) - def build_from_decision( - self, - parent_context: PlanContext, - decision: Decision, - resource_filter: list[str] - ) -> BoundedContext: - """Build context bounded to decision scope.""" - # Filter files to only those relevant - relevant_files = self._filter_files( - parent_context.files, - resource_filter - ) + return decision.decision_id + ``` - # Include decision chain for reference - decision_chain = self._get_decision_chain(decision) + - [ ] Commit: "feat(context): implement create_subplan_decision()" - return BoundedContext( - files=relevant_files, - decision_chain=decision_chain, - parent_context_ref=parent_context.context_id, - boundary=resource_filter - ) + - [ ] **E2.6** [Jeff] Execute phase processes subplan decisions: + - [ ] **E2.6a** [Jeff] Add subplan processing to execute phase: - def _filter_files( - self, - files: dict[str, FileContent], - patterns: list[str] - ) -> dict[str, FileContent]: - """Filter files to match patterns.""" - import fnmatch - result = {} - for path, content in files.items(): - if any(fnmatch.fnmatch(path, p) for p in patterns): - result[path] = content - return result - ``` - - [ ] Commit: "feat(context): implement ContextBuilder for bounded contexts" - - [ ] **E2.9** [Rui] Write integration tests for subplan spawning: - - [ ] **E2.9a** [Rui] Spawn scenarios: - - [ ] Scenario: SUBPLAN_SPAWN decision creates child plan - - [ ] Given strategy produces SUBPLAN_SPAWN decision - - [ ] When execute phase processes decisions - - [ ] Then child plan is created with correct parent_plan_id - - [ ] And decision.downstream_plan_ids contains subplan ID - - [ ] Scenario: spawn_batch creates multiple subplans - - [ ] Given 3 SUBPLAN_SPAWN decisions - - [ ] When spawn_batch is called - - [ ] Then 3 subplans are created - - [ ] Commit: "test(behave): add subplan spawn scenarios" - - [ ] **E2.9b** [Rui] Execution order scenarios: - - [ ] Scenario: Sequential subplans execute in order - - [ ] Given 3 sequential subplans S1, S2, S3 - - [ ] When executed - - [ ] Then S1 completes before S2 starts - - [ ] And S2 completes before S3 starts - - [ ] Scenario: Parallel subplans execute concurrently - - [ ] Given 3 parallel subplans - - [ ] When executed with max_parallel=3 - - [ ] Then all 3 start at approximately same time - - [ ] Commit: "test(behave): add execution order scenarios" - - [ ] **E2.9c** [Rui] Failure scenarios: - - [ ] Scenario: Failed sequential subplan stops processing - - [ ] Given sequential subplans S1, S2, S3 - - [ ] When S2 fails - - [ ] Then S3 is not started - - [ ] And parent enters ERRORED state - - [ ] Scenario: Failed parallel subplan allows others to finish - - [ ] Given parallel subplans S1, S2, S3 - - [ ] And fail_fast=False - - [ ] When S2 fails - - [ ] Then S1 and S3 continue to completion - - [ ] Commit: "test(behave): add subplan failure scenarios" + ```python + # In PlanLifecycleService.execute_execution() + + async def _process_subplan_decisions(self, plan: Plan) -> None: + """Process SUBPLAN_SPAWN decisions after strategy.""" + # Get all pending subplan decisions + decisions = self._decision_service.get_by_plan_and_type( + plan.plan_id, DecisionType.SUBPLAN_SPAWN + ) + + if not decisions: + return + + # Group by execution mode + parallel_decisions = [] + sequential_decisions = [] + dependency_decisions = [] + + for d in decisions: + mode = self._get_execution_mode(d) + if mode == ExecutionMode.PARALLEL: + parallel_decisions.append(d) + elif mode == ExecutionMode.SEQUENTIAL: + sequential_decisions.append(d) + else: + dependency_decisions.append(d) + + # Execute in appropriate order + if parallel_decisions: + await self._execute_parallel_subplans(plan, parallel_decisions) + if sequential_decisions: + await self._execute_sequential_subplans(plan, sequential_decisions) + if dependency_decisions: + await self._execute_dependency_ordered_subplans(plan, dependency_decisions) + ``` + + - [ ] Commit: "feat(service): add subplan decision processing to execute phase" + + - [ ] **E2.6b** [Jeff] Implement sequential execution: + + ```python + async def _execute_sequential_subplans( + self, + parent: Plan, + decisions: list[Decision] + ) -> None: + """Execute subplans one at a time in order.""" + for decision in decisions: + subplan = self._subplan_service.spawn_subplan( + parent_plan=parent, + decision=decision, + action_name=self._extract_action_name(decision) + ) + + # Execute and wait + await self._execute_subplan(subplan) + + # Check result + status = self._get_subplan_status(parent, subplan.plan_id) + if status.status == ProcessingState.ERRORED: + if parent.subplan_config.fail_fast: + raise SubplanFailedError(subplan.plan_id, status.error) + # Otherwise continue to next + ``` + + - [ ] Commit: "feat(service): implement sequential subplan execution" + + - [ ] **E2.7** [Luis] Implement subplan status tracking: + - [ ] **E2.7a** [Luis] Create status update mechanism: + + ```python + class SubplanStatusTracker: + """Track and update subplan statuses.""" + + def __init__(self, plan_repo: LifecyclePlanRepository): + self._plan_repo = plan_repo + self._listeners: dict[str, list[Callable]] = {} + + def update_status( + self, + parent_plan_id: str, + subplan_id: str, + new_status: ProcessingState, + error: str | None = None + ) -> None: + """Update status of a subplan.""" + parent = self._plan_repo.get_by_id(parent_plan_id) + if not parent: + return + + # Find and update status + for status in parent.subplan_statuses: + if status.subplan_id == subplan_id: + status.status = new_status + if new_status == ProcessingState.PROCESSING: + status.started_at = datetime.utcnow() + elif new_status in (ProcessingState.COMPLETE, ProcessingState.ERRORED): + status.completed_at = datetime.utcnow() + if error: + status.error = error + break + + # Persist + self._plan_repo.update(parent) + + # Notify listeners + self._notify_listeners(parent_plan_id, subplan_id, new_status) + + def subscribe(self, parent_plan_id: str, callback: Callable) -> None: + """Subscribe to status updates for a parent plan.""" + self._listeners.setdefault(parent_plan_id, []).append(callback) + ``` + + - [ ] Commit: "feat(service): implement SubplanStatusTracker" + + - [ ] **E2.7b** [Luis] Determine parent state from subplan states: + + ```python + def compute_parent_state(self, parent: Plan) -> ProcessingState: + """Compute parent state based on subplan states.""" + statuses = parent.subplan_statuses + + if not statuses: + return parent.state + + # Count states + errored = sum(1 for s in statuses if s.status == ProcessingState.ERRORED) + complete = sum(1 for s in statuses if s.status == ProcessingState.COMPLETE) + processing = sum(1 for s in statuses if s.status == ProcessingState.PROCESSING) + + config = parent.subplan_config or SubplanConfig() + + # Determine parent state + if processing > 0: + return ProcessingState.PROCESSING + + if errored > 0: + if config.execution_mode == ExecutionMode.PARALLEL: + # Parallel: error only if ALL failed + if errored == len(statuses): + return ProcessingState.ERRORED + else: + # Sequential: error on first failure + return ProcessingState.ERRORED + + if complete == len(statuses): + return ProcessingState.COMPLETE + + return ProcessingState.QUEUED # Some still pending + ``` + + - [ ] Commit: "feat(service): implement parent state computation" + + - [ ] **E2.8** [Luis] Implement bounded context for subplans: + - [ ] **E2.8a** [Luis] Create `ContextBuilder` for bounded contexts: + + ```python + class ContextBuilder: + """Build bounded contexts for subplans.""" + + def build_from_decision( + self, + parent_context: PlanContext, + decision: Decision, + resource_filter: list[str] + ) -> BoundedContext: + """Build context bounded to decision scope.""" + # Filter files to only those relevant + relevant_files = self._filter_files( + parent_context.files, + resource_filter + ) + + # Include decision chain for reference + decision_chain = self._get_decision_chain(decision) + + return BoundedContext( + files=relevant_files, + decision_chain=decision_chain, + parent_context_ref=parent_context.context_id, + boundary=resource_filter + ) + + def _filter_files( + self, + files: dict[str, FileContent], + patterns: list[str] + ) -> dict[str, FileContent]: + """Filter files to match patterns.""" + import fnmatch + result = {} + for path, content in files.items(): + if any(fnmatch.fnmatch(path, p) for p in patterns): + result[path] = content + return result + ``` + + - [ ] Commit: "feat(context): implement ContextBuilder for bounded contexts" + + - [ ] **E2.9** [Rui] Write integration tests for subplan spawning: + - [ ] **E2.9a** [Rui] Spawn scenarios: + - [ ] Scenario: SUBPLAN_SPAWN decision creates child plan + - [ ] Given strategy produces SUBPLAN_SPAWN decision + - [ ] When execute phase processes decisions + - [ ] Then child plan is created with correct parent_plan_id + - [ ] And decision.downstream_plan_ids contains subplan ID + - [ ] Scenario: spawn_batch creates multiple subplans + - [ ] Given 3 SUBPLAN_SPAWN decisions + - [ ] When spawn_batch is called + - [ ] Then 3 subplans are created + - [ ] Commit: "test(behave): add subplan spawn scenarios" + - [ ] **E2.9b** [Rui] Execution order scenarios: + - [ ] Scenario: Sequential subplans execute in order + - [ ] Given 3 sequential subplans S1, S2, S3 + - [ ] When executed + - [ ] Then S1 completes before S2 starts + - [ ] And S2 completes before S3 starts + - [ ] Scenario: Parallel subplans execute concurrently + - [ ] Given 3 parallel subplans + - [ ] When executed with max_parallel=3 + - [ ] Then all 3 start at approximately same time + - [ ] Commit: "test(behave): add execution order scenarios" + - [ ] **E2.9c** [Rui] Failure scenarios: + - [ ] Scenario: Failed sequential subplan stops processing + - [ ] Given sequential subplans S1, S2, S3 + - [ ] When S2 fails + - [ ] Then S3 is not started + - [ ] And parent enters ERRORED state + - [ ] Scenario: Failed parallel subplan allows others to finish + - [ ] Given parallel subplans S1, S2, S3 + - [ ] And fail_fast=False + - [ ] When S2 fails + - [ ] Then S1 and S3 continue to completion + - [ ] Commit: "test(behave): add subplan failure scenarios" - [ ] **Stage E3: Parallel Execution** (Day 19) **[Jeff + Luis]** - **SEQUENTIAL ORDER**: E3.1 (AsyncExecutor) → E3.2 (Semaphore) → E3.3 (Timeouts) → E3.4 (DAG) → E3.5 (Isolation) → E3.6 (Tests) + **SEQUENTIAL ORDER**: E3.1 (AsyncExecutor) → E3.2 (Semaphore) → E3.3 (Timeouts) → E3.4 (DAG) → E3.5 (Isolation) → E3.6 (Tests) + - [ ] **E3.1** [Jeff] Implement async subplan executor: + - [ ] **E3.1a** [Jeff] Create `AsyncSubplanExecutor` class: - - [ ] **E3.1** [Jeff] Implement async subplan executor: - - [ ] **E3.1a** [Jeff] Create `AsyncSubplanExecutor` class: - ```python - class AsyncSubplanExecutor: - """Execute subplans asynchronously with concurrency control.""" + ```python + class AsyncSubplanExecutor: + """Execute subplans asynchronously with concurrency control.""" - def __init__( - self, - plan_service: PlanLifecycleService, - status_tracker: SubplanStatusTracker - ): - self._plan_service = plan_service - self._status_tracker = status_tracker + def __init__( + self, + plan_service: PlanLifecycleService, + status_tracker: SubplanStatusTracker + ): + self._plan_service = plan_service + self._status_tracker = status_tracker - async def execute_parallel( - self, - parent: Plan, - subplans: list[Plan], - config: SubplanConfig - ) -> list[SubplanResult]: - """Execute subplans in parallel with concurrency limit.""" - semaphore = asyncio.Semaphore(config.max_parallel) + async def execute_parallel( + self, + parent: Plan, + subplans: list[Plan], + config: SubplanConfig + ) -> list[SubplanResult]: + """Execute subplans in parallel with concurrency limit.""" + semaphore = asyncio.Semaphore(config.max_parallel) - async def execute_with_limit(subplan: Plan) -> SubplanResult: - async with semaphore: - return await self._execute_single(subplan, config) + async def execute_with_limit(subplan: Plan) -> SubplanResult: + async with semaphore: + return await self._execute_single(subplan, config) - tasks = [execute_with_limit(sp) for sp in subplans] - results = await asyncio.gather(*tasks, return_exceptions=True) + tasks = [execute_with_limit(sp) for sp in subplans] + results = await asyncio.gather(*tasks, return_exceptions=True) - return self._process_results(results, subplans) - ``` - - [ ] Commit: "feat(executor): add AsyncSubplanExecutor" - - [ ] **E3.1b** [Jeff] Implement single subplan execution: - ```python - async def _execute_single( - self, - subplan: Plan, - config: SubplanConfig - ) -> SubplanResult: - """Execute a single subplan with timeout.""" - try: - # Apply timeout if configured - if config.timeout_per_subplan_seconds: - result = await asyncio.wait_for( - self._run_subplan(subplan), - timeout=config.timeout_per_subplan_seconds - ) - else: - result = await self._run_subplan(subplan) + return self._process_results(results, subplans) + ``` - return SubplanResult( - subplan_id=subplan.plan_id, - success=True, - changeset=result.changeset - ) + - [ ] Commit: "feat(executor): add AsyncSubplanExecutor" - except asyncio.TimeoutError: - self._status_tracker.update_status( - subplan.parent_plan_id, - subplan.plan_id, - ProcessingState.ERRORED, - error="Timeout exceeded" - ) - return SubplanResult( - subplan_id=subplan.plan_id, - success=False, - error="TimeoutError" - ) + - [ ] **E3.1b** [Jeff] Implement single subplan execution: - except Exception as e: - self._status_tracker.update_status( - subplan.parent_plan_id, - subplan.plan_id, - ProcessingState.ERRORED, - error=str(e) - ) - return SubplanResult( - subplan_id=subplan.plan_id, - success=False, - error=str(e) - ) - ``` - - [ ] Commit: "feat(executor): implement single subplan execution with timeout" - - [ ] **E3.2** [Jeff] Implement dependency-ordered execution: - - [ ] **E3.2a** [Jeff] Build and validate dependency DAG: - ```python - def build_dependency_dag( - self, - decisions: list[Decision] - ) -> DependencyGraph: - """Build DAG from subplan decisions with depends_on.""" - graph = DependencyGraph() + ```python + async def _execute_single( + self, + subplan: Plan, + config: SubplanConfig + ) -> SubplanResult: + """Execute a single subplan with timeout.""" + try: + # Apply timeout if configured + if config.timeout_per_subplan_seconds: + result = await asyncio.wait_for( + self._run_subplan(subplan), + timeout=config.timeout_per_subplan_seconds + ) + else: + result = await self._run_subplan(subplan) - for decision in decisions: - graph.add_node(decision.decision_id) - depends_on = self._get_depends_on(decision) - for dep_id in depends_on: - graph.add_edge(dep_id, decision.decision_id) + return SubplanResult( + subplan_id=subplan.plan_id, + success=True, + changeset=result.changeset + ) - # Validate no cycles - if graph.has_cycle(): - cycle = graph.find_cycle() - raise CircularDependencyError(f"Cycle detected: {' -> '.join(cycle)}") + except asyncio.TimeoutError: + self._status_tracker.update_status( + subplan.parent_plan_id, + subplan.plan_id, + ProcessingState.ERRORED, + error="Timeout exceeded" + ) + return SubplanResult( + subplan_id=subplan.plan_id, + success=False, + error="TimeoutError" + ) - return graph - ``` - - [ ] Commit: "feat(executor): implement dependency DAG building" - - [ ] **E3.2b** [Jeff] Execute in topological order: - ```python - async def execute_dependency_ordered( - self, - parent: Plan, - decisions: list[Decision], - config: SubplanConfig - ) -> list[SubplanResult]: - """Execute subplans respecting dependency order.""" - dag = self.build_dependency_dag(decisions) - execution_order = dag.topological_sort() + except Exception as e: + self._status_tracker.update_status( + subplan.parent_plan_id, + subplan.plan_id, + ProcessingState.ERRORED, + error=str(e) + ) + return SubplanResult( + subplan_id=subplan.plan_id, + success=False, + error=str(e) + ) + ``` - results = [] - completed: set[str] = set() + - [ ] Commit: "feat(executor): implement single subplan execution with timeout" - # Process in waves - each wave contains independent nodes - while execution_order: - # Find all nodes whose dependencies are satisfied - ready = [ - node for node in execution_order - if all(dep in completed for dep in dag.get_dependencies(node)) - ] + - [ ] **E3.2** [Jeff] Implement dependency-ordered execution: + - [ ] **E3.2a** [Jeff] Build and validate dependency DAG: - if not ready: - break # Stuck - shouldn't happen with valid DAG + ```python + def build_dependency_dag( + self, + decisions: list[Decision] + ) -> DependencyGraph: + """Build DAG from subplan decisions with depends_on.""" + graph = DependencyGraph() - # Execute ready nodes in parallel - ready_decisions = [d for d in decisions if d.decision_id in ready] - subplans = [self._spawn_subplan(parent, d) for d in ready_decisions] + for decision in decisions: + graph.add_node(decision.decision_id) + depends_on = self._get_depends_on(decision) + for dep_id in depends_on: + graph.add_edge(dep_id, decision.decision_id) - wave_results = await self.execute_parallel(parent, subplans, config) - results.extend(wave_results) + # Validate no cycles + if graph.has_cycle(): + cycle = graph.find_cycle() + raise CircularDependencyError(f"Cycle detected: {' -> '.join(cycle)}") - # Mark completed - for r in wave_results: - if r.success: - completed.add(r.decision_id) + return graph + ``` - # Remove from order - execution_order = [n for n in execution_order if n not in ready] + - [ ] Commit: "feat(executor): implement dependency DAG building" - return results - ``` - - [ ] Commit: "feat(executor): implement dependency-ordered execution" - - [ ] **E3.3** [Luis] Implement subplan isolation: - - [ ] **E3.3a** [Luis] Ensure separate sandboxes: - ```python - def ensure_isolated_sandbox( - self, - subplan: Plan, - resource_service: ResourceService - ) -> None: - """Ensure subplan has its own isolated sandbox.""" - for resource_id in subplan.project_ids: - resource = self._get_resource(resource_id) - # Each subplan gets unique sandbox for same resource - sandbox = resource_service.access_resource( - plan_id=subplan.plan_id, # Use subplan ID, not parent - resource=resource, - mode=AccessMode.WRITE - ) - # Sandbox is isolated by plan_id - ``` - - [ ] Commit: "feat(executor): ensure isolated sandboxes for subplans" - - [ ] **E3.3b** [Luis] Prevent cross-subplan visibility: - ```python - def validate_isolation( - self, - subplan: Plan, - other_subplans: list[Plan] - ) -> None: - """Verify subplan cannot access other subplans' state.""" - subplan_sandbox = self._get_sandbox(subplan.plan_id) + - [ ] **E3.2b** [Jeff] Execute in topological order: - for other in other_subplans: - if other.plan_id == subplan.plan_id: - continue - other_sandbox = self._get_sandbox(other.plan_id) + ```python + async def execute_dependency_ordered( + self, + parent: Plan, + decisions: list[Decision], + config: SubplanConfig + ) -> list[SubplanResult]: + """Execute subplans respecting dependency order.""" + dag = self.build_dependency_dag(decisions) + execution_order = dag.topological_sort() - # Verify different paths - if subplan_sandbox.sandbox_path == other_sandbox.sandbox_path: - raise IsolationViolationError( - f"Subplans {subplan.plan_id} and {other.plan_id} share sandbox" - ) - ``` - - [ ] Commit: "feat(executor): add isolation validation" - - [ ] **E3.4** [Rui] Write tests for parallel execution: - - [ ] **E3.4a** [Rui] Concurrency scenarios: - - [ ] Scenario: 10 independent subplans run with max_parallel=5 - - [ ] Given 10 subplans with no dependencies - - [ ] And max_parallel=5 - - [ ] When executed in parallel - - [ ] Then at most 5 run concurrently at any time - - [ ] And all 10 complete successfully - - [ ] Commit: "test(behave): add concurrency limit scenarios" - - [ ] **E3.4b** [Rui] Dependency scenarios: - - [ ] Scenario: Dependency chain executes in correct order - - [ ] Given subplans A -> B -> C (B depends on A, C depends on B) - - [ ] When executed with dependency ordering - - [ ] Then A completes before B starts - - [ ] And B completes before C starts - - [ ] Scenario: Diamond dependency executes correctly - - [ ] Given A -> B, A -> C, B -> D, C -> D - - [ ] When executed - - [ ] Then A runs first - - [ ] Then B and C run in parallel - - [ ] Then D runs last - - [ ] Commit: "test(behave): add dependency ordering scenarios" - - [ ] **E3.4c** [Rui] Timeout scenarios: - - [ ] Scenario: Subplan timeout triggers failure - - [ ] Given subplan with timeout_per_subplan_seconds=10 - - [ ] When subplan takes 15 seconds - - [ ] Then subplan is marked ERRORED with TimeoutError - - [ ] Commit: "test(behave): add timeout scenarios" + results = [] + completed: set[str] = set() + + # Process in waves - each wave contains independent nodes + while execution_order: + # Find all nodes whose dependencies are satisfied + ready = [ + node for node in execution_order + if all(dep in completed for dep in dag.get_dependencies(node)) + ] + + if not ready: + break # Stuck - shouldn't happen with valid DAG + + # Execute ready nodes in parallel + ready_decisions = [d for d in decisions if d.decision_id in ready] + subplans = [self._spawn_subplan(parent, d) for d in ready_decisions] + + wave_results = await self.execute_parallel(parent, subplans, config) + results.extend(wave_results) + + # Mark completed + for r in wave_results: + if r.success: + completed.add(r.decision_id) + + # Remove from order + execution_order = [n for n in execution_order if n not in ready] + + return results + ``` + + - [ ] Commit: "feat(executor): implement dependency-ordered execution" + + - [ ] **E3.3** [Luis] Implement subplan isolation: + - [ ] **E3.3a** [Luis] Ensure separate sandboxes: + + ```python + def ensure_isolated_sandbox( + self, + subplan: Plan, + resource_service: ResourceService + ) -> None: + """Ensure subplan has its own isolated sandbox.""" + for resource_id in subplan.project_ids: + resource = self._get_resource(resource_id) + # Each subplan gets unique sandbox for same resource + sandbox = resource_service.access_resource( + plan_id=subplan.plan_id, # Use subplan ID, not parent + resource=resource, + mode=AccessMode.WRITE + ) + # Sandbox is isolated by plan_id + ``` + + - [ ] Commit: "feat(executor): ensure isolated sandboxes for subplans" + + - [ ] **E3.3b** [Luis] Prevent cross-subplan visibility: + + ```python + def validate_isolation( + self, + subplan: Plan, + other_subplans: list[Plan] + ) -> None: + """Verify subplan cannot access other subplans' state.""" + subplan_sandbox = self._get_sandbox(subplan.plan_id) + + for other in other_subplans: + if other.plan_id == subplan.plan_id: + continue + other_sandbox = self._get_sandbox(other.plan_id) + + # Verify different paths + if subplan_sandbox.sandbox_path == other_sandbox.sandbox_path: + raise IsolationViolationError( + f"Subplans {subplan.plan_id} and {other.plan_id} share sandbox" + ) + ``` + + - [ ] Commit: "feat(executor): add isolation validation" + + - [ ] **E3.4** [Rui] Write tests for parallel execution: + - [ ] **E3.4a** [Rui] Concurrency scenarios: + - [ ] Scenario: 10 independent subplans run with max_parallel=5 + - [ ] Given 10 subplans with no dependencies + - [ ] And max_parallel=5 + - [ ] When executed in parallel + - [ ] Then at most 5 run concurrently at any time + - [ ] And all 10 complete successfully + - [ ] Commit: "test(behave): add concurrency limit scenarios" + - [ ] **E3.4b** [Rui] Dependency scenarios: + - [ ] Scenario: Dependency chain executes in correct order + - [ ] Given subplans A -> B -> C (B depends on A, C depends on B) + - [ ] When executed with dependency ordering + - [ ] Then A completes before B starts + - [ ] And B completes before C starts + - [ ] Scenario: Diamond dependency executes correctly + - [ ] Given A -> B, A -> C, B -> D, C -> D + - [ ] When executed + - [ ] Then A runs first + - [ ] Then B and C run in parallel + - [ ] Then D runs last + - [ ] Commit: "test(behave): add dependency ordering scenarios" + - [ ] **E3.4c** [Rui] Timeout scenarios: + - [ ] Scenario: Subplan timeout triggers failure + - [ ] Given subplan with timeout_per_subplan_seconds=10 + - [ ] When subplan takes 15 seconds + - [ ] Then subplan is marked ERRORED with TimeoutError + - [ ] Commit: "test(behave): add timeout scenarios" - [ ] **Stage E4: Result Merging** (Day 20) **[Jeff + Luis]** - **SEQUENTIAL ORDER**: E4.1 (MergeService) → E4.2 (MergeResult) → E4.3 (ThreeWayMerge) → E4.4 (SequentialMerge) → E4.5 (Validation) → E4.6 (Tests) + **SEQUENTIAL ORDER**: E4.1 (MergeService) → E4.2 (MergeResult) → E4.3 (ThreeWayMerge) → E4.4 (SequentialMerge) → E4.5 (Validation) → E4.6 (Tests) + - [ ] **E4.1** [Jeff] Create `MergeService` in `src/cleveragents/application/services/merge_service.py`: + - [ ] **E4.1a** [Jeff] Define service class: - - [ ] **E4.1** [Jeff] Create `MergeService` in `src/cleveragents/application/services/merge_service.py`: - - [ ] **E4.1a** [Jeff] Define service class: - ```python - class MergeService: - """Service for merging subplan results.""" + ```python + class MergeService: + """Service for merging subplan results.""" - def __init__( - self, - sandbox_manager: SandboxManager, - validation_service: ValidationService - ): - self._sandbox_manager = sandbox_manager - self._validation_service = validation_service - ``` - - [ ] Commit: "feat(service): add MergeService scaffold" - - [ ] **E4.1b** [Jeff] Implement `merge_subplan_results()`: - ```python - def merge_subplan_results( - self, - parent: Plan, - subplans: list[Plan], - strategy: MergeStrategy = MergeStrategy.GIT_THREE_WAY - ) -> MergeResult: - """Merge changesets from all completed subplans.""" - # Collect changesets - changesets = [sp.changeset for sp in subplans if sp.changeset] + def __init__( + self, + sandbox_manager: SandboxManager, + validation_service: ValidationService + ): + self._sandbox_manager = sandbox_manager + self._validation_service = validation_service + ``` - # Group changes by file - changes_by_file: dict[str, list[Change]] = {} - for cs in changesets: - for change in cs.changes: - changes_by_file.setdefault(change.path, []).append(change) + - [ ] Commit: "feat(service): add MergeService scaffold" - # Detect and handle conflicts - merged_changes = [] - conflicts = [] + - [ ] **E4.1b** [Jeff] Implement `merge_subplan_results()`: - for path, changes in changes_by_file.items(): - if len(changes) == 1: - merged_changes.append(changes[0]) - else: - # Multiple subplans modified same file - result = self._merge_file_changes(path, changes, strategy) - if result.has_conflict: - conflicts.append(result) - merged_changes.append(result.merged_change) + ```python + def merge_subplan_results( + self, + parent: Plan, + subplans: list[Plan], + strategy: MergeStrategy = MergeStrategy.GIT_THREE_WAY + ) -> MergeResult: + """Merge changesets from all completed subplans.""" + # Collect changesets + changesets = [sp.changeset for sp in subplans if sp.changeset] - return MergeResult( - merged_changeset=ChangeSet(changes=merged_changes), - conflicts=conflicts, - source_subplan_ids=[sp.plan_id for sp in subplans] - ) - ``` - - [ ] Commit: "feat(service): implement merge_subplan_results()" - - [ ] **E4.2** [Jeff] Define merge result types: - - [ ] **E4.2a** [Jeff] Define MergeResult dataclass: - ```python - @dataclass - class MergeResult: - """Result of merging subplan changesets.""" - merged_changeset: ChangeSet - conflicts: list["FileConflict"] - source_subplan_ids: list[str] + # Group changes by file + changes_by_file: dict[str, list[Change]] = {} + for cs in changesets: + for change in cs.changes: + changes_by_file.setdefault(change.path, []).append(change) - @property - def has_conflicts(self) -> bool: - return len(self.conflicts) > 0 + # Detect and handle conflicts + merged_changes = [] + conflicts = [] - @property - def conflict_count(self) -> int: - return len(self.conflicts) + for path, changes in changes_by_file.items(): + if len(changes) == 1: + merged_changes.append(changes[0]) + else: + # Multiple subplans modified same file + result = self._merge_file_changes(path, changes, strategy) + if result.has_conflict: + conflicts.append(result) + merged_changes.append(result.merged_change) - @dataclass - class FileConflict: - """Conflict in a single file.""" - path: str - conflict_regions: list["ConflictRegion"] - subplan_ids: list[str] # Which subplans caused conflict - merged_content_with_markers: str + return MergeResult( + merged_changeset=ChangeSet(changes=merged_changes), + conflicts=conflicts, + source_subplan_ids=[sp.plan_id for sp in subplans] + ) + ``` - @dataclass - class ConflictRegion: - """Region of conflict within a file.""" - start_line: int - end_line: int - ours_content: str # From first subplan - theirs_content: str # From second subplan - ``` - - [ ] Commit: "feat(domain): define merge result types" - - [ ] **E4.3** [Jeff] Implement git-style three-way merge: - - [ ] **E4.3a** [Jeff] Implement `_merge_file_changes()`: - ```python - def _merge_file_changes( - self, - path: str, - changes: list[Change], - strategy: MergeStrategy - ) -> FileMergeResult: - """Merge multiple changes to same file.""" - if strategy == MergeStrategy.GIT_THREE_WAY: - return self._git_three_way_merge(path, changes) - elif strategy == MergeStrategy.SEQUENTIAL_APPLY: - return self._sequential_merge(path, changes) - elif strategy == MergeStrategy.LAST_WINS: - return FileMergeResult( - merged_change=changes[-1], - has_conflict=False - ) - else: - raise ValueError(f"Unknown strategy: {strategy}") - ``` - - [ ] Commit: "feat(service): implement merge strategy dispatch" - - [ ] **E4.3b** [Jeff] Implement `_git_three_way_merge()`: - ```python - def _git_three_way_merge( - self, - path: str, - changes: list[Change] - ) -> FileMergeResult: - """Perform git-style three-way merge.""" - import subprocess - import tempfile + - [ ] Commit: "feat(service): implement merge_subplan_results()" - # Get base (original) content - base_content = self._get_base_content(path) + - [ ] **E4.2** [Jeff] Define merge result types: + - [ ] **E4.2a** [Jeff] Define MergeResult dataclass: - # For now, handle 2 changes; extend for more - if len(changes) != 2: - # Fall back to sequential for >2 changes - return self._sequential_merge(path, changes) + ```python + @dataclass + class MergeResult: + """Result of merging subplan changesets.""" + merged_changeset: ChangeSet + conflicts: list["FileConflict"] + source_subplan_ids: list[str] - ours = changes[0].content or base_content - theirs = changes[1].content or base_content + @property + def has_conflicts(self) -> bool: + return len(self.conflicts) > 0 - # Write to temp files - with tempfile.NamedTemporaryFile(mode='w', suffix='.base', delete=False) as f: - f.write(base_content) - base_path = f.name - with tempfile.NamedTemporaryFile(mode='w', suffix='.ours', delete=False) as f: - f.write(ours) - ours_path = f.name - with tempfile.NamedTemporaryFile(mode='w', suffix='.theirs', delete=False) as f: - f.write(theirs) - theirs_path = f.name + @property + def conflict_count(self) -> int: + return len(self.conflicts) - try: - # Run git merge-file - result = subprocess.run( - ['git', 'merge-file', '-p', ours_path, base_path, theirs_path], - capture_output=True, - text=True - ) + @dataclass + class FileConflict: + """Conflict in a single file.""" + path: str + conflict_regions: list["ConflictRegion"] + subplan_ids: list[str] # Which subplans caused conflict + merged_content_with_markers: str - merged_content = result.stdout - has_conflict = result.returncode != 0 + @dataclass + class ConflictRegion: + """Region of conflict within a file.""" + start_line: int + end_line: int + ours_content: str # From first subplan + theirs_content: str # From second subplan + ``` - # Parse conflict markers if present - conflicts = [] - if has_conflict: - conflicts = self._parse_conflict_markers(merged_content) + - [ ] Commit: "feat(domain): define merge result types" - merged_change = Change( - path=path, - operation=OperationType.MODIFY, - content=merged_content - ) + - [ ] **E4.3** [Jeff] Implement git-style three-way merge: + - [ ] **E4.3a** [Jeff] Implement `_merge_file_changes()`: - return FileMergeResult( - merged_change=merged_change, - has_conflict=has_conflict, - conflict_regions=conflicts - ) - finally: - # Cleanup temp files - for p in [base_path, ours_path, theirs_path]: - os.unlink(p) - ``` - - [ ] Commit: "feat(service): implement git three-way merge" - - [ ] **E4.4** [Luis] Implement sequential merge: - - [ ] **E4.4a** [Luis] Apply changes in order: - ```python - def _sequential_merge( - self, - path: str, - changes: list[Change] - ) -> FileMergeResult: - """Apply changes sequentially in completion order.""" - current_content = self._get_base_content(path) + ```python + def _merge_file_changes( + self, + path: str, + changes: list[Change], + strategy: MergeStrategy + ) -> FileMergeResult: + """Merge multiple changes to same file.""" + if strategy == MergeStrategy.GIT_THREE_WAY: + return self._git_three_way_merge(path, changes) + elif strategy == MergeStrategy.SEQUENTIAL_APPLY: + return self._sequential_merge(path, changes) + elif strategy == MergeStrategy.LAST_WINS: + return FileMergeResult( + merged_change=changes[-1], + has_conflict=False + ) + else: + raise ValueError(f"Unknown strategy: {strategy}") + ``` - for change in changes: - if change.edits: - # Apply edits - current_content = self._apply_edits(current_content, change.edits) - elif change.content: - # Full replacement - current_content = change.content + - [ ] Commit: "feat(service): implement merge strategy dispatch" - return FileMergeResult( - merged_change=Change( - path=path, - operation=OperationType.MODIFY, - content=current_content - ), - has_conflict=False - ) - ``` - - [ ] Commit: "feat(service): implement sequential merge" - - [ ] **E4.5** [Luis] Implement post-merge validation: - - [ ] **E4.5a** [Luis] Validate merged state: - ```python - async def validate_merged_result( - self, - parent: Plan, - merge_result: MergeResult - ) -> ValidationResult: - """Run validation on merged changes.""" - # Apply merged changes to temporary sandbox - temp_sandbox = self._sandbox_manager.create_temp_sandbox( - parent.plan_id, suffix="_merge_validation" - ) + - [ ] **E4.3b** [Jeff] Implement `_git_three_way_merge()`: - try: - # Apply merged changes - for change in merge_result.merged_changeset.changes: - self._apply_change_to_sandbox(temp_sandbox, change) + ```python + def _git_three_way_merge( + self, + path: str, + changes: list[Change] + ) -> FileMergeResult: + """Perform git-style three-way merge.""" + import subprocess + import tempfile - # Run validation - result = await self._validation_service.validate_changeset( - merge_result.merged_changeset, - parent.project - ) + # Get base (original) content + base_content = self._get_base_content(path) - if not result.passed: - logger.warning( - f"Post-merge validation failed: {result.errors}" - ) + # For now, handle 2 changes; extend for more + if len(changes) != 2: + # Fall back to sequential for >2 changes + return self._sequential_merge(path, changes) - return result - finally: - temp_sandbox.cleanup() - ``` - - [ ] Commit: "feat(service): implement post-merge validation" - - [ ] **E4.5b** [Luis] Handle validation failures: - ```python - async def handle_validation_failure( - self, - parent: Plan, - merge_result: MergeResult, - validation_result: ValidationResult - ) -> MergeRecoveryAction: - """Determine recovery action for failed validation.""" - # Options: - # 1. Retry with different merge strategy - # 2. Escalate to user - # 3. Fall back to sequential execution + ours = changes[0].content or base_content + theirs = changes[1].content or base_content - if parent.subplan_config.merge_strategy == MergeStrategy.GIT_THREE_WAY: - # Try sequential as fallback - return MergeRecoveryAction.RETRY_SEQUENTIAL + # Write to temp files + with tempfile.NamedTemporaryFile(mode='w', suffix='.base', delete=False) as f: + f.write(base_content) + base_path = f.name + with tempfile.NamedTemporaryFile(mode='w', suffix='.ours', delete=False) as f: + f.write(ours) + ours_path = f.name + with tempfile.NamedTemporaryFile(mode='w', suffix='.theirs', delete=False) as f: + f.write(theirs) + theirs_path = f.name - # Escalate to user - return MergeRecoveryAction.ESCALATE_TO_USER - ``` - - [ ] Commit: "feat(service): implement validation failure handling" - - [ ] **E4.6** [Rui] Write tests for result merging: - - [ ] **E4.6a** [Rui] Clean merge scenarios: - - [ ] Scenario: Two subplans modifying different files merge cleanly - - [ ] Given subplan A modifies file1.py - - [ ] And subplan B modifies file2.py - - [ ] When merged - - [ ] Then both changes are in merged_changeset - - [ ] And has_conflicts is False - - [ ] Scenario: Same file different lines merges cleanly - - [ ] Given subplan A changes line 10 of file.py - - [ ] And subplan B changes line 50 of file.py - - [ ] When merged with GIT_THREE_WAY - - [ ] Then both changes are preserved - - [ ] And has_conflicts is False - - [ ] Commit: "test(behave): add clean merge scenarios" - - [ ] **E4.6b** [Rui] Conflict scenarios: - - [ ] Scenario: Same lines creates conflict markers - - [ ] Given subplan A changes line 10 to "version A" - - [ ] And subplan B changes line 10 to "version B" - - [ ] When merged with GIT_THREE_WAY - - [ ] Then has_conflicts is True - - [ ] And merged content contains conflict markers - - [ ] Scenario: LAST_WINS strategy has no conflicts - - [ ] Given conflicting changes - - [ ] When merged with LAST_WINS - - [ ] Then has_conflicts is False - - [ ] And later change overwrites earlier - - [ ] Commit: "test(behave): add conflict merge scenarios" - - [ ] **E4.6c** [Rui] Validation scenarios: - - [ ] Scenario: Post-merge validation catches broken code - - [ ] Given merged code with syntax error - - [ ] When post-merge validation runs - - [ ] Then validation fails - - [ ] And recovery action is suggested - - [ ] Commit: "test(behave): add post-merge validation scenarios" + try: + # Run git merge-file + result = subprocess.run( + ['git', 'merge-file', '-p', ours_path, base_path, theirs_path], + capture_output=True, + text=True + ) + + merged_content = result.stdout + has_conflict = result.returncode != 0 + + # Parse conflict markers if present + conflicts = [] + if has_conflict: + conflicts = self._parse_conflict_markers(merged_content) + + merged_change = Change( + path=path, + operation=OperationType.MODIFY, + content=merged_content + ) + + return FileMergeResult( + merged_change=merged_change, + has_conflict=has_conflict, + conflict_regions=conflicts + ) + finally: + # Cleanup temp files + for p in [base_path, ours_path, theirs_path]: + os.unlink(p) + ``` + + - [ ] Commit: "feat(service): implement git three-way merge" + + - [ ] **E4.4** [Luis] Implement sequential merge: + - [ ] **E4.4a** [Luis] Apply changes in order: + + ```python + def _sequential_merge( + self, + path: str, + changes: list[Change] + ) -> FileMergeResult: + """Apply changes sequentially in completion order.""" + current_content = self._get_base_content(path) + + for change in changes: + if change.edits: + # Apply edits + current_content = self._apply_edits(current_content, change.edits) + elif change.content: + # Full replacement + current_content = change.content + + return FileMergeResult( + merged_change=Change( + path=path, + operation=OperationType.MODIFY, + content=current_content + ), + has_conflict=False + ) + ``` + + - [ ] Commit: "feat(service): implement sequential merge" + + - [ ] **E4.5** [Luis] Implement post-merge validation: + - [ ] **E4.5a** [Luis] Validate merged state: + + ```python + async def validate_merged_result( + self, + parent: Plan, + merge_result: MergeResult + ) -> ValidationResult: + """Run validation on merged changes.""" + # Apply merged changes to temporary sandbox + temp_sandbox = self._sandbox_manager.create_temp_sandbox( + parent.plan_id, suffix="_merge_validation" + ) + + try: + # Apply merged changes + for change in merge_result.merged_changeset.changes: + self._apply_change_to_sandbox(temp_sandbox, change) + + # Run validation + result = await self._validation_service.validate_changeset( + merge_result.merged_changeset, + parent.project + ) + + if not result.passed: + logger.warning( + f"Post-merge validation failed: {result.errors}" + ) + + return result + finally: + temp_sandbox.cleanup() + ``` + + - [ ] Commit: "feat(service): implement post-merge validation" + + - [ ] **E4.5b** [Luis] Handle validation failures: + + ```python + async def handle_validation_failure( + self, + parent: Plan, + merge_result: MergeResult, + validation_result: ValidationResult + ) -> MergeRecoveryAction: + """Determine recovery action for failed validation.""" + # Options: + # 1. Retry with different merge strategy + # 2. Escalate to user + # 3. Fall back to sequential execution + + if parent.subplan_config.merge_strategy == MergeStrategy.GIT_THREE_WAY: + # Try sequential as fallback + return MergeRecoveryAction.RETRY_SEQUENTIAL + + # Escalate to user + return MergeRecoveryAction.ESCALATE_TO_USER + ``` + + - [ ] Commit: "feat(service): implement validation failure handling" + + - [ ] **E4.6** [Rui] Write tests for result merging: + - [ ] **E4.6a** [Rui] Clean merge scenarios: + - [ ] Scenario: Two subplans modifying different files merge cleanly + - [ ] Given subplan A modifies file1.py + - [ ] And subplan B modifies file2.py + - [ ] When merged + - [ ] Then both changes are in merged_changeset + - [ ] And has_conflicts is False + - [ ] Scenario: Same file different lines merges cleanly + - [ ] Given subplan A changes line 10 of file.py + - [ ] And subplan B changes line 50 of file.py + - [ ] When merged with GIT_THREE_WAY + - [ ] Then both changes are preserved + - [ ] And has_conflicts is False + - [ ] Commit: "test(behave): add clean merge scenarios" + - [ ] **E4.6b** [Rui] Conflict scenarios: + - [ ] Scenario: Same lines creates conflict markers + - [ ] Given subplan A changes line 10 to "version A" + - [ ] And subplan B changes line 10 to "version B" + - [ ] When merged with GIT_THREE_WAY + - [ ] Then has_conflicts is True + - [ ] And merged content contains conflict markers + - [ ] Scenario: LAST_WINS strategy has no conflicts + - [ ] Given conflicting changes + - [ ] When merged with LAST_WINS + - [ ] Then has_conflicts is False + - [ ] And later change overwrites earlier + - [ ] Commit: "test(behave): add conflict merge scenarios" + - [ ] **E4.6c** [Rui] Validation scenarios: + - [ ] Scenario: Post-merge validation catches broken code + - [ ] Given merged code with syntax error + - [ ] When post-merge validation runs + - [ ] Then validation fails + - [ ] And recovery action is suggested + - [ ] Commit: "test(behave): add post-merge validation scenarios" - [ ] **Stage E5: Multi-Project Plans** (Day 25) **[Hamza]** - **SEQUENTIAL ORDER**: E5.1 (Model extension) → E5.2 (Strategy changes) → E5.3 (Apply per project) → E5.4 (Cross-project deps) → E5.5 (Tests) + **SEQUENTIAL ORDER**: E5.1 (Model extension) → E5.2 (Strategy changes) → E5.3 (Apply per project) → E5.4 (Cross-project deps) → E5.5 (Tests) + - [ ] **E5.1** [Hamza] Extend Plan model for multiple projects: + - [ ] **E5.1a** [Hamza] Add projects field to Plan: - - [ ] **E5.1** [Hamza] Extend Plan model for multiple projects: - - [ ] **E5.1a** [Hamza] Add projects field to Plan: - ```python - # In Plan model - project_ids: list[str] = Field( - default_factory=list, - description="Project IDs this plan targets" - ) + ```python + # In Plan model + project_ids: list[str] = Field( + default_factory=list, + description="Project IDs this plan targets" + ) - @property - def is_multi_project(self) -> bool: - """Check if plan targets multiple projects.""" - return len(self.project_ids) > 1 - ``` - - [ ] Commit: "feat(domain): add multi-project support to Plan" - - [ ] **E5.1b** [Hamza] Add per-project sandbox tracking: - ```python - project_sandboxes: dict[str, str] = Field( - default_factory=dict, - description="Map of project_id to sandbox_id" - ) + @property + def is_multi_project(self) -> bool: + """Check if plan targets multiple projects.""" + return len(self.project_ids) > 1 + ``` - project_apply_status: dict[str, ApplyStatus] = Field( - default_factory=dict, - description="Apply status per project" - ) - ``` - - [ ] Commit: "feat(domain): add per-project sandbox tracking" - - [ ] **E5.2** [Hamza] Update strategy to include project assignments: - - [ ] **E5.2a** [Hamza] Add project field to subplan decisions: - ```python - # Strategy actor can specify project for subplan - context.create_subplan_decision( - action_name="local/code-fix", - description="Fix auth in backend", - target_files=["src/auth/*.py"], - target_project="backend" # New field - ) - ``` - - [ ] Commit: "feat(actor): add project targeting to subplan decisions" - - [ ] **E5.2b** [Hamza] Build cross-project dependency graph: - ```python - def build_cross_project_dag( - self, - decisions: list[Decision] - ) -> CrossProjectDAG: - """Build dependency graph that spans projects.""" - dag = CrossProjectDAG() + - [ ] Commit: "feat(domain): add multi-project support to Plan" - for decision in decisions: - project = self._get_target_project(decision) - dag.add_node(decision.decision_id, project=project) + - [ ] **E5.1b** [Hamza] Add per-project sandbox tracking: - for dep_id in self._get_depends_on(decision): - dep_project = self._get_project_for_decision(dep_id) - dag.add_edge(dep_id, decision.decision_id) + ```python + project_sandboxes: dict[str, str] = Field( + default_factory=dict, + description="Map of project_id to sandbox_id" + ) - # Track cross-project edges - if dep_project != project: - dag.mark_cross_project_edge(dep_id, decision.decision_id) + project_apply_status: dict[str, ApplyStatus] = Field( + default_factory=dict, + description="Apply status per project" + ) + ``` - return dag - ``` - - [ ] Commit: "feat(service): implement cross-project dependency tracking" - - [ ] **E5.3** [Hamza] Apply commits per project: - - [ ] **E5.3a** [Hamza] Implement per-project apply: - ```python - async def apply_multi_project( - self, - plan: Plan - ) -> MultiProjectApplyResult: - """Apply changes to each project separately.""" - results = {} + - [ ] Commit: "feat(domain): add per-project sandbox tracking" - for project_id in plan.project_ids: - try: - result = await self._apply_single_project(plan, project_id) - results[project_id] = ApplyStatus.SUCCESS - except Exception as e: - results[project_id] = ApplyStatus.FAILED - logger.error(f"Apply failed for project {project_id}: {e}") + - [ ] **E5.2** [Hamza] Update strategy to include project assignments: + - [ ] **E5.2a** [Hamza] Add project field to subplan decisions: - # Don't fail others unless they depend on this one - if self._has_dependents(project_id, plan): - logger.warning(f"Dependents of {project_id} will also fail") + ```python + # Strategy actor can specify project for subplan + context.create_subplan_decision( + action_name="local/code-fix", + description="Fix auth in backend", + target_files=["src/auth/*.py"], + target_project="backend" # New field + ) + ``` - return MultiProjectApplyResult( - project_statuses=results, - fully_applied=all(s == ApplyStatus.SUCCESS for s in results.values()) - ) - ``` - - [ ] Commit: "feat(service): implement per-project apply" - - [ ] **E5.3b** [Hamza] Support partial apply: - ```python - async def apply_partial( - self, - plan: Plan, - project_ids: list[str] - ) -> MultiProjectApplyResult: - """Apply only to specified projects.""" - # Validate selected projects are valid - invalid = set(project_ids) - set(plan.project_ids) - if invalid: - raise InvalidProjectError(f"Projects not in plan: {invalid}") + - [ ] Commit: "feat(actor): add project targeting to subplan decisions" - # Check dependency constraints - for pid in project_ids: - deps = self._get_project_dependencies(plan, pid) - missing_deps = deps - set(project_ids) - if missing_deps: - raise DependencyNotAppliedError( - f"Project {pid} depends on unapplied: {missing_deps}" - ) + - [ ] **E5.2b** [Hamza] Build cross-project dependency graph: - return await self._apply_projects(plan, project_ids) - ``` - - [ ] Commit: "feat(service): implement partial project apply" - - [ ] **E5.4** [Hamza] Handle cross-project dependencies: - - [ ] **E5.4a** [Hamza] Enforce dependency order in apply: - ```python - def get_project_apply_order( - self, - plan: Plan - ) -> list[str]: - """Get order to apply projects respecting dependencies.""" - dag = self._build_project_dependency_dag(plan) - return dag.topological_sort() + ```python + def build_cross_project_dag( + self, + decisions: list[Decision] + ) -> CrossProjectDAG: + """Build dependency graph that spans projects.""" + dag = CrossProjectDAG() - async def apply_in_dependency_order( - self, - plan: Plan - ) -> MultiProjectApplyResult: - """Apply projects in correct dependency order.""" - order = self.get_project_apply_order(plan) - results = {} + for decision in decisions: + project = self._get_target_project(decision) + dag.add_node(decision.decision_id, project=project) - for project_id in order: - # Check all dependencies succeeded - deps = self._get_project_dependencies(plan, project_id) - if not all(results.get(d) == ApplyStatus.SUCCESS for d in deps): - results[project_id] = ApplyStatus.SKIPPED_DEPENDENCY_FAILED - continue + for dep_id in self._get_depends_on(decision): + dep_project = self._get_project_for_decision(dep_id) + dag.add_edge(dep_id, decision.decision_id) - # Apply this project - result = await self._apply_single_project(plan, project_id) - results[project_id] = result + # Track cross-project edges + if dep_project != project: + dag.mark_cross_project_edge(dep_id, decision.decision_id) - return MultiProjectApplyResult(project_statuses=results) - ``` - - [ ] Commit: "feat(service): implement dependency-ordered project apply" - - [ ] **E5.5** [Rui] Write end-to-end tests for multi-project: - - [ ] **E5.5a** [Rui] Multi-project scenarios: - - [ ] Scenario: Plan targets two projects with separate sandboxes - - [ ] Given plan with project_ids=[proj1, proj2] - - [ ] When plan executes - - [ ] Then each project has separate sandbox - - [ ] Scenario: Changes applied to each project separately - - [ ] Given completed multi-project plan - - [ ] When apply is called - - [ ] Then proj1 gets its changes - - [ ] And proj2 gets its changes - - [ ] Commit: "test(behave): add multi-project scenarios" - - [ ] **E5.5b** [Rui] Cross-project dependency scenarios: - - [ ] Scenario: Dependent project waits for dependency - - [ ] Given proj2 depends on proj1 changes - - [ ] When apply runs - - [ ] Then proj1 is applied first - - [ ] And proj2 is applied after proj1 succeeds - - [ ] Scenario: Dependent project skipped if dependency fails - - [ ] Given proj2 depends on proj1 - - [ ] And proj1 apply fails - - [ ] Then proj2 is marked SKIPPED_DEPENDENCY_FAILED - - [ ] Commit: "test(behave): add cross-project dependency scenarios" + return dag + ``` + + - [ ] Commit: "feat(service): implement cross-project dependency tracking" + + - [ ] **E5.3** [Hamza] Apply commits per project: + - [ ] **E5.3a** [Hamza] Implement per-project apply: + + ```python + async def apply_multi_project( + self, + plan: Plan + ) -> MultiProjectApplyResult: + """Apply changes to each project separately.""" + results = {} + + for project_id in plan.project_ids: + try: + result = await self._apply_single_project(plan, project_id) + results[project_id] = ApplyStatus.SUCCESS + except Exception as e: + results[project_id] = ApplyStatus.FAILED + logger.error(f"Apply failed for project {project_id}: {e}") + + # Don't fail others unless they depend on this one + if self._has_dependents(project_id, plan): + logger.warning(f"Dependents of {project_id} will also fail") + + return MultiProjectApplyResult( + project_statuses=results, + fully_applied=all(s == ApplyStatus.SUCCESS for s in results.values()) + ) + ``` + + - [ ] Commit: "feat(service): implement per-project apply" + + - [ ] **E5.3b** [Hamza] Support partial apply: + + ```python + async def apply_partial( + self, + plan: Plan, + project_ids: list[str] + ) -> MultiProjectApplyResult: + """Apply only to specified projects.""" + # Validate selected projects are valid + invalid = set(project_ids) - set(plan.project_ids) + if invalid: + raise InvalidProjectError(f"Projects not in plan: {invalid}") + + # Check dependency constraints + for pid in project_ids: + deps = self._get_project_dependencies(plan, pid) + missing_deps = deps - set(project_ids) + if missing_deps: + raise DependencyNotAppliedError( + f"Project {pid} depends on unapplied: {missing_deps}" + ) + + return await self._apply_projects(plan, project_ids) + ``` + + - [ ] Commit: "feat(service): implement partial project apply" + + - [ ] **E5.4** [Hamza] Handle cross-project dependencies: + - [ ] **E5.4a** [Hamza] Enforce dependency order in apply: + + ```python + def get_project_apply_order( + self, + plan: Plan + ) -> list[str]: + """Get order to apply projects respecting dependencies.""" + dag = self._build_project_dependency_dag(plan) + return dag.topological_sort() + + async def apply_in_dependency_order( + self, + plan: Plan + ) -> MultiProjectApplyResult: + """Apply projects in correct dependency order.""" + order = self.get_project_apply_order(plan) + results = {} + + for project_id in order: + # Check all dependencies succeeded + deps = self._get_project_dependencies(plan, project_id) + if not all(results.get(d) == ApplyStatus.SUCCESS for d in deps): + results[project_id] = ApplyStatus.SKIPPED_DEPENDENCY_FAILED + continue + + # Apply this project + result = await self._apply_single_project(plan, project_id) + results[project_id] = result + + return MultiProjectApplyResult(project_statuses=results) + ``` + + - [ ] Commit: "feat(service): implement dependency-ordered project apply" + + - [ ] **E5.5** [Rui] Write end-to-end tests for multi-project: + - [ ] **E5.5a** [Rui] Multi-project scenarios: + - [ ] Scenario: Plan targets two projects with separate sandboxes + - [ ] Given plan with project_ids=[proj1, proj2] + - [ ] When plan executes + - [ ] Then each project has separate sandbox + - [ ] Scenario: Changes applied to each project separately + - [ ] Given completed multi-project plan + - [ ] When apply is called + - [ ] Then proj1 gets its changes + - [ ] And proj2 gets its changes + - [ ] Commit: "test(behave): add multi-project scenarios" + - [ ] **E5.5b** [Rui] Cross-project dependency scenarios: + - [ ] Scenario: Dependent project waits for dependency + - [ ] Given proj2 depends on proj1 changes + - [ ] When apply runs + - [ ] Then proj1 is applied first + - [ ] And proj2 is applied after proj1 succeeds + - [ ] Scenario: Dependent project skipped if dependency fails + - [ ] Given proj2 depends on proj1 + - [ ] And proj1 apply fails + - [ ] Then proj2 is marked SKIPPED_DEPENDENCY_FAILED + - [ ] Commit: "test(behave): add cross-project dependency scenarios" **M5 SUCCESS CRITERIA** (Day 25): + - [ ] Plans can spawn subplans from SUBPLAN_SPAWN decisions - [ ] Sequential subplan execution works (execute in order) - [ ] Parallel subplan execution works (concurrent with max_parallel limit) @@ -8015,34 +8509,34 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation These stubs ensure the client architecture supports future server connectivity without implementing it: - [ ] **Stage F0: Server Client Interface Stubs** (Day 28-29) **[Luis - Required]** - - [ ] **F0.1** [Luis] Create `src/cleveragents/interfaces/server_client.py` with protocol stubs: - - [ ] `class ServerClient(Protocol):` with all method signatures for client-to-server communication - - [ ] `async def connect(server_url: str) -> None: raise NotImplementedError("Server connectivity not yet implemented")` - - [ ] `async def disconnect() -> None: raise NotImplementedError(...)` - - [ ] `async def sync_action(action: Action) -> Action: raise NotImplementedError(...)` - - [ ] `async def request_remote_execution(plan_id: str) -> str: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add ServerClient protocol stub" - - [ ] **F0.2** [Luis] Create `src/cleveragents/interfaces/remote_execution_client.py`: - - [ ] `class RemoteExecutionClient(Protocol):` - protocol for requesting remote plan execution from server - - [ ] `async def submit_plan(plan_id: str, server_url: str) -> str: raise NotImplementedError(...)` - - [ ] `async def poll_status(execution_id: str) -> ExecutionStatus: raise NotImplementedError(...)` - - [ ] `async def fetch_results(execution_id: str) -> ExecutionResult: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add RemoteExecutionClient protocol stub" - - [ ] **F0.3** [Luis] Create `src/cleveragents/interfaces/auth_client.py`: - - [ ] `class AuthClient(Protocol):` - protocol for client authentication with server - - [ ] `async def authenticate(credentials: Credentials) -> AuthToken: raise NotImplementedError(...)` - - [ ] `async def validate_token(token: str) -> TokenValidation: raise NotImplementedError(...)` - - [ ] `async def refresh_token(token: str) -> AuthToken: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add AuthClient protocol stub" - - [ ] **F0.4** [Luis] Add `agents [--data-dir PATH] [--config-path PATH] connect` CLI command as stub: - - [ ] Add to `src/cleveragents/cli/commands/server_client.py` - - [ ] Command signature: `@click.command("connect") @click.argument("server_url")` - - [ ] Implementation: `click.echo("Server connectivity not yet implemented. Coming soon!")`; `raise SystemExit(1)` - - [ ] Commit: "feat(cli): add connect command stub" - - [ ] **F0.5** [Rui] Write minimal tests verifying stubs raise NotImplementedError: - - [ ] Test: Calling ServerClient methods raises NotImplementedError - - [ ] Test: `agents [--data-dir PATH] [--config-path PATH] connect` displays "not implemented" message and exits - - [ ] Commit: "test(behave): add server client stub tests" + - [ ] **F0.1** [Luis] Create `src/cleveragents/interfaces/server_client.py` with protocol stubs: + - [ ] `class ServerClient(Protocol):` with all method signatures for client-to-server communication + - [ ] `async def connect(server_url: str) -> None: raise NotImplementedError("Server connectivity not yet implemented")` + - [ ] `async def disconnect() -> None: raise NotImplementedError(...)` + - [ ] `async def sync_action(action: Action) -> Action: raise NotImplementedError(...)` + - [ ] `async def request_remote_execution(plan_id: str) -> str: raise NotImplementedError(...)` + - [ ] Commit: "feat(interfaces): add ServerClient protocol stub" + - [ ] **F0.2** [Luis] Create `src/cleveragents/interfaces/remote_execution_client.py`: + - [ ] `class RemoteExecutionClient(Protocol):` - protocol for requesting remote plan execution from server + - [ ] `async def submit_plan(plan_id: str, server_url: str) -> str: raise NotImplementedError(...)` + - [ ] `async def poll_status(execution_id: str) -> ExecutionStatus: raise NotImplementedError(...)` + - [ ] `async def fetch_results(execution_id: str) -> ExecutionResult: raise NotImplementedError(...)` + - [ ] Commit: "feat(interfaces): add RemoteExecutionClient protocol stub" + - [ ] **F0.3** [Luis] Create `src/cleveragents/interfaces/auth_client.py`: + - [ ] `class AuthClient(Protocol):` - protocol for client authentication with server + - [ ] `async def authenticate(credentials: Credentials) -> AuthToken: raise NotImplementedError(...)` + - [ ] `async def validate_token(token: str) -> TokenValidation: raise NotImplementedError(...)` + - [ ] `async def refresh_token(token: str) -> AuthToken: raise NotImplementedError(...)` + - [ ] Commit: "feat(interfaces): add AuthClient protocol stub" + - [ ] **F0.4** [Luis] Add `agents [--data-dir PATH] [--config-path PATH] connect` CLI command as stub: + - [ ] Add to `src/cleveragents/cli/commands/server_client.py` + - [ ] Command signature: `@click.command("connect") @click.argument("server_url")` + - [ ] Implementation: `click.echo("Server connectivity not yet implemented. Coming soon!")`; `raise SystemExit(1)` + - [ ] Commit: "feat(cli): add connect command stub" + - [ ] **F0.5** [Rui] Write minimal tests verifying stubs raise NotImplementedError: + - [ ] Test: Calling ServerClient methods raises NotImplementedError + - [ ] Test: `agents [--data-dir PATH] [--config-path PATH] connect` displays "not implemented" message and exits + - [ ] Commit: "test(behave): add server client stub tests" --- @@ -8051,30 +8545,31 @@ These stubs ensure the client architecture supports future server connectivity w > **These stages are OUT OF SCOPE for the 30-day timeline.** Do not begin work on them until after Day 30 milestone is achieved. Note: These stages implement **client-side** connectivity; the server is a separate project. - [ ] **Stage F1: Server Client Infrastructure** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F1.1** [Luis] Create HTTP client in `src/cleveragents/infrastructure/server_client.py` - - [ ] **F1.2** [Luis] Connection health check and version negotiation - - [ ] **F1.3** [Luis] API client code generation from server OpenAPI spec - - [ ] **F1.4** [Rui] Client connection tests (with mock server) + - [ ] **F1.1** [Luis] Create HTTP client in `src/cleveragents/infrastructure/server_client.py` + - [ ] **F1.2** [Luis] Connection health check and version negotiation + - [ ] **F1.3** [Luis] API client code generation from server OpenAPI spec + - [ ] **F1.4** [Rui] Client connection tests (with mock server) - [ ] **Stage F2: Plan Sync Client** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F2.1** [Luis] Sync local actions to server - - [ ] **F2.2** [Luis] Request plan creation on server - - [ ] **F2.3** [Luis] Request plan execution on server - - [ ] **F2.4** [Luis] Request `agents [--data-dir PATH] [--config-path PATH] plan apply` on server - - [ ] **F2.5** [Luis] Fetch plan status from server - - [ ] **F2.6** [Rui] Client-side API integration tests (with mock server) + - [ ] **F2.1** [Luis] Sync local actions to server + - [ ] **F2.2** [Luis] Request plan creation on server + - [ ] **F2.3** [Luis] Request plan execution on server + - [ ] **F2.4** [Luis] Request `agents [--data-dir PATH] [--config-path PATH] plan apply` on server + - [ ] **F2.5** [Luis] Fetch plan status from server + - [ ] **F2.6** [Rui] Client-side API integration tests (with mock server) - [ ] **Stage F3: WebSocket Client** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F3.1** [Luis] WebSocket client for receiving plan updates from server - - [ ] **F3.2** [Luis] Handle phase transitions, node completions from server - - [ ] **F3.3** [Rui] WebSocket client tests (with mock server) + - [ ] **F3.1** [Luis] WebSocket client for receiving plan updates from server + - [ ] **F3.2** [Luis] Handle phase transitions, node completions from server + - [ ] **F3.3** [Rui] WebSocket client tests (with mock server) - [ ] **Stage F4: Remote Project Support** (Post-Day 30) **[Hamza]** **[DEFERRED]** - - [ ] **F4.1** [Hamza] Client can specify remote resources for server execution - - [ ] **F4.2** [Hamza] Client sends execution requests to server for remote resources - - [ ] **F4.3** [Rui] End-to-end tests for remote execution (with mock server) + - [ ] **F4.1** [Hamza] Client can specify remote resources for server execution + - [ ] **F4.2** [Hamza] Client sends execution requests to server for remote resources + - [ ] **F4.3** [Rui] End-to-end tests for remote execution (with mock server) **M7 SUCCESS CRITERIA** (Post-Day 30): + - [ ] `agents [--data-dir PATH] [--config-path PATH] connect ` establishes connection to an external server - [ ] Plans can be synced and executed on a remote server - [ ] Real-time updates received via WebSocket from server @@ -8083,6 +8578,7 @@ These stubs ensure the client architecture supports future server connectivity w **--- MERGE POINT 2: Day 30 - Large Project Autonomy Target (LOCAL MODE ONLY) ---** By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is deferred): + - [ ] Handle projects with 10,000+ files using hierarchical decomposition - [ ] Port source code from one language to another autonomously - [ ] Use hierarchical subplans (5+ levels deep) for large tasks @@ -8098,157 +8594,158 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Target: Milestone M7 (+35 days)** - [ ] **Stage G1: Automation Levels Enhancement** (Day 31-32) **[Luis]** - - [ ] **G1.1** [Luis] Full manual mode with decision prompts: - - [ ] Every decision point pauses for human input - - [ ] Display context, alternatives considered, recommendation - - [ ] Accept explicit choice or custom guidance - - [ ] Record user decisions in decision tree - - [ ] **G1.2** [Luis] Review-before-apply with diff display: - - [ ] AI makes all decisions autonomously during Strategize - - [ ] Execution completes in sandbox - - [ ] Human reviews complete diff before apply: - - [ ] Show changed files summary - - [ ] Show full unified diff with syntax highlighting - - [ ] Show risk warnings (auth code, migrations, etc.) - - [ ] User can approve, reject, or correct specific decisions - - [ ] **G1.3** [Luis] Full automation with confidence escalation: - - [ ] AI makes all decisions autonomously - - [ ] Execution proceeds through apply without pause - - [ ] Human notified of completion - - [ ] Rollback available if issues detected post-apply - - [ ] EVEN in full automation, critical decisions escalate: - - [ ] If confidence below threshold, request human guidance - - [ ] If touching critical files (defined by project), escalate - - [ ] If cost exceeds budget, escalate - - [ ] **G1.4** [Luis] Progressive trust building (track success rates): - - [ ] Track decision success rates per decision type - - [ ] Track codebase familiarity scores per project - - [ ] Confidence increases with successful history - - [ ] Allow automatic upgrade: manual -> review -> full - - [ ] After N successful plans in manual, suggest review mode - - [ ] After N successful plans in review, suggest full mode - - [ ] **G1.5** [Luis] Implement `AutonomyController` class: - - [ ] Method `assess_decision_confidence(decision, context) -> float` - - [ ] Method `should_escalate(decision, confidence, automation_level) -> bool` - - [ ] Method `get_historical_success(decision_type) -> float` - - [ ] Method `get_familiarity_score(project) -> float` - - [ ] **G1.6** [Rui] Tests for each automation level: - - [ ] Scenario: Manual mode pauses at each decision - - [ ] Scenario: Review-before-apply shows diff before apply - - [ ] Scenario: Full automation completes without pause - - [ ] Scenario: Low confidence in full automation escalates - - [ ] Scenario: Progressive trust upgrade suggestion shown + - [ ] **G1.1** [Luis] Full manual mode with decision prompts: + - [ ] Every decision point pauses for human input + - [ ] Display context, alternatives considered, recommendation + - [ ] Accept explicit choice or custom guidance + - [ ] Record user decisions in decision tree + - [ ] **G1.2** [Luis] Review-before-apply with diff display: + - [ ] AI makes all decisions autonomously during Strategize + - [ ] Execution completes in sandbox + - [ ] Human reviews complete diff before apply: + - [ ] Show changed files summary + - [ ] Show full unified diff with syntax highlighting + - [ ] Show risk warnings (auth code, migrations, etc.) + - [ ] User can approve, reject, or correct specific decisions + - [ ] **G1.3** [Luis] Full automation with confidence escalation: + - [ ] AI makes all decisions autonomously + - [ ] Execution proceeds through apply without pause + - [ ] Human notified of completion + - [ ] Rollback available if issues detected post-apply + - [ ] EVEN in full automation, critical decisions escalate: + - [ ] If confidence below threshold, request human guidance + - [ ] If touching critical files (defined by project), escalate + - [ ] If cost exceeds budget, escalate + - [ ] **G1.4** [Luis] Progressive trust building (track success rates): + - [ ] Track decision success rates per decision type + - [ ] Track codebase familiarity scores per project + - [ ] Confidence increases with successful history + - [ ] Allow automatic upgrade: manual -> review -> full + - [ ] After N successful plans in manual, suggest review mode + - [ ] After N successful plans in review, suggest full mode + - [ ] **G1.5** [Luis] Implement `AutonomyController` class: + - [ ] Method `assess_decision_confidence(decision, context) -> float` + - [ ] Method `should_escalate(decision, confidence, automation_level) -> bool` + - [ ] Method `get_historical_success(decision_type) -> float` + - [ ] Method `get_familiarity_score(project) -> float` + - [ ] **G1.6** [Rui] Tests for each automation level: + - [ ] Scenario: Manual mode pauses at each decision + - [ ] Scenario: Review-before-apply shows diff before apply + - [ ] Scenario: Full automation completes without pause + - [ ] Scenario: Low confidence in full automation escalates + - [ ] Scenario: Progressive trust upgrade suggestion shown - [ ] **Stage G2: Checkpointing & Rollback** (Day 32-33) **[Luis]** - - [ ] **G2.1** [Luis] Skill-level checkpoint declarations - - [ ] **G2.2** [Luis] Plan-level rollback policy - - [ ] **G2.3** [Luis] Rollback to checkpoint command - - [ ] **G2.4** [Rui] Checkpoint/rollback tests + - [ ] **G2.1** [Luis] Skill-level checkpoint declarations + - [ ] **G2.2** [Luis] Plan-level rollback policy + - [ ] **G2.3** [Luis] Rollback to checkpoint command + - [ ] **G2.4** [Rui] Checkpoint/rollback tests - [ ] **Stage G3: Semantic Validation Framework** (Day 33-34) **[Luis]** - - [ ] **G3.1** [Luis] Decision-time validation in Strategize: - - [ ] Every decision includes semantic validation - - [ ] Record `validation_performed` list on each decision - - [ ] Validate chosen option against alternatives - - [ ] Check for breaking changes, compatibility issues - - [ ] **G3.2** [Luis] Execution-time semantic guards: - - [ ] Actor configs can include validation nodes - - [ ] `validate_api_compatibility` - check for breaking API changes - - [ ] `validate_type_safety` - check types are preserved - - [ ] `auto_migrate` - generate migration plan for breaking changes - - [ ] Guards can auto-fix simple issues or escalate complex ones - - [ ] **G3.3** [Luis] Invariant enforcement system: - - [ ] User-defined invariants per project (see Section 15) - - [ ] Check invariants at each major step - - [ ] Fail execution on violation (if severity=error) - - [ ] Record invariant check results in plan metadata - - [ ] **G3.4** [Luis] Error pattern database: - - [ ] Store historical failures with context - - [ ] Identify patterns: "Async conversion in X module often causes Y" - - [ ] Before execution, check for known patterns - - [ ] Add preventive checks based on patterns - - [ ] Learn from successful corrections - - [ ] **G3.5** [Luis] Implement `SemanticValidationService`: - - [ ] Method `validate_decision(decision) -> ValidationResult` - - [ ] Method `check_invariants(project, changes) -> list[InvariantResult]` - - [ ] Method `check_error_patterns(context) -> list[PatternMatch]` - - [ ] Method `suggest_preventive_checks(pattern) -> list[Check]` - - [ ] **G3.6** [Rui] Semantic validation tests: - - [ ] Scenario: Decision with breaking API change is flagged - - [ ] Scenario: Invariant violation detected during execution - - [ ] Scenario: Known error pattern triggers preventive check + - [ ] **G3.1** [Luis] Decision-time validation in Strategize: + - [ ] Every decision includes semantic validation + - [ ] Record `validation_performed` list on each decision + - [ ] Validate chosen option against alternatives + - [ ] Check for breaking changes, compatibility issues + - [ ] **G3.2** [Luis] Execution-time semantic guards: + - [ ] Actor configs can include validation nodes + - [ ] `validate_api_compatibility` - check for breaking API changes + - [ ] `validate_type_safety` - check types are preserved + - [ ] `auto_migrate` - generate migration plan for breaking changes + - [ ] Guards can auto-fix simple issues or escalate complex ones + - [ ] **G3.3** [Luis] Invariant enforcement system: + - [ ] User-defined invariants per project (see Section 15) + - [ ] Check invariants at each major step + - [ ] Fail execution on violation (if severity=error) + - [ ] Record invariant check results in plan metadata + - [ ] **G3.4** [Luis] Error pattern database: + - [ ] Store historical failures with context + - [ ] Identify patterns: "Async conversion in X module often causes Y" + - [ ] Before execution, check for known patterns + - [ ] Add preventive checks based on patterns + - [ ] Learn from successful corrections + - [ ] **G3.5** [Luis] Implement `SemanticValidationService`: + - [ ] Method `validate_decision(decision) -> ValidationResult` + - [ ] Method `check_invariants(project, changes) -> list[InvariantResult]` + - [ ] Method `check_error_patterns(context) -> list[PatternMatch]` + - [ ] Method `suggest_preventive_checks(pattern) -> list[Check]` + - [ ] **G3.6** [Rui] Semantic validation tests: + - [ ] Scenario: Decision with breaking API change is flagged + - [ ] Scenario: Invariant violation detected during execution + - [ ] Scenario: Known error pattern triggers preventive check - [ ] **Stage G4: Context Tiers** (Day 34-35) **[Hamza]** - - [ ] **G4.1** [Hamza] Hot context management (10-20 files): - - [ ] Track files currently in LLM context window - - [ ] Implement LRU eviction when context limit reached - - [ ] Prioritize files based on current task relevance - - [ ] Support explicit pinning of critical files - - [ ] **G4.2** [Hamza] Warm context with vector search: - - [ ] Recent decisions and their contexts from current plan tree - - [ ] Indexed embeddings from project files - - [ ] Vector search results for quick retrieval - - [ ] Decision chain that led to current work - - [ ] **G4.3** [Hamza] Cold context for historical decisions: - - [ ] Historical decisions from past plans on this codebase - - [ ] Past refactoring patterns ("last time we did X...") - - [ ] Cross-project learnings (if enabled) - - [ ] Queryable but not in active memory - - [ ] **G4.4** [Hamza] Per-actor context views: - - [ ] Implement `ActorContextView` service - - [ ] Strategist view: architecture docs, READMEs, module boundaries, dependency graphs - - [ ] Executor view: precise code sections for edits, test files - - [ ] Reviewer view: diffs, tests, risk zones, style guides - - [ ] Each actor gets filtered view based on role - - [ ] **G4.5** [Hamza] Implement promotion/demotion algorithms: - - [ ] Analyze current query/task - - [ ] Promote relevant data upward (cold -> warm -> hot) - - [ ] Demote stale data out of hot to keep prompts tight - - [ ] Preserve complete context snapshots for every decision - - [ ] **G4.6** [Rui] Context tier tests: - - [ ] Scenario: Hot context respects file limit - - [ ] Scenario: Warm context includes recent decisions - - [ ] Scenario: Actor receives role-appropriate context view - - [ ] Scenario: Promotion moves relevant cold data to warm + - [ ] **G4.1** [Hamza] Hot context management (10-20 files): + - [ ] Track files currently in LLM context window + - [ ] Implement LRU eviction when context limit reached + - [ ] Prioritize files based on current task relevance + - [ ] Support explicit pinning of critical files + - [ ] **G4.2** [Hamza] Warm context with vector search: + - [ ] Recent decisions and their contexts from current plan tree + - [ ] Indexed embeddings from project files + - [ ] Vector search results for quick retrieval + - [ ] Decision chain that led to current work + - [ ] **G4.3** [Hamza] Cold context for historical decisions: + - [ ] Historical decisions from past plans on this codebase + - [ ] Past refactoring patterns ("last time we did X...") + - [ ] Cross-project learnings (if enabled) + - [ ] Queryable but not in active memory + - [ ] **G4.4** [Hamza] Per-actor context views: + - [ ] Implement `ActorContextView` service + - [ ] Strategist view: architecture docs, READMEs, module boundaries, dependency graphs + - [ ] Executor view: precise code sections for edits, test files + - [ ] Reviewer view: diffs, tests, risk zones, style guides + - [ ] Each actor gets filtered view based on role + - [ ] **G4.5** [Hamza] Implement promotion/demotion algorithms: + - [ ] Analyze current query/task + - [ ] Promote relevant data upward (cold -> warm -> hot) + - [ ] Demote stale data out of hot to keep prompts tight + - [ ] Preserve complete context snapshots for every decision + - [ ] **G4.6** [Rui] Context tier tests: + - [ ] Scenario: Hot context respects file limit + - [ ] Scenario: Warm context includes recent decisions + - [ ] Scenario: Actor receives role-appropriate context view + - [ ] Scenario: Promotion moves relevant cold data to warm - [ ] **Stage G5: Cost & Risk Estimation** (Day 35) **[Hamza]** - - [ ] **G5.1** [Hamza] Estimation actor implementation: - - [ ] Create optional `estimation_actor` role in action model - - [ ] Estimation actor runs after Strategize completes, before Execute - - [ ] Input: strategy output, project context - - [ ] Output: cost estimate, risk assessment, time estimate - - [ ] **G5.2** [Hamza] Token/cost estimation: - - [ ] Analyze strategy to estimate number of LLM calls - - [ ] Estimate tokens per call based on context size - - [ ] Map model costs to get dollar estimate - - [ ] Estimate number of steps/subplans - - [ ] Provide confidence interval (min/expected/max) - - [ ] **G5.3** [Hamza] Risk assessment: - - [ ] Analyze strategy for risky operations: - - [ ] Touching auth/security code - - [ ] Database migrations - - [ ] Public API changes - - [ ] Infrastructure changes - - [ ] Calculate risk score (low/medium/high) - - [ ] Identify specific risk factors - - [ ] Estimate likelihood of rollback needed - - [ ] **G5.4** [Hamza] Display estimation before execute: - - [ ] Show estimated cost before user confirms execute - - [ ] Support `--yes` to bypass confirmation in automation contexts - - [ ] Show risk assessment summary - - [ ] Allow user to abort if cost too high - - [ ] **G5.5** [Rui] Estimation tests: - - [ ] Scenario: Estimation actor produces cost estimate - - [ ] Scenario: High-risk strategy flagged appropriately - - [ ] Scenario: User can abort based on estimate + - [ ] **G5.1** [Hamza] Estimation actor implementation: + - [ ] Create optional `estimation_actor` role in action model + - [ ] Estimation actor runs after Strategize completes, before Execute + - [ ] Input: strategy output, project context + - [ ] Output: cost estimate, risk assessment, time estimate + - [ ] **G5.2** [Hamza] Token/cost estimation: + - [ ] Analyze strategy to estimate number of LLM calls + - [ ] Estimate tokens per call based on context size + - [ ] Map model costs to get dollar estimate + - [ ] Estimate number of steps/subplans + - [ ] Provide confidence interval (min/expected/max) + - [ ] **G5.3** [Hamza] Risk assessment: + - [ ] Analyze strategy for risky operations: + - [ ] Touching auth/security code + - [ ] Database migrations + - [ ] Public API changes + - [ ] Infrastructure changes + - [ ] Calculate risk score (low/medium/high) + - [ ] Identify specific risk factors + - [ ] Estimate likelihood of rollback needed + - [ ] **G5.4** [Hamza] Display estimation before execute: + - [ ] Show estimated cost before user confirms execute + - [ ] Support `--yes` to bypass confirmation in automation contexts + - [ ] Show risk assessment summary + - [ ] Allow user to abort if cost too high + - [ ] **G5.5** [Rui] Estimation tests: + - [ ] Scenario: Estimation actor produces cost estimate + - [ ] Scenario: High-risk strategy flagged appropriately + - [ ] Scenario: User can abort based on estimate - [ ] **Stage G6: Full CLI Polish** (Day 35) **[All]** - - [ ] **G6.1** [All] Consistent help text - - [ ] **G6.2** [All] Rich terminal output - - [ ] **G6.3** [All] Progress indicators - - [ ] **G6.4** [All] Error messages with recovery suggestions + - [ ] **G6.1** [All] Consistent help text + - [ ] **G6.2** [All] Rich terminal output + - [ ] **G6.3** [All] Progress indicators + - [ ] **G6.4** [All] Error messages with recovery suggestions **M7 SUCCESS CRITERIA**: + - [ ] All spec features implemented - [ ] Full test coverage (>85%) - [ ] Documentation complete @@ -8261,56 +8758,56 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Note**: Quality automation setup has been moved to Section 0 (Days 1-3) as a prerequisite. - [ ] **Stage 10A: Async Infrastructure** (Days 10-12) **[Luis]** - - [ ] Code: Implement async patterns per ADR-002 - - [ ] **10A.1** [Luis] Implement async command execution - - [X] **10A.2** Implement the 33 retry patterns with tenacity (COMPLETED 2025-11-17) - - [X] **10A.3** Add circuit breaker for failures (COMPLETED 2025-11-17) - - [ ] **10A.4** [Luis] Add background workers (convert 7 concurrency patterns to asyncio tasks) - - [ ] **10A.5** [Luis] Integrate retry patterns into new services + - [ ] Code: Implement async patterns per ADR-002 + - [ ] **10A.1** [Luis] Implement async command execution + - [x] **10A.2** Implement the 33 retry patterns with tenacity (COMPLETED 2025-11-17) + - [x] **10A.3** Add circuit breaker for failures (COMPLETED 2025-11-17) + - [ ] **10A.4** [Luis] Add background workers (convert 7 concurrency patterns to asyncio tasks) + - [ ] **10A.5** [Luis] Integrate retry patterns into new services - [ ] **Stage 10B: Selective Quality Review** (Days 4-8) **[Brent]** - - [ ] Focus Areas: Only review critical items - - [ ] **10B.1** [Brent] Review architectural decisions in PRs: - - [ ] Service layer design choices - - [ ] Database schema decisions - - [ ] API contract definitions - - [ ] Skip: formatting, simple CRUD, test files - - [ ] **10B.2** [Brent] Review complex algorithms: - - [ ] Decision tree traversal logic - - [ ] Merge conflict resolution - - [ ] Dependency closure computation - - [ ] Skip: straightforward implementations - - [ ] **10B.3** [Brent] Review security-sensitive code: - - [ ] Authentication/authorization - - [ ] Input validation - - [ ] Sandbox boundaries - - [ ] Skip: code already scanned by bandit - - [X] **10B.4** [Brent] Monitor automated quality metrics - COMPLETED 2026-02-10: - - [X] Daily check of CI/CD dashboard - baseline established (all green) - - [X] Weekly quality report generation - quality baseline documented in Phase 2 Notes - - [X] Escalate only if metrics drop - no issues found, all metrics at healthy levels - - [X] Fix – Missing langchain-anthropic dependency (caused all 105 feature files to fail import) - - [X] Fix – Rich table wrapping in plan_lifecycle_cli_coverage.feature (console width patch) + - [ ] Focus Areas: Only review critical items + - [ ] **10B.1** [Brent] Review architectural decisions in PRs: + - [ ] Service layer design choices + - [ ] Database schema decisions + - [ ] API contract definitions + - [ ] Skip: formatting, simple CRUD, test files + - [ ] **10B.2** [Brent] Review complex algorithms: + - [ ] Decision tree traversal logic + - [ ] Merge conflict resolution + - [ ] Dependency closure computation + - [ ] Skip: straightforward implementations + - [ ] **10B.3** [Brent] Review security-sensitive code: + - [ ] Authentication/authorization + - [ ] Input validation + - [ ] Sandbox boundaries + - [ ] Skip: code already scanned by bandit + - [x] **10B.4** [Brent] Monitor automated quality metrics - COMPLETED 2026-02-10: + - [x] Daily check of CI/CD dashboard - baseline established (all green) + - [x] Weekly quality report generation - quality baseline documented in Phase 2 Notes + - [x] Escalate only if metrics drop - no issues found, all metrics at healthy levels + - [x] Fix – Missing langchain-anthropic dependency (caused all 105 feature files to fail import) + - [x] Fix – Rich table wrapping in plan_lifecycle_cli_coverage.feature (console width patch) - [ ] **Stage 10C: Validation Testing Support** (Days 9-30) **[Brent + Luis]** - - [ ] High-impact testing work - - [x] **10C.1** [Brent] Create edge case test scenarios: - - [x] Concurrent plan execution edge cases - - [x] Resource conflict scenarios - - [x] Validation failure chains - - [x] Rollback edge cases - - [ ] **10C.2** [Brent + Luis] Implement semantic validation tests: - - [ ] API compatibility validation - - [ ] Business invariant preservation - - [ ] Cross-resource consistency - - [ ] **10C.3** [Brent] Performance testing for scale: - - [ ] 10K+ file repository handling - - [ ] Memory usage profiling - - [ ] Context tier performance - - [x] **10C.4** [Brent] Create validation test fixtures: - - [x] Invalid code samples - - [x] Edge case project structures - - [x] Malformed input data + - [ ] High-impact testing work + - [x] **10C.1** [Brent] Create edge case test scenarios: + - [x] Concurrent plan execution edge cases + - [x] Resource conflict scenarios + - [x] Validation failure chains + - [x] Rollback edge cases + - [ ] **10C.2** [Brent + Luis] Implement semantic validation tests: + - [ ] API compatibility validation + - [ ] Business invariant preservation + - [ ] Cross-resource consistency + - [ ] **10C.3** [Brent] Performance testing for scale: + - [ ] 10K+ file repository handling + - [ ] Memory usage profiling + - [ ] Context tier performance + - [x] **10C.4** [Brent] Create validation test fixtures: + - [x] Invalid code samples + - [x] Edge case project structures + - [x] Malformed input data --- @@ -8320,456 +8817,506 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Note**: With automated quality gates in place (Section 0), Brent only reviews security changes after automated scanning. -- [ ] **Stage SEC1: Remove eval() Vulnerability** (Day 1-2) **[Luis - CRITICAL]** - - [ ] Test with code containing unused imports - - [ ] Commit: "feat(qa): add ruff linting to pre-commit" +- [ ] **Stage SEC1: Remove eval() Vulnerability** (Day 1-2) **[Luis - CRITICAL]** - [ ] Test with code containing unused imports - [ ] Commit: "feat(qa): add ruff linting to pre-commit" - [ ] **0.1e** [Brent] Add pyright type checking hook: - ```yaml - - repo: local - hooks: - - id: pyright - name: Type check with pyright - entry: pyright - language: system - types: [python] - require_serial: true - ``` - - [ ] Ensure pyright is installed via dev dependencies - - [ ] Test with code containing type errors - - [ ] Commit: "feat(qa): add pyright to pre-commit" + + ```yaml + - repo: local + hooks: + - id: pyright + name: Type check with pyright + entry: pyright + language: system + types: [python] + require_serial: true + ``` + + - [ ] Ensure pyright is installed via dev dependencies + - [ ] Test with code containing type errors + - [ ] Commit: "feat(qa): add pyright to pre-commit" + - [ ] **0.1f** [Brent] Add security scanning with bandit: - - [ ] Add `bandit[toml]>=1.7.5` to dev dependencies - - [ ] Create `pyproject.toml` section for bandit config: - ```toml - [tool.bandit] - exclude_dirs = ["tests", "features", "benchmarks"] - skips = ["B101"] # Skip assert_used test in test files - ``` - - [ ] Add bandit hook: - ```yaml - - repo: https://github.com/PyCQA/bandit - rev: '1.7.5' - hooks: - - id: bandit - args: ['-c', 'pyproject.toml'] - additional_dependencies: ["bandit[toml]"] - ``` - - [ ] Commit: "feat(qa): add bandit security scanning" + - [ ] Add `bandit[toml]>=1.7.5` to dev dependencies + - [ ] Create `pyproject.toml` section for bandit config: + ```toml + [tool.bandit] + exclude_dirs = ["tests", "features", "benchmarks"] + skips = ["B101"] # Skip assert_used test in test files + ``` + - [ ] Add bandit hook: + ```yaml + - repo: https://github.com/PyCQA/bandit + rev: "1.7.5" + hooks: + - id: bandit + args: ["-c", "pyproject.toml"] + additional_dependencies: ["bandit[toml]"] + ``` + - [ ] Commit: "feat(qa): add bandit security scanning" - [ ] **0.1g** [Brent] Add vulture for dead code detection: - - [ ] Add `vulture>=2.10` to dev dependencies - - [ ] Create `vulture_whitelist.py` for false positives - - [ ] Add vulture hook: - ```yaml - - repo: local - hooks: - - id: vulture - name: Find dead code with vulture - entry: vulture - language: system - types: [python] - args: [--min-confidence, "80", "--exclude", "*/tests/*,*/features/*"] - ``` - - [ ] Commit: "feat(qa): add vulture dead code detection" + - [ ] Add `vulture>=2.10` to dev dependencies + - [ ] Create `vulture_whitelist.py` for false positives + - [ ] Add vulture hook: + ```yaml + - repo: local + hooks: + - id: vulture + name: Find dead code with vulture + entry: vulture + language: system + types: [python] + args: + [ + --min-confidence, + "80", + "--exclude", + "*/tests/*,*/features/*", + ] + ``` + - [ ] Commit: "feat(qa): add vulture dead code detection" - [ ] **0.1h** [Brent] Add test runner hook for changed files: - ```yaml - - repo: local - hooks: - - id: pytest-changed - name: Run tests for changed files - entry: bash -c 'git diff --cached --name-only | grep -E "\.py$" | xargs -I {} pytest tests/{} 2>/dev/null || true' - language: system - pass_filenames: false - always_run: true - ``` - - [ ] Commit: "feat(qa): add test runner for changed files" + + ```yaml + - repo: local + hooks: + - id: pytest-changed + name: Run tests for changed files + entry: bash -c 'git diff --cached --name-only | grep -E "\.py$" | xargs -I {} pytest tests/{} 2>/dev/null || true' + language: system + pass_filenames: false + always_run: true + ``` + + - [ ] Commit: "feat(qa): add test runner for changed files" + - [ ] **0.1i** [Brent] Add semgrep for pattern-based checks: - - [ ] Install semgrep: Add `semgrep>=1.45.0` to dev dependencies - - [ ] Create `.semgrep.yml` with initial rules: - ```yaml - rules: - - id: no-eval - pattern: eval(...) - message: "eval() is dangerous and banned" - languages: [python] - severity: ERROR - - id: no-exec - pattern: exec(...) - message: "exec() is dangerous and banned" - languages: [python] - severity: ERROR - - id: no-bare-except - pattern: | - try: - ... - except: - ... - message: "Use specific exception types" - languages: [python] - severity: WARNING - ``` - - [ ] Add semgrep hook: - ```yaml - - repo: local - hooks: - - id: semgrep - name: Scan with semgrep - entry: semgrep --config=.semgrep.yml - language: system - types: [python] - ``` - - [ ] Commit: "feat(qa): add semgrep pattern scanning" + - [ ] Install semgrep: Add `semgrep>=1.45.0` to dev dependencies + - [ ] Create `.semgrep.yml` with initial rules: + ```yaml + rules: + - id: no-eval + pattern: eval(...) + message: "eval() is dangerous and banned" + languages: [python] + severity: ERROR + - id: no-exec + pattern: exec(...) + message: "exec() is dangerous and banned" + languages: [python] + severity: ERROR + - id: no-bare-except + pattern: | + try: + ... + except: + ... + message: "Use specific exception types" + languages: [python] + severity: WARNING + ``` + - [ ] Add semgrep hook: + ```yaml + - repo: local + hooks: + - id: semgrep + name: Scan with semgrep + entry: semgrep --config=.semgrep.yml + language: system + types: [python] + ``` + - [ ] Commit: "feat(qa): add semgrep pattern scanning" - [ ] **0.1j** [Brent] Add commit message linting: - ```yaml - - repo: https://github.com/commitizen-tools/commitizen - rev: v3.13.0 - hooks: - - id: commitizen - stages: [commit-msg] - ``` - - [ ] Configure conventional commits format - - [ ] Commit: "feat(qa): add commit message linting" + + ```yaml + - repo: https://github.com/commitizen-tools/commitizen + rev: v3.13.0 + hooks: + - id: commitizen + stages: [commit-msg] + ``` + + - [ ] Configure conventional commits format + - [ ] Commit: "feat(qa): add commit message linting" + - [ ] **0.1k** [Brent] Create developer setup script `scripts/setup-dev.sh`: - ```bash - #!/bin/bash - set -euo pipefail - echo "Setting up pre-commit hooks..." - pip install pre-commit - pre-commit install - pre-commit install --hook-type commit-msg - echo "Running initial quality checks..." - pre-commit run --all-files - echo "Developer environment ready!" - ``` - - [ ] Make executable: `chmod +x scripts/setup-dev.sh` - - [ ] Update README.md developer setup instructions - - [ ] Commit: "feat(qa): add developer setup script" - **Day 2: CI/CD Pipeline with GitHub Actions (or GitLab CI equivalent)** + ```bash + #!/bin/bash + set -euo pipefail + echo "Setting up pre-commit hooks..." + pip install pre-commit + pre-commit install + pre-commit install --hook-type commit-msg + echo "Running initial quality checks..." + pre-commit run --all-files + echo "Developer environment ready!" + ``` - - [ ] **0.2** [Brent] Create GitHub Actions workflow for PR validation: - - [ ] **0.2a** [Brent] Create `.github/workflows/pr-validation.yml`: - ```yaml - name: PR Validation - on: - pull_request: - types: [opened, synchronize, reopened] - jobs: - quality-gates: - name: Quality Gates - runs-on: ubuntu-latest - ``` - - [ ] Commit: "feat(ci): add PR validation workflow scaffold" - - [ ] **0.2b** [Brent] Add Python setup and dependency caching: - ```yaml - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 # For proper git history + - [ ] Make executable: `chmod +x scripts/setup-dev.sh` + - [ ] Update README.md developer setup instructions + - [ ] Commit: "feat(qa): add developer setup script" - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: '3.13' + **Day 2: CI/CD Pipeline with GitHub Actions (or GitLab CI equivalent)** + - [ ] **0.2** [Brent] Create GitHub Actions workflow for PR validation: + - [ ] **0.2a** [Brent] Create `.github/workflows/pr-validation.yml`: - - name: Cache pip packages - uses: actions/cache@v4 - with: - path: ~/.cache/pip - key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }} - restore-keys: | - ${{ runner.os }}-pip- - ``` - - [ ] Commit: "feat(ci): add Python setup with caching" - - [ ] **0.2c** [Brent] Install dependencies and run formattin check: - ```yaml - - name: Install dependencies - run: | - pip install -e .[dev,tests] - pip install pre-commit + ```yaml + name: PR Validation + on: + pull_request: + types: [opened, synchronize, reopened] + jobs: + quality-gates: + name: Quality Gates + runs-on: ubuntu-latest + ``` - - name: Check code formatting - run: | - pre-commit run ruff-format --all-files --show-diff-on-failure - ``` - - [ ] Commit: "feat(ci): add formatting check" - - [ ] **0.2d** [Brent] Add linting step: - ```yaml - - name: Lint with Ruff - run: | - ruff check . --output-format=github - ``` - - [ ] Use GitHub annotations format for inline PR comments - - [ ] Commit: "feat(ci): add linting with GitHub annotations" - - [ ] **0.2e** [Brent] Add type checking step: - ```yaml - - name: Type check with pyright - run: | - pyright --outputjson > pyright-results.json || true - python scripts/parse-pyright-results.py pyright-results.json - ``` - - [ ] Create `scripts/parse-pyright-results.py` to format errors as GitHub annotations - - [ ] Commit: "feat(ci): add type checking with annotations" - - [ ] **0.2f** [Brent] Add security scanning: - ```yaml - - name: Security scan with bandit - run: | - bandit -r src/ -f json -o bandit-results.json || true - python scripts/parse-bandit-results.py bandit-results.json + - [ ] Commit: "feat(ci): add PR validation workflow scaffold" - - name: Check for vulnerabilities - run: | - pip install safety - safety check --json > safety-results.json || true - python scripts/parse-safety-results.py safety-results.json - ``` - - [ ] Create parsing scripts for annotations - - [ ] Commit: "feat(ci): add security scanning" - - [ ] **0.2g** [Brent] Add test execution with coverage: - ```yaml - - name: Run tests with coverage - run: | - nox -s unit_tests -- --junit-xml=test-results.xml - nox -s coverage_report + - [ ] **0.2b** [Brent] Add Python setup and dependency caching: - - name: Upload test results - if: always() - uses: actions/upload-artifact@v4 - with: - name: test-results - path: test-results.xml + ```yaml + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # For proper git history - - name: Comment coverage on PR - uses: py-cov-action/python-coverage-comment-action@v3 - with: - GITHUB_TOKEN: ${{ github.token }} - MINIMUM_GREEN: 85 - MINIMUM_ORANGE: 70 - ``` - - [ ] Commit: "feat(ci): add test execution with coverage reporting" - - [ ] **0.2h** [Brent] Add semgrep scanning: - ```yaml - - name: Semgrep scan - uses: returntocorp/semgrep-action@v1 - with: - config: .semgrep.yml - generateSarif: true + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.13" - - name: Upload SARIF - uses: github/codeql-action/upload-sarif@v3 - with: - sarif_file: semgrep.sarif - ``` - - [ ] Commit: "feat(ci): add semgrep scanning with SARIF" - - [ ] **0.2i** [Brent] Add PR comment summary: - ```yaml - - name: Generate quality report - if: always() - run: | - python scripts/generate-quality-report.py \ - --coverage coverage.xml \ - --pyright pyright-results.json \ - --bandit bandit-results.json \ - --output pr-comment.md + - name: Cache pip packages + uses: actions/cache@v4 + with: + path: ~/.cache/pip + key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }} + restore-keys: | + ${{ runner.os }}-pip- + ``` - - name: Comment on PR - if: always() - uses: actions/github-script@v7 - with: - script: | - const fs = require('fs'); - const comment = fs.readFileSync('pr-comment.md', 'utf8'); - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: comment - }); - ``` - - [ ] Create `scripts/generate-quality-report.py` to aggregate results - - [ ] Commit: "feat(ci): add PR quality report comment" - - [ ] **0.2j** [Brent] Add job failure conditions: - ```yaml - - name: Check quality gates - run: | - python scripts/check-quality-gates.py \ - --coverage-min 85 \ - --type-errors-max 0 \ - --security-issues-max 0 - ``` - - [ ] Script exits with code 1 if any gate fails - - [ ] Commit: "feat(ci): add quality gate enforcement" - - [ ] **0.2k** [Brent] Create branch protection rules documentation: - - [ ] Document in `docs/development/branch-protection.md`: - - [ ] Require PR validation workflow to pass - - [ ] Require at least 1 review (Brent reviews all) - - [ ] Dismiss stale reviews on new commits - - [ ] Require branches to be up to date before merging - - [ ] Commit: "docs(qa): add branch protection documentation" + - [ ] Commit: "feat(ci): add Python setup with caching" - **Day 3: Advanced Automation and Monitoring** + - [ ] **0.2c** [Brent] Install dependencies and run formattin check: - - [ ] **0.3** [Brent] Set up advanced quality automation: - - [ ] **0.3a** [Brent] Create nightly quality check workflow `.github/workflows/nightly-quality.yml`: - ```yaml - name: Nightly Quality Check - on: - schedule: - - cron: '0 0 * * *' # Run at midnight UTC - workflow_dispatch: # Allow manual trigger - ``` - - [ ] Run full test suite including slow tests - - [ ] Run mutation testing with mutmut - - [ ] Generate comprehensive reports - - [ ] Commit: "feat(ci): add nightly quality checks" - - [ ] **0.3b** [Brent] Add dependency update automation: - ```yaml - name: Dependency Updates - on: - schedule: - - cron: '0 9 * * MON' # Weekly on Mondays - jobs: - update-deps: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - name: Update dependencies + ```yaml + - name: Install dependencies run: | - pip install pip-tools - pip-compile --upgrade - pip install .[dev,tests] - pre-commit autoupdate - - name: Create Pull Request - uses: peter-evans/create-pull-request@v5 + pip install -e .[dev,tests] + pip install pre-commit + + - name: Check code formatting + run: | + pre-commit run ruff-format --all-files --show-diff-on-failure + ``` + + - [ ] Commit: "feat(ci): add formatting check" + + - [ ] **0.2d** [Brent] Add linting step: + + ```yaml + - name: Lint with Ruff + run: | + ruff check . --output-format=github + ``` + + - [ ] Use GitHub annotations format for inline PR comments + - [ ] Commit: "feat(ci): add linting with GitHub annotations" + + - [ ] **0.2e** [Brent] Add type checking step: + + ```yaml + - name: Type check with pyright + run: | + pyright --outputjson > pyright-results.json || true + python scripts/parse-pyright-results.py pyright-results.json + ``` + + - [ ] Create `scripts/parse-pyright-results.py` to format errors as GitHub annotations + - [ ] Commit: "feat(ci): add type checking with annotations" + + - [ ] **0.2f** [Brent] Add security scanning: + + ```yaml + - name: Security scan with bandit + run: | + bandit -r src/ -f json -o bandit-results.json || true + python scripts/parse-bandit-results.py bandit-results.json + + - name: Check for vulnerabilities + run: | + pip install safety + safety check --json > safety-results.json || true + python scripts/parse-safety-results.py safety-results.json + ``` + + - [ ] Create parsing scripts for annotations + - [ ] Commit: "feat(ci): add security scanning" + + - [ ] **0.2g** [Brent] Add test execution with coverage: + + ```yaml + - name: Run tests with coverage + run: | + nox -s unit_tests -- --junit-xml=test-results.xml + nox -s coverage_report + + - name: Upload test results + if: always() + uses: actions/upload-artifact@v4 with: - title: "chore: update dependencies" - body: "Automated dependency updates" - branch: deps/automated-update - ``` - - [ ] Commit: "feat(ci): add dependency update automation" - - [ ] **0.3c** [Brent] Create complexity monitoring: - - [ ] Install `radon>=6.0.1` for complexity analysis - - [ ] Add to pre-commit: - ```yaml - - repo: local - hooks: - - id: complexity-check - name: Check code complexity - entry: radon cc src/ -nb -s - language: system - pass_filenames: false - ``` - - [ ] Add to CI pipeline with threshold enforcement - - [ ] Commit: "feat(qa): add complexity monitoring" - - [ ] **0.3d** [Brent] Set up performance regression detection: - - [ ] Create `benchmarks/` directory with ASV benchmarks - - [ ] Add benchmark job to CI: - ```yaml - - name: Run benchmarks - run: | - asv machine --yes - asv run HEAD^..HEAD - asv compare HEAD^ HEAD - ``` - - [ ] Fail if performance regresses >10% - - [ ] Commit: "feat(ci): add performance regression detection" - - [ ] **0.3e** [Brent] Create quality metrics dashboard script: - - [ ] Script `scripts/generate-metrics-dashboard.py`: - - [ ] Aggregate coverage trends - - [ ] Track type checking progress - - [ ] Monitor code complexity trends - - [ ] Count TODO/FIXME/HACK comments - - [ ] Generate markdown report - - [ ] Run weekly and post to team - - [ ] Commit: "feat(qa): add quality metrics dashboard" - - [ ] **0.3f** [Brent] Set up documentation quality checks: - - [ ] Add doc linting: - ```yaml - - repo: local - hooks: - - id: doc-quality - name: Check documentation quality - entry: python scripts/check-docstrings.py - language: system - types: [python] - ``` - - [ ] Verify all public functions have docstrings - - [ ] Check docstring format (Google style) - - [ ] Commit: "feat(qa): add documentation quality checks" - - [ ] **0.3g** [Brent] Create ADR compliance checker: - - [ ] Script `scripts/check-adr-compliance.py`: - - [ ] Parse ADRs from `docs/architecture/decisions/` - - [ ] Check code against ADR requirements - - [ ] Flag violations (e.g., sync code in async modules) - - [ ] Add to pre-commit and CI - - [ ] Commit: "feat(qa): add ADR compliance checking" - - [ ] **0.3h** [Brent] Set up coverage delta checking: - - [ ] Modify CI to track coverage changes: - ```yaml - - name: Check coverage delta - run: | - git fetch origin main - nox -s coverage_report -- --compare-branch=origin/main - python scripts/check-coverage-delta.py --min-delta=-0.5 - ``` - - [ ] Fail if coverage drops more than 0.5% - - [ ] Commit: "feat(ci): add coverage delta enforcement" - - [ ] **0.3i** [Brent] Create PR template with quality checklist: - - [ ] Create `.github/pull_request_template.md`: - ```markdown - ## Description - Brief description of changes + name: test-results + path: test-results.xml - ## Quality Checklist - - [ ] Tests added/updated for new functionality - - [ ] Type hints added for all new functions - - [ ] Docstrings added/updated - - [ ] No new linting warnings - - [ ] Coverage maintained or increased - - [ ] ADRs followed - - [ ] Security implications considered + - name: Comment coverage on PR + uses: py-cov-action/python-coverage-comment-action@v3 + with: + GITHUB_TOKEN: ${{ github.token }} + MINIMUM_GREEN: 85 + MINIMUM_ORANGE: 70 + ``` - ## Testing - How has this been tested? - ``` - - [ ] Commit: "feat(qa): add PR template with quality checklist" - - [ ] **0.3j** [Brent] Document quality automation setup: - - [ ] Create `docs/development/quality-automation.md`: - - [ ] Pre-commit hook reference - - [ ] CI/CD pipeline overview - - [ ] How to run quality checks locally - - [ ] How to handle quality gate failures - - [ ] Exemption process (when needed) - - [ ] Add to developer onboarding - - [ ] Commit: "docs(qa): document quality automation" + - [ ] Commit: "feat(ci): add test execution with coverage reporting" - **Post Day 3: Transition Plan** + - [ ] **0.2h** [Brent] Add semgrep scanning: - - [ ] **0.4** [Brent] Transition to selective manual review (Days 4-8): - - [ ] **0.4a** [Brent] Focus manual reviews on: - - [ ] Architectural decisions - - [ ] Complex algorithms - - [ ] API contracts - - [ ] Security-sensitive code - - [ ] Skip reviewing: formatting, basic types, simple CRUD - - [ ] **0.4b** [Brent] Create review priority matrix: - - [ ] P0: Security, API changes, architecture - - [ ] P1: Complex business logic, algorithms - - [ ] P2: Normal features - - [ ] P3: Tests, docs, refactoring (trust automation) + ```yaml + - name: Semgrep scan + uses: returntocorp/semgrep-action@v1 + with: + config: .semgrep.yml + generateSarif: true - - [ ] **0.5** [Brent] Transition to high-impact work (After Day 8): - - [ ] **0.5a** [Luis + Brent] Move to validation pipeline work: - - [ ] Help implement semantic validation in execution actors - - [ ] Create validation test suites - - [ ] Document validation patterns - - [ ] **0.5b** [Luis + Brent] Assist with change tracking edge cases: - - [ ] Test tool-based change tracking thoroughly - - [ ] Find and fix edge cases - - [ ] Create comprehensive test scenarios + - name: Upload SARIF + uses: github/codeql-action/upload-sarif@v3 + with: + sarif_file: semgrep.sarif + ``` + - [ ] Commit: "feat(ci): add semgrep scanning with SARIF" + - [ ] **0.2i** [Brent] Add PR comment summary: + ```yaml + - name: Generate quality report + if: always() + run: | + python scripts/generate-quality-report.py \ + --coverage coverage.xml \ + --pyright pyright-results.json \ + --bandit bandit-results.json \ + --output pr-comment.md + - name: Comment on PR + if: always() + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + const comment = fs.readFileSync('pr-comment.md', 'utf8'); + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); + ``` + + - [ ] Create `scripts/generate-quality-report.py` to aggregate results + - [ ] Commit: "feat(ci): add PR quality report comment" + + - [ ] **0.2j** [Brent] Add job failure conditions: + + ```yaml + - name: Check quality gates + run: | + python scripts/check-quality-gates.py \ + --coverage-min 85 \ + --type-errors-max 0 \ + --security-issues-max 0 + ``` + + - [ ] Script exits with code 1 if any gate fails + - [ ] Commit: "feat(ci): add quality gate enforcement" + + - [ ] **0.2k** [Brent] Create branch protection rules documentation: + - [ ] Document in `docs/development/branch-protection.md`: + - [ ] Require PR validation workflow to pass + - [ ] Require at least 1 review (Brent reviews all) + - [ ] Dismiss stale reviews on new commits + - [ ] Require branches to be up to date before merging + - [ ] Commit: "docs(qa): add branch protection documentation" + + **Day 3: Advanced Automation and Monitoring** + - [ ] **0.3** [Brent] Set up advanced quality automation: + - [ ] **0.3a** [Brent] Create nightly quality check workflow `.github/workflows/nightly-quality.yml`: + + ```yaml + name: Nightly Quality Check + on: + schedule: + - cron: "0 0 * * *" # Run at midnight UTC + workflow_dispatch: # Allow manual trigger + ``` + + - [ ] Run full test suite including slow tests + - [ ] Run mutation testing with mutmut + - [ ] Generate comprehensive reports + - [ ] Commit: "feat(ci): add nightly quality checks" + + - [ ] **0.3b** [Brent] Add dependency update automation: + + ```yaml + name: Dependency Updates + on: + schedule: + - cron: "0 9 * * MON" # Weekly on Mondays + jobs: + update-deps: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Update dependencies + run: | + pip install pip-tools + pip-compile --upgrade + pip install .[dev,tests] + pre-commit autoupdate + - name: Create Pull Request + uses: peter-evans/create-pull-request@v5 + with: + title: "chore: update dependencies" + body: "Automated dependency updates" + branch: deps/automated-update + ``` + + - [ ] Commit: "feat(ci): add dependency update automation" + + - [ ] **0.3c** [Brent] Create complexity monitoring: + - [ ] Install `radon>=6.0.1` for complexity analysis + - [ ] Add to pre-commit: + ```yaml + - repo: local + hooks: + - id: complexity-check + name: Check code complexity + entry: radon cc src/ -nb -s + language: system + pass_filenames: false + ``` + - [ ] Add to CI pipeline with threshold enforcement + - [ ] Commit: "feat(qa): add complexity monitoring" + - [ ] **0.3d** [Brent] Set up performance regression detection: + - [ ] Create `benchmarks/` directory with ASV benchmarks + - [ ] Add benchmark job to CI: + ```yaml + - name: Run benchmarks + run: | + asv machine --yes + asv run HEAD^..HEAD + asv compare HEAD^ HEAD + ``` + - [ ] Fail if performance regresses >10% + - [ ] Commit: "feat(ci): add performance regression detection" + - [ ] **0.3e** [Brent] Create quality metrics dashboard script: + - [ ] Script `scripts/generate-metrics-dashboard.py`: + - [ ] Aggregate coverage trends + - [ ] Track type checking progress + - [ ] Monitor code complexity trends + - [ ] Count TODO/FIXME/HACK comments + - [ ] Generate markdown report + - [ ] Run weekly and post to team + - [ ] Commit: "feat(qa): add quality metrics dashboard" + - [ ] **0.3f** [Brent] Set up documentation quality checks: + - [ ] Add doc linting: + ```yaml + - repo: local + hooks: + - id: doc-quality + name: Check documentation quality + entry: python scripts/check-docstrings.py + language: system + types: [python] + ``` + - [ ] Verify all public functions have docstrings + - [ ] Check docstring format (Google style) + - [ ] Commit: "feat(qa): add documentation quality checks" + - [ ] **0.3g** [Brent] Create ADR compliance checker: + - [ ] Script `scripts/check-adr-compliance.py`: + - [ ] Parse ADRs from `docs/architecture/decisions/` + - [ ] Check code against ADR requirements + - [ ] Flag violations (e.g., sync code in async modules) + - [ ] Add to pre-commit and CI + - [ ] Commit: "feat(qa): add ADR compliance checking" + - [ ] **0.3h** [Brent] Set up coverage delta checking: + - [ ] Modify CI to track coverage changes: + ```yaml + - name: Check coverage delta + run: | + git fetch origin main + nox -s coverage_report -- --compare-branch=origin/main + python scripts/check-coverage-delta.py --min-delta=-0.5 + ``` + - [ ] Fail if coverage drops more than 0.5% + - [ ] Commit: "feat(ci): add coverage delta enforcement" + - [ ] **0.3i** [Brent] Create PR template with quality checklist: + - [ ] Create `.github/pull_request_template.md`: + + ```markdown + ## Description + + Brief description of changes + + ## Quality Checklist + + - [ ] Tests added/updated for new functionality + - [ ] Type hints added for all new functions + - [ ] Docstrings added/updated + - [ ] No new linting warnings + - [ ] Coverage maintained or increased + - [ ] ADRs followed + - [ ] Security implications considered + + ## Testing + + How has this been tested? + ``` + + - [ ] Commit: "feat(qa): add PR template with quality checklist" + + - [ ] **0.3j** [Brent] Document quality automation setup: + - [ ] Create `docs/development/quality-automation.md`: + - [ ] Pre-commit hook reference + - [ ] CI/CD pipeline overview + - [ ] How to run quality checks locally + - [ ] How to handle quality gate failures + - [ ] Exemption process (when needed) + - [ ] Add to developer onboarding + - [ ] Commit: "docs(qa): document quality automation" + + **Post Day 3: Transition Plan** + - [ ] **0.4** [Brent] Transition to selective manual review (Days 4-8): + - [ ] **0.4a** [Brent] Focus manual reviews on: + - [ ] Architectural decisions + - [ ] Complex algorithms + - [ ] API contracts + - [ ] Security-sensitive code + - [ ] Skip reviewing: formatting, basic types, simple CRUD + - [ ] **0.4b** [Brent] Create review priority matrix: + - [ ] P0: Security, API changes, architecture + - [ ] P1: Complex business logic, algorithms + - [ ] P2: Normal features + - [ ] P3: Tests, docs, refactoring (trust automation) + + - [ ] **0.5** [Brent] Transition to high-impact work (After Day 8): + - [ ] **0.5a** [Luis + Brent] Move to validation pipeline work: + - [ ] Help implement semantic validation in execution actors + - [ ] Create validation test suites + - [ ] Document validation patterns + - [ ] **0.5b** [Luis + Brent] Assist with change tracking edge cases: + - [ ] Test tool-based change tracking thoroughly + - [ ] Find and fix edge cases + - [ ] Create comprehensive test scenarios --- @@ -8780,160 +9327,160 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **CRITICAL SECURITY BLOCKERS** (Must be addressed before any production use) - [ ] **Stage SEC1: Remove eval() Vulnerability** (Day 1-2) **[Luis - CRITICAL]** - - [ ] Code: Remove all eval() from config parsing - - [ ] **SEC1.1** [Luis] Audit all files for `eval()` usage in `src/cleveragents/`: - - [ ] Search for `eval(`, `exec(`, `compile(` calls - - [ ] Document each occurrence with file path and line number - - [ ] Classify as: (a) test-only, (b) removable, (c) requires redesign - - [ ] **SEC1.2** [Luis] Replace eval-based config transforms: - - [ ] Create whitelist of allowed transform operators - - [ ] Implement safe expression parser (no arbitrary code execution) - - [ ] Or: transforms must reference named functions from a registry - - [ ] **SEC1.3** [Luis] Remove eval from reactive routing config: - - [ ] Audit `src/cleveragents/reactive/config_parser.py` - - [ ] Replace dynamic code execution with safe alternatives - - [ ] **SEC1.4** [Brent] Code review all eval removal changes - - [ ] Tests: Security tests - - [ ] **SEC1.5** [Rui] Write Behave scenarios in `features/security_eval.feature`: - - [ ] Scenario: Config with code injection attempt is rejected - - [ ] Scenario: Malicious transform expression does not execute - - [ ] Scenario: Valid config still works after eval removal + - [ ] Code: Remove all eval() from config parsing + - [ ] **SEC1.1** [Luis] Audit all files for `eval()` usage in `src/cleveragents/`: + - [ ] Search for `eval(`, `exec(`, `compile(` calls + - [ ] Document each occurrence with file path and line number + - [ ] Classify as: (a) test-only, (b) removable, (c) requires redesign + - [ ] **SEC1.2** [Luis] Replace eval-based config transforms: + - [ ] Create whitelist of allowed transform operators + - [ ] Implement safe expression parser (no arbitrary code execution) + - [ ] Or: transforms must reference named functions from a registry + - [ ] **SEC1.3** [Luis] Remove eval from reactive routing config: + - [ ] Audit `src/cleveragents/reactive/config_parser.py` + - [ ] Replace dynamic code execution with safe alternatives + - [ ] **SEC1.4** [Brent] Code review all eval removal changes + - [ ] Tests: Security tests + - [ ] **SEC1.5** [Rui] Write Behave scenarios in `features/security_eval.feature`: + - [ ] Scenario: Config with code injection attempt is rejected + - [ ] Scenario: Malicious transform expression does not execute + - [ ] Scenario: Valid config still works after eval removal - [ ] **Stage SEC2: Template Injection Prevention** (Day 3-4) **[Luis]** - - [ ] Code: Secure template rendering - - [ ] **SEC2.1** [Luis] Replace `str.format()` with safe template engine: - - [ ] Use Jinja2 with sandboxed environment - - [ ] Or: restrict token set severely (only `{variable}` substitution) - - [ ] Prevent access to `__class__`, `__globals__`, etc. - - [ ] **SEC2.2** [Luis] Implement input sanitization for prompts: - - [ ] Treat user input as data, not instruction overrides - - [ ] Escape special characters in user-provided text - - [ ] Strict role separation in prompts (system vs user) - - [ ] **SEC2.3** [Luis] Add prompt injection mitigations: - - [ ] Detect common injection patterns - - [ ] Warn or reject suspicious inputs - - [ ] Log potential injection attempts - - [ ] Tests: Template security tests - - [ ] **SEC2.4** [Rui] Write Behave scenarios in `features/security_templates.feature`: - - [ ] Scenario: Template with Jinja2 injection attempt fails safely - - [ ] Scenario: User input with special characters is escaped - - [ ] Scenario: Prompt injection attempt is detected + - [ ] Code: Secure template rendering + - [ ] **SEC2.1** [Luis] Replace `str.format()` with safe template engine: + - [ ] Use Jinja2 with sandboxed environment + - [ ] Or: restrict token set severely (only `{variable}` substitution) + - [ ] Prevent access to `__class__`, `__globals__`, etc. + - [ ] **SEC2.2** [Luis] Implement input sanitization for prompts: + - [ ] Treat user input as data, not instruction overrides + - [ ] Escape special characters in user-provided text + - [ ] Strict role separation in prompts (system vs user) + - [ ] **SEC2.3** [Luis] Add prompt injection mitigations: + - [ ] Detect common injection patterns + - [ ] Warn or reject suspicious inputs + - [ ] Log potential injection attempts + - [ ] Tests: Template security tests + - [ ] **SEC2.4** [Rui] Write Behave scenarios in `features/security_templates.feature`: + - [ ] Scenario: Template with Jinja2 injection attempt fails safely + - [ ] Scenario: User input with special characters is escaped + - [ ] Scenario: Prompt injection attempt is detected - [ ] **Stage SEC3: Exception Handling Audit** (Day 4-5) **[Luis]** - - [ ] Code: Stop swallowing exceptions (automated checks + manual fixes) - - [ ] **SEC3.1** [Luis] Run automated exception handling audit: - - [ ] Use semgrep rules to find bare `except:` or `except Exception:` - - [ ] Use vulture to find unreachable exception handlers - - [ ] Document each occurrence for manual review - - [ ] **SEC3.2** [Luis] Fix silent exception handling: - - [ ] Capture exception details - - [ ] Attach to message metadata or error state - - [ ] Fail the stream/plan with clear error state - - [ ] Log at appropriate level (error, not debug) - - [ ] **SEC3.3** [Luis] Add error context propagation: - - [ ] Errors should include stack trace reference - - [ ] Errors should identify the component that failed - - [ ] Errors should suggest recovery actions where possible - - [ ] **SEC3.4** [Brent] Review complex exception handling patterns only: - - [ ] Review async exception handling edge cases - - [ ] Validate error propagation across actor boundaries - - [ ] Tests: Exception handling tests - - [ ] **SEC3.5** [Rui] Write Behave scenarios in `features/security_exceptions.feature`: - - [ ] Scenario: Component failure surfaces as clear error - - [ ] Scenario: Error includes actionable information - - [ ] Scenario: No silent failures in normal operation + - [ ] Code: Stop swallowing exceptions (automated checks + manual fixes) + - [ ] **SEC3.1** [Luis] Run automated exception handling audit: + - [ ] Use semgrep rules to find bare `except:` or `except Exception:` + - [ ] Use vulture to find unreachable exception handlers + - [ ] Document each occurrence for manual review + - [ ] **SEC3.2** [Luis] Fix silent exception handling: + - [ ] Capture exception details + - [ ] Attach to message metadata or error state + - [ ] Fail the stream/plan with clear error state + - [ ] Log at appropriate level (error, not debug) + - [ ] **SEC3.3** [Luis] Add error context propagation: + - [ ] Errors should include stack trace reference + - [ ] Errors should identify the component that failed + - [ ] Errors should suggest recovery actions where possible + - [ ] **SEC3.4** [Brent] Review complex exception handling patterns only: + - [ ] Review async exception handling edge cases + - [ ] Validate error propagation across actor boundaries + - [ ] Tests: Exception handling tests + - [ ] **SEC3.5** [Rui] Write Behave scenarios in `features/security_exceptions.feature`: + - [ ] Scenario: Component failure surfaces as clear error + - [ ] Scenario: Error includes actionable information + - [ ] Scenario: No silent failures in normal operation - [ ] **Stage SEC4: Async Lifecycle Correctness** (Day 5-6) **[Luis]** - - [ ] Code: Fix async resource leaks (automated detection + fixes) - - [ ] **SEC4.1** [Luis] Run automated async pattern detection: - - [ ] Use semgrep to find all `asyncio.new_event_loop()` calls - - [ ] Use custom linter to detect unclosed resources - - [ ] Generate report of potential resource leaks - - [ ] **SEC4.2** [Luis] Fix RxPy subscription leaks: - - [ ] Ensure subscriptions are disposed on shutdown - - [ ] Dispose subscriptions on stream reconfiguration - - [ ] Track active subscriptions for debugging - - [ ] **SEC4.3** [Luis] Fix LangGraph checkpoint file leaks: - - [ ] Audit checkpoint file creation - - [ ] Implement cleanup for old checkpoint files - - [ ] Add retention policy for checkpoints - - [ ] **SEC4.4** [Brent] Selective review of async patterns: - - [ ] Review only complex async state machines - - [ ] Validate concurrent access patterns - - [ ] Tests: Resource leak tests - - [ ] **SEC4.5** [Rui] Write Behave scenarios in `features/security_async.feature`: - - [ ] Scenario: Long-running process does not leak memory - - [ ] Scenario: Shutdown cleans up all subscriptions - - [ ] Scenario: Checkpoint files cleaned after retention period + - [ ] Code: Fix async resource leaks (automated detection + fixes) + - [ ] **SEC4.1** [Luis] Run automated async pattern detection: + - [ ] Use semgrep to find all `asyncio.new_event_loop()` calls + - [ ] Use custom linter to detect unclosed resources + - [ ] Generate report of potential resource leaks + - [ ] **SEC4.2** [Luis] Fix RxPy subscription leaks: + - [ ] Ensure subscriptions are disposed on shutdown + - [ ] Dispose subscriptions on stream reconfiguration + - [ ] Track active subscriptions for debugging + - [ ] **SEC4.3** [Luis] Fix LangGraph checkpoint file leaks: + - [ ] Audit checkpoint file creation + - [ ] Implement cleanup for old checkpoint files + - [ ] Add retention policy for checkpoints + - [ ] **SEC4.4** [Brent] Selective review of async patterns: + - [ ] Review only complex async state machines + - [ ] Validate concurrent access patterns + - [ ] Tests: Resource leak tests + - [ ] **SEC4.5** [Rui] Write Behave scenarios in `features/security_async.feature`: + - [ ] Scenario: Long-running process does not leak memory + - [ ] Scenario: Shutdown cleans up all subscriptions + - [ ] Scenario: Checkpoint files cleaned after retention period - [ ] **Stage SEC5: Secrets Management** (Day 6-7) **[Hamza]** - - [ ] Code: Secure credential handling - - [ ] **SEC5.1** [Hamza] Implement secrets masking in logs: - - [ ] Detect API keys in log output - - [ ] Mask sensitive values (show only last 4 chars) - - [ ] Never log full credentials - - [ ] **SEC5.2** [Hamza] Secure environment variable handling: - - [ ] Validate API key format before use - - [ ] Clear error if required key missing - - [ ] Support secrets from file (for containerized environments) - - [ ] **SEC5.3** [Hamza] Prevent secrets in generated code: - - [ ] Detect hardcoded API keys in LLM output - - [ ] Warn if generated code contains potential secrets - - [ ] Block apply if secrets detected without override - - [ ] Tests: Secrets management tests - - [ ] **SEC5.4** [Rui] Write Behave scenarios in `features/security_secrets.feature`: - - [ ] Scenario: API key in log output is masked - - [ ] Scenario: Generated code with hardcoded key is flagged - - [ ] Scenario: Missing API key produces clear error + - [ ] Code: Secure credential handling + - [ ] **SEC5.1** [Hamza] Implement secrets masking in logs: + - [ ] Detect API keys in log output + - [ ] Mask sensitive values (show only last 4 chars) + - [ ] Never log full credentials + - [ ] **SEC5.2** [Hamza] Secure environment variable handling: + - [ ] Validate API key format before use + - [ ] Clear error if required key missing + - [ ] Support secrets from file (for containerized environments) + - [ ] **SEC5.3** [Hamza] Prevent secrets in generated code: + - [ ] Detect hardcoded API keys in LLM output + - [ ] Warn if generated code contains potential secrets + - [ ] Block apply if secrets detected without override + - [ ] Tests: Secrets management tests + - [ ] **SEC5.4** [Rui] Write Behave scenarios in `features/security_secrets.feature`: + - [ ] Scenario: API key in log output is masked + - [ ] Scenario: Generated code with hardcoded key is flagged + - [ ] Scenario: Missing API key produces clear error - [ ] **Stage SEC6: Read-Only Action Enforcement** (Day 7-8) **[Luis]** - - [ ] Code: Enforce read_only actions - - [ ] **SEC6.1** [Luis] Add skill metadata validation at execution time: - - [ ] Check if action has `read_only: true` - - [ ] If read_only, verify all skills have `read_only: true` metadata - - [ ] Block execution if write skill detected in read-only action - - [ ] **SEC6.2** [Luis] Implement safety profile validation (DEFERRED to post-30; see Stage POST1): - - [ ] Action can specify `safety_profile` with: - - [ ] `allowed_skill_categories: list[str] | None` - - [ ] `denied_skill_categories: list[str] | None` - - [ ] `require_checkpoints: bool` - - [ ] `require_sandbox: bool` - - [ ] `require_human_approval: bool` (apply approval gate) - - [ ] `max_cost_usd: float | None` - - [ ] `max_retries: int` - - [ ] Add action-create CLI flags to populate SafetyProfile: - - [ ] `--require-sandbox` - - [ ] `--require-checkpoints` - - [ ] `--require-apply-approval` - - [ ] `--allow-skill-category ` (repeatable) - - [ ] `--deny-skill-category ` (repeatable) - - [ ] `--max-cost-usd ` - - [ ] `--max-retries ` - - [ ] Enforce safety profile at execution time - - [ ] Tests: Safety enforcement tests - - [ ] **SEC6.3** [Rui] Write Behave scenarios in `features/security_readonly.feature`: - - [ ] Scenario: Read-only action blocked from using write skill - - [ ] Scenario: Safety profile with require_sandbox enforced - - [ ] Scenario: Safety profile with require_checkpoints enforced + - [ ] Code: Enforce read_only actions + - [ ] **SEC6.1** [Luis] Add skill metadata validation at execution time: + - [ ] Check if action has `read_only: true` + - [ ] If read_only, verify all skills have `read_only: true` metadata + - [ ] Block execution if write skill detected in read-only action + - [ ] **SEC6.2** [Luis] Implement safety profile validation (DEFERRED to post-30; see Stage POST1): + - [ ] Action can specify `safety_profile` with: + - [ ] `allowed_skill_categories: list[str] | None` + - [ ] `denied_skill_categories: list[str] | None` + - [ ] `require_checkpoints: bool` + - [ ] `require_sandbox: bool` + - [ ] `require_human_approval: bool` (apply approval gate) + - [ ] `max_cost_usd: float | None` + - [ ] `max_retries: int` + - [ ] Add action-create CLI flags to populate SafetyProfile: + - [ ] `--require-sandbox` + - [ ] `--require-checkpoints` + - [ ] `--require-apply-approval` + - [ ] `--allow-skill-category ` (repeatable) + - [ ] `--deny-skill-category ` (repeatable) + - [ ] `--max-cost-usd ` + - [ ] `--max-retries ` + - [ ] Enforce safety profile at execution time + - [ ] Tests: Safety enforcement tests + - [ ] **SEC6.3** [Rui] Write Behave scenarios in `features/security_readonly.feature`: + - [ ] Scenario: Read-only action blocked from using write skill + - [ ] Scenario: Safety profile with require_sandbox enforced + - [ ] Scenario: Safety profile with require_checkpoints enforced - [ ] **Stage SEC7: Audit Logging** (Day 8-9) **[Hamza]** - - [ ] Code: Comprehensive audit trail - - [ ] **SEC7.1** [Hamza] Implement apply audit logging: - - [ ] Record who applied, what changed, when, why - - [ ] Include plan ID, action ID, project ID - - [ ] Include changeset summary (files affected) - - [ ] Store in audit table with retention policy - - [ ] **SEC7.2** [Hamza] Create Alembic migration for `audit_log` table: - - [ ] Schema: `audit_id`, `event_type`, `user_id`, `plan_id`, `details`, `created_at` - - [ ] Index on `event_type`, `plan_id`, `created_at` - - [ ] **SEC7.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] audit list` CLI command: - - [ ] List recent audit events - - [ ] Filter by event type, plan, date range - - [ ] Tests: Audit logging tests - - [ ] **SEC7.4** [Rui] Write Behave scenarios in `features/security_audit.feature`: - - [ ] Scenario: Apply creates audit log entry - - [ ] Scenario: Audit log queryable by plan ID - - [ ] Scenario: Audit list CLI shows recent events + - [ ] Code: Comprehensive audit trail + - [ ] **SEC7.1** [Hamza] Implement apply audit logging: + - [ ] Record who applied, what changed, when, why + - [ ] Include plan ID, action ID, project ID + - [ ] Include changeset summary (files affected) + - [ ] Store in audit table with retention policy + - [ ] **SEC7.2** [Hamza] Create Alembic migration for `audit_log` table: + - [ ] Schema: `audit_id`, `event_type`, `user_id`, `plan_id`, `details`, `created_at` + - [ ] Index on `event_type`, `plan_id`, `created_at` + - [ ] **SEC7.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] audit list` CLI command: + - [ ] List recent audit events + - [ ] Filter by event type, plan, date range + - [ ] Tests: Audit logging tests + - [ ] **SEC7.4** [Rui] Write Behave scenarios in `features/security_audit.feature`: + - [ ] Scenario: Apply creates audit log entry + - [ ] Scenario: Audit log queryable by plan ID + - [ ] Scenario: Audit list CLI shows recent events --- @@ -8942,130 +9489,130 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Target: Days 8-12** - [ ] **Stage SESS1: Session Management** (Day 8-9) **[Hamza]** - - [ ] Code: Implement stable session persistence - - [ ] **SESS1.1** [Hamza] Define `Session` model in `src/cleveragents/domain/models/core/session.py`: - - [ ] Field `session_id: str` - ULID identifier - - [ ] Field `user_id: str | None` - optional user identity - - [ ] Field `automation_level: AutomationLevel` - session-level setting - - [ ] Field `current_plan_id: str | None` - active plan - - [ ] Field `plan_history: list[str]` - recent plan IDs - - [ ] Field `created_at: datetime` - - [ ] Field `last_active_at: datetime` - - [ ] Field `metadata: dict[str, Any]` - extensible metadata - - [ ] **SESS1.2** [Hamza] Create `SessionService` in `src/cleveragents/application/services/session_service.py`: - - [ ] Method `create_session() -> Session` - create new session with ULID - - [ ] Method `get_session(session_id: str) -> Session | None` - - [ ] Method `get_or_create_session() -> Session` - resume or create - - [ ] Method `update_activity(session_id: str) -> None` - touch last_active_at - - [ ] Method `set_current_plan(session_id: str, plan_id: str) -> None` - - [ ] Method `set_automation_level(session_id: str, level: AutomationLevel) -> None` - - [ ] **SESS1.3** [Hamza] Create Alembic migration for `sessions` table: - - [ ] Schema matching Session model - - [ ] Index on `last_active_at` for cleanup queries - - [ ] **SESS1.4** [Hamza] Implement session persistence across CLI invocations: - - [ ] Store session ID in `~/.cleveragents/session` - - [ ] Resume session on CLI startup if exists - - [ ] Clear session file on `agents [--data-dir PATH] [--config-path PATH] session end` command - - [ ] **SESS1.5** [Hamza] Add session CLI commands: - - [ ] `agents [--data-dir PATH] [--config-path PATH] session start` - create new session explicitly - - [ ] `agents [--data-dir PATH] [--config-path PATH] session end` - end current session - - [ ] `agents [--data-dir PATH] [--config-path PATH] session set automation-level ` - set session automation - - [ ] `agents [--data-dir PATH] [--config-path PATH] session info` - show current session details - - [ ] `agents [--data-dir PATH] [--config-path PATH] session tell "" [--session ] [--actor ]` - send a prompt in the active session - - [ ] Tests: Session tests - - [ ] **SESS1.6** [Rui] Write Behave scenarios in `features/session_management.feature`: - - [ ] Scenario: Session persists across CLI invocations - - [ ] Scenario: Session automation level overrides global - - [ ] Scenario: Session end clears session state - - [ ] Scenario: New CLI invocation resumes existing session - - [ ] Scenario: `agents [--data-dir PATH] [--config-path PATH] session tell "..."` uses current session and actor + - [ ] Code: Implement stable session persistence + - [ ] **SESS1.1** [Hamza] Define `Session` model in `src/cleveragents/domain/models/core/session.py`: + - [ ] Field `session_id: str` - ULID identifier + - [ ] Field `user_id: str | None` - optional user identity + - [ ] Field `automation_level: AutomationLevel` - session-level setting + - [ ] Field `current_plan_id: str | None` - active plan + - [ ] Field `plan_history: list[str]` - recent plan IDs + - [ ] Field `created_at: datetime` + - [ ] Field `last_active_at: datetime` + - [ ] Field `metadata: dict[str, Any]` - extensible metadata + - [ ] **SESS1.2** [Hamza] Create `SessionService` in `src/cleveragents/application/services/session_service.py`: + - [ ] Method `create_session() -> Session` - create new session with ULID + - [ ] Method `get_session(session_id: str) -> Session | None` + - [ ] Method `get_or_create_session() -> Session` - resume or create + - [ ] Method `update_activity(session_id: str) -> None` - touch last_active_at + - [ ] Method `set_current_plan(session_id: str, plan_id: str) -> None` + - [ ] Method `set_automation_level(session_id: str, level: AutomationLevel) -> None` + - [ ] **SESS1.3** [Hamza] Create Alembic migration for `sessions` table: + - [ ] Schema matching Session model + - [ ] Index on `last_active_at` for cleanup queries + - [ ] **SESS1.4** [Hamza] Implement session persistence across CLI invocations: + - [ ] Store session ID in `~/.cleveragents/session` + - [ ] Resume session on CLI startup if exists + - [ ] Clear session file on `agents [--data-dir PATH] [--config-path PATH] session end` command + - [ ] **SESS1.5** [Hamza] Add session CLI commands: + - [ ] `agents [--data-dir PATH] [--config-path PATH] session start` - create new session explicitly + - [ ] `agents [--data-dir PATH] [--config-path PATH] session end` - end current session + - [ ] `agents [--data-dir PATH] [--config-path PATH] session set automation-level ` - set session automation + - [ ] `agents [--data-dir PATH] [--config-path PATH] session info` - show current session details + - [ ] `agents [--data-dir PATH] [--config-path PATH] session tell "" [--session ] [--actor ]` - send a prompt in the active session + - [ ] Tests: Session tests + - [ ] **SESS1.6** [Rui] Write Behave scenarios in `features/session_management.feature`: + - [ ] Scenario: Session persists across CLI invocations + - [ ] Scenario: Session automation level overrides global + - [ ] Scenario: Session end clears session state + - [ ] Scenario: New CLI invocation resumes existing session + - [ ] Scenario: `agents [--data-dir PATH] [--config-path PATH] session tell "..."` uses current session and actor - [ ] **Stage SESS2: Memory Service Persistence** (Day 9-10) **[Hamza]** - - [ ] Code: Fix memory loss between invocations - - [ ] **SESS2.1** [Hamza] Update `MemoryService` to use session-based storage: - - [ ] Store conversation history keyed by session_id - - [ ] Load history on session resume - - [ ] Support configurable history limits - - [ ] **SESS2.2** [Hamza] Create Alembic migration for `conversation_history` table: - - [ ] Schema: `history_id`, `session_id`, `plan_id`, `role`, `content`, `created_at` - - [ ] Foreign key to sessions - - [ ] Index on `session_id`, `plan_id` - - [ ] **SESS2.3** [Hamza] Add explicit memory configuration: - - [ ] `CLEVERAGENTS_MEMORY_BACKEND=sqlite|redis|memory` - - [ ] Document that `memory` backend loses history between invocations - - [ ] Default to `sqlite` for persistence - - [ ] **SESS2.4** [Hamza] Surface warning if memory not persistent: - - [ ] On first CLI invocation, warn if memory backend is `memory` - - [ ] Suggest configuring persistent backend - - [ ] Tests: Memory persistence tests - - [ ] **SESS2.5** [Rui] Write Behave scenarios in `features/memory_persistence.feature`: - - [ ] Scenario: Conversation history survives CLI restart - - [ ] Scenario: Memory backend warning shown for in-memory mode - - [ ] Scenario: Plan-specific memory isolated from other plans + - [ ] Code: Fix memory loss between invocations + - [ ] **SESS2.1** [Hamza] Update `MemoryService` to use session-based storage: + - [ ] Store conversation history keyed by session_id + - [ ] Load history on session resume + - [ ] Support configurable history limits + - [ ] **SESS2.2** [Hamza] Create Alembic migration for `conversation_history` table: + - [ ] Schema: `history_id`, `session_id`, `plan_id`, `role`, `content`, `created_at` + - [ ] Foreign key to sessions + - [ ] Index on `session_id`, `plan_id` + - [ ] **SESS2.3** [Hamza] Add explicit memory configuration: + - [ ] `CLEVERAGENTS_MEMORY_BACKEND=sqlite|redis|memory` + - [ ] Document that `memory` backend loses history between invocations + - [ ] Default to `sqlite` for persistence + - [ ] **SESS2.4** [Hamza] Surface warning if memory not persistent: + - [ ] On first CLI invocation, warn if memory backend is `memory` + - [ ] Suggest configuring persistent backend + - [ ] Tests: Memory persistence tests + - [ ] **SESS2.5** [Rui] Write Behave scenarios in `features/memory_persistence.feature`: + - [ ] Scenario: Conversation history survives CLI restart + - [ ] Scenario: Memory backend warning shown for in-memory mode + - [ ] Scenario: Plan-specific memory isolated from other plans - [ ] **Stage PROV1: Provider Fixes** (Day 10-11) **[Luis]** - - [ ] Code: Fix provider issues - - [ ] **PROV1.1** [Luis] Remove FakeListLLM as default behavior: - - [ ] Audit where FakeListLLM is used outside tests - - [ ] Ensure production code never falls back to FakeListLLM - - [ ] If no provider configured, fail fast with clear message: - ``` - Error: No LLM provider configured. - Set OPENAI_API_KEY, ANTHROPIC_API_KEY, or configure an actor. - See: agents [--data-dir PATH] [--config-path PATH] help providers - ``` - - [ ] **PROV1.2** [Luis] Fix auto-debug hardcoded provider: - - [ ] Auto-debug currently hardcoded to OpenAI GPT-4 - - [ ] Change to use configured default actor - - [ ] Or: use action/actor system (auto-debug is just an action) - - [ ] **PROV1.3** [Luis] Verify OpenRouter implementation: - - [ ] OpenRouter listed in spec but may not be fully implemented - - [ ] Test OpenRouter adapter with real API - - [ ] Fix any issues found - - [ ] **PROV1.4** [Luis] Implement provider auto-detection: - - [ ] On startup, detect which API keys are configured - - [ ] Register only available providers - - [ ] Clear message about which providers are available: - ``` - Available providers: openai (gpt-4, gpt-3.5-turbo), anthropic (claude-3-opus) - Missing: google (GEMINI_API_KEY not set) - ``` - - [ ] Tests: Provider tests - - [ ] **PROV1.5** [Rui] Write Behave scenarios in `features/provider_fixes.feature`: - - [ ] Scenario: No provider configured produces clear error - - [ ] Scenario: Auto-debug uses configured actor not hardcoded - - [ ] Scenario: Provider detection shows available providers - - [ ] Scenario: FakeListLLM only used in test mode + - [ ] Code: Fix provider issues + - [ ] **PROV1.1** [Luis] Remove FakeListLLM as default behavior: + - [ ] Audit where FakeListLLM is used outside tests + - [ ] Ensure production code never falls back to FakeListLLM + - [ ] If no provider configured, fail fast with clear message: + ``` + Error: No LLM provider configured. + Set OPENAI_API_KEY, ANTHROPIC_API_KEY, or configure an actor. + See: agents [--data-dir PATH] [--config-path PATH] help providers + ``` + - [ ] **PROV1.2** [Luis] Fix auto-debug hardcoded provider: + - [ ] Auto-debug currently hardcoded to OpenAI GPT-4 + - [ ] Change to use configured default actor + - [ ] Or: use action/actor system (auto-debug is just an action) + - [ ] **PROV1.3** [Luis] Verify OpenRouter implementation: + - [ ] OpenRouter listed in spec but may not be fully implemented + - [ ] Test OpenRouter adapter with real API + - [ ] Fix any issues found + - [ ] **PROV1.4** [Luis] Implement provider auto-detection: + - [ ] On startup, detect which API keys are configured + - [ ] Register only available providers + - [ ] Clear message about which providers are available: + ``` + Available providers: openai (gpt-4, gpt-3.5-turbo), anthropic (claude-3-opus) + Missing: google (GEMINI_API_KEY not set) + ``` + - [ ] Tests: Provider tests + - [ ] **PROV1.5** [Rui] Write Behave scenarios in `features/provider_fixes.feature`: + - [ ] Scenario: No provider configured produces clear error + - [ ] Scenario: Auto-debug uses configured actor not hardcoded + - [ ] Scenario: Provider detection shows available providers + - [ ] Scenario: FakeListLLM only used in test mode - [ ] **Stage PROV2: Provider Fallback & Cost Controls** (Day 11-12) **[Luis]** - - [ ] Code: Implement cost controls and fallback - - [ ] **PROV2.1** [Luis] Add token tracking per plan: - - [ ] Track input tokens, output tokens per LLM call - - [ ] Aggregate by plan, phase, actor - - [ ] Store in plan metadata - - [ ] **PROV2.2** [Luis] Add cost estimation: - - [ ] Map model to token cost ($ per 1K tokens) - - [ ] Calculate estimated cost per call - - [ ] Aggregate plan total cost - - [ ] **PROV2.3** [Luis] Implement budget limits: - - [ ] Per-plan max cost limit - - [ ] Per-session max cost limit - - [ ] Global max cost limit - - [ ] Warn when approaching limit, fail when exceeded - - [ ] **PROV2.4** [Luis] Implement rate limiting: - - [ ] Per-actor max calls per minute - - [ ] Per-plan max retries - - [ ] Exponential backoff on rate limit errors - - [ ] **PROV2.5** [Luis] Implement provider fallback: - - [ ] If primary provider fails, try fallback provider - - [ ] Configurable fallback order - - [ ] Log fallback events - - [ ] Tests: Cost control tests - - [ ] **PROV2.6** [Rui] Write Behave scenarios in `features/cost_controls.feature`: - - [ ] Scenario: Plan cost tracked and reported - - [ ] Scenario: Plan exceeding budget limit fails - - [ ] Scenario: Rate limit triggers exponential backoff - - [ ] Scenario: Provider fallback on transient failure + - [ ] Code: Implement cost controls and fallback + - [ ] **PROV2.1** [Luis] Add token tracking per plan: + - [ ] Track input tokens, output tokens per LLM call + - [ ] Aggregate by plan, phase, actor + - [ ] Store in plan metadata + - [ ] **PROV2.2** [Luis] Add cost estimation: + - [ ] Map model to token cost ($ per 1K tokens) + - [ ] Calculate estimated cost per call + - [ ] Aggregate plan total cost + - [ ] **PROV2.3** [Luis] Implement budget limits: + - [ ] Per-plan max cost limit + - [ ] Per-session max cost limit + - [ ] Global max cost limit + - [ ] Warn when approaching limit, fail when exceeded + - [ ] **PROV2.4** [Luis] Implement rate limiting: + - [ ] Per-actor max calls per minute + - [ ] Per-plan max retries + - [ ] Exponential backoff on rate limit errors + - [ ] **PROV2.5** [Luis] Implement provider fallback: + - [ ] If primary provider fails, try fallback provider + - [ ] Configurable fallback order + - [ ] Log fallback events + - [ ] Tests: Cost control tests + - [ ] **PROV2.6** [Rui] Write Behave scenarios in `features/cost_controls.feature`: + - [ ] Scenario: Plan cost tracked and reported + - [ ] Scenario: Plan exceeding budget limit fails + - [ ] Scenario: Rate limit triggers exponential backoff + - [ ] Scenario: Provider fallback on transient failure --- @@ -9074,269 +9621,269 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Commands missing from initial plan** - [ ] **Stage CLI0: Core System Commands** (Day 10) **[Hamza]** - - [ ] Code: Implement core CLI metadata commands - - [ ] **CLI0.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] version`: - - [ ] Print CLI version and build metadata - - [ ] Include config path and data dir in verbose output - - [ ] **CLI0.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] info`: - - [ ] Show configuration summary (data dir, config path, providers, defaults) - - [ ] Show current session if present - - [ ] **CLI0.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] diagnostics`: - - [ ] Run self-checks (config readable, DB reachable, write access) - - [ ] Print warnings for missing providers or invalid config - - [ ] Tests: Core CLI command tests - - [ ] **CLI0.4** [Rui] Write Behave scenarios in `features/cli_core.feature`: - - [ ] Scenario: Version prints semantic version - - [ ] Scenario: Info prints config and data paths - - [ ] Scenario: Diagnostics reports missing providers + - [ ] Code: Implement core CLI metadata commands + - [ ] **CLI0.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] version`: + - [ ] Print CLI version and build metadata + - [ ] Include config path and data dir in verbose output + - [ ] **CLI0.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] info`: + - [ ] Show configuration summary (data dir, config path, providers, defaults) + - [ ] Show current session if present + - [ ] **CLI0.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] diagnostics`: + - [ ] Run self-checks (config readable, DB reachable, write access) + - [ ] Print warnings for missing providers or invalid config + - [ ] Tests: Core CLI command tests + - [ ] **CLI0.4** [Rui] Write Behave scenarios in `features/cli_core.feature`: + - [ ] Scenario: Version prints semantic version + - [ ] Scenario: Info prints config and data paths + - [ ] Scenario: Diagnostics reports missing providers - [ ] **Stage CLI1: Plan Interaction Commands** (Day 10-11) **[Hamza]** - - [ ] Code: Implement additional plan commands - - [ ] **CLI1.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan prompt ""`: - - [ ] Provide additional instructions to a stuck plan - - [ ] Works when plan is in errored state - - [ ] Resumes execution with new guidance - - [ ] **CLI1.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan diff `: - - [ ] Show diff of changes made by plan - - [ ] Works for plans in Execute or Apply phase - - [ ] Color-coded unified diff output - - [ ] **CLI1.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan diff --correction `: - - [ ] Compare old vs new after correction - - [ ] Show what changed between correction attempts - - [ ] **CLI1.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan artifacts `: - - [ ] List all artifacts produced by plan - - [ ] Show file paths, operation types, sizes - - [ ] Tests: Plan interaction CLI tests - - [ ] **CLI1.5** [Rui] Write Behave scenarios in `features/plan_interaction_cli.feature`: - - [ ] Scenario: Plan prompt resumes stuck plan - - [ ] Scenario: Plan diff shows color-coded output - - [ ] Scenario: Correction diff comparison works + - [ ] Code: Implement additional plan commands + - [ ] **CLI1.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan prompt ""`: + - [ ] Provide additional instructions to a stuck plan + - [ ] Works when plan is in errored state + - [ ] Resumes execution with new guidance + - [ ] **CLI1.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan diff `: + - [ ] Show diff of changes made by plan + - [ ] Works for plans in Execute or Apply phase + - [ ] Color-coded unified diff output + - [ ] **CLI1.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan diff --correction `: + - [ ] Compare old vs new after correction + - [ ] Show what changed between correction attempts + - [ ] **CLI1.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan artifacts `: + - [ ] List all artifacts produced by plan + - [ ] Show file paths, operation types, sizes + - [ ] Tests: Plan interaction CLI tests + - [ ] **CLI1.5** [Rui] Write Behave scenarios in `features/plan_interaction_cli.feature`: + - [ ] Scenario: Plan prompt resumes stuck plan + - [ ] Scenario: Plan diff shows color-coded output + - [ ] Scenario: Correction diff comparison works - [ ] **Stage CLI2: Configuration Commands** (Day 11-12) **[Hamza]** - - [ ] Code: Implement config commands - - [ ] **CLI2.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config set `: - - [ ] Set global configuration values - - [ ] Supported keys: `automation-level`, `default-actor`, `log-level` - - [ ] Persist to `~/.cleveragents/config.toml` - - [ ] **CLI2.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config get `: - - [ ] Get current configuration value - - [ ] Show source (default, file, environment) - - [ ] **CLI2.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config list`: - - [ ] List all configuration values - - [ ] Show current value and source - - [ ] **CLI2.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] providers list`: - - [ ] List available providers - - [ ] Show which are configured vs missing API keys - - [ ] Tests: Config CLI tests - - [ ] **CLI2.5** [Rui] Write Behave scenarios in `features/config_cli.feature`: - - [ ] Scenario: Config set persists value - - [ ] Scenario: Config get shows current value - - [ ] Scenario: Providers list shows available/missing + - [ ] Code: Implement config commands + - [ ] **CLI2.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config set `: + - [ ] Set global configuration values + - [ ] Supported keys: `automation-level`, `default-actor`, `log-level` + - [ ] Persist to `~/.cleveragents/config.toml` + - [ ] **CLI2.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config get `: + - [ ] Get current configuration value + - [ ] Show source (default, file, environment) + - [ ] **CLI2.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] config list`: + - [ ] List all configuration values + - [ ] Show current value and source + - [ ] **CLI2.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] providers list`: + - [ ] List available providers + - [ ] Show which are configured vs missing API keys + - [ ] Tests: Config CLI tests + - [ ] **CLI2.5** [Rui] Write Behave scenarios in `features/config_cli.feature`: + - [ ] Scenario: Config set persists value + - [ ] Scenario: Config get shows current value + - [ ] Scenario: Providers list shows available/missing - [ ] **Stage CLI3: Context Commands** (Day 12-13) **[Hamza]** - - [ ] Code: Implement project/actor context policy commands - - [ ] **CLI3.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project context set`: - - [ ] Flags: `--project `, `--policy hot|warm|cold`, `--hot-max-tokens ` - - [ ] `--hot-max-tokens` is a soft cap; allow null/omitted to disable - - [ ] LLM hard context limit can override soft cap - - [ ] **CLI3.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project context show`: - - [ ] Display current policy and hot/warm/cold sizing - - [ ] **CLI3.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] actor context set`: - - [ ] Reuse same arguments/behavior as project context set and legacy context commands - - [ ] **CLI3.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] actor context show`: - - [ ] Reuse same output format as project context show - - [ ] Tests: Context command tests - - [ ] **CLI3.5** [Rui] Write Behave scenarios in `features/context_cli.feature`: - - [ ] Scenario: Project context set updates policy - - [ ] Scenario: Project context show displays hot/warm/cold tiers - - [ ] Scenario: Actor context set mirrors project context args - - [ ] Scenario: Actor context show displays policy summary + - [ ] Code: Implement project/actor context policy commands + - [ ] **CLI3.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project context set`: + - [ ] Flags: `--project `, `--policy hot|warm|cold`, `--hot-max-tokens ` + - [ ] `--hot-max-tokens` is a soft cap; allow null/omitted to disable + - [ ] LLM hard context limit can override soft cap + - [ ] **CLI3.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] project context show`: + - [ ] Display current policy and hot/warm/cold sizing + - [ ] **CLI3.3** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] actor context set`: + - [ ] Reuse same arguments/behavior as project context set and legacy context commands + - [ ] **CLI3.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] actor context show`: + - [ ] Reuse same output format as project context show + - [ ] Tests: Context command tests + - [ ] **CLI3.5** [Rui] Write Behave scenarios in `features/context_cli.feature`: + - [ ] Scenario: Project context set updates policy + - [ ] Scenario: Project context show displays hot/warm/cold tiers + - [ ] Scenario: Actor context set mirrors project context args + - [ ] Scenario: Actor context show displays policy summary --- ### Section 14: Concurrency & Cleanup [Days 12-14] - [ ] **Stage CONC1: Plan Locking** (Day 12) **[Luis]** - - [ ] Code: Prevent concurrent plan modification - - [ ] **CONC1.1** [Luis] Implement plan-level locking: - - [ ] Database row lock on plan during execution - - [ ] Or: advisory lock using plan_id - - [ ] Prevent two processes from executing same plan - - [ ] **CONC1.2** [Luis] Implement project-level locking: - - [ ] Lock project during apply (can't apply two plans to same project) - - [ ] Allow parallel plans in different sandboxes - - [ ] **CONC1.3** [Luis] Add lock timeout and retry: - - [ ] Configurable lock wait timeout - - [ ] Clear error if lock cannot be acquired - - [ ] Tests: Concurrency tests - - [ ] **CONC1.4** [Rui] Write Behave scenarios in `features/concurrency.feature`: - - [ ] Scenario: Two processes cannot execute same plan - - [ ] Scenario: Two applies to same project blocked - - [ ] Scenario: Lock timeout produces clear error + - [ ] Code: Prevent concurrent plan modification + - [ ] **CONC1.1** [Luis] Implement plan-level locking: + - [ ] Database row lock on plan during execution + - [ ] Or: advisory lock using plan_id + - [ ] Prevent two processes from executing same plan + - [ ] **CONC1.2** [Luis] Implement project-level locking: + - [ ] Lock project during apply (can't apply two plans to same project) + - [ ] Allow parallel plans in different sandboxes + - [ ] **CONC1.3** [Luis] Add lock timeout and retry: + - [ ] Configurable lock wait timeout + - [ ] Clear error if lock cannot be acquired + - [ ] Tests: Concurrency tests + - [ ] **CONC1.4** [Rui] Write Behave scenarios in `features/concurrency.feature`: + - [ ] Scenario: Two processes cannot execute same plan + - [ ] Scenario: Two applies to same project blocked + - [ ] Scenario: Lock timeout produces clear error - [ ] **Stage CONC2: Resumable Execution** (Day 12-13) **[Luis]** - - [ ] Code: Implement plan resume capability - - [ ] **CONC2.1** [Luis] Persist step-level progress: - - [ ] Record each completed step in plan - - [ ] Store intermediate state for resume - - [ ] **CONC2.2** [Luis] Implement `agents [--data-dir PATH] [--config-path PATH] plan resume `: - - [ ] Detect where plan was interrupted - - [ ] Resume from last completed step - - [ ] Restore sandbox state - - [ ] **CONC2.3** [Luis] Handle graceful shutdown: - - [ ] On SIGINT/SIGTERM, save progress before exit - - [ ] Mark plan as "interrupted" not "errored" - - [ ] Tests: Resume tests - - [ ] **CONC2.4** [Rui] Write Behave scenarios in `features/plan_resume.feature`: - - [ ] Scenario: Interrupted plan can be resumed - - [ ] Scenario: Resume continues from correct step - - [ ] Scenario: Graceful shutdown saves progress + - [ ] Code: Implement plan resume capability + - [ ] **CONC2.1** [Luis] Persist step-level progress: + - [ ] Record each completed step in plan + - [ ] Store intermediate state for resume + - [ ] **CONC2.2** [Luis] Implement `agents [--data-dir PATH] [--config-path PATH] plan resume `: + - [ ] Detect where plan was interrupted + - [ ] Resume from last completed step + - [ ] Restore sandbox state + - [ ] **CONC2.3** [Luis] Handle graceful shutdown: + - [ ] On SIGINT/SIGTERM, save progress before exit + - [ ] Mark plan as "interrupted" not "errored" + - [ ] Tests: Resume tests + - [ ] **CONC2.4** [Rui] Write Behave scenarios in `features/plan_resume.feature`: + - [ ] Scenario: Interrupted plan can be resumed + - [ ] Scenario: Resume continues from correct step + - [ ] Scenario: Graceful shutdown saves progress - [ ] **Stage CONC3: Garbage Collection** (Day 13-14) **[Hamza]** - - [ ] Code: Cleanup abandoned resources - - [ ] **CONC3.1** [Hamza] Implement sandbox garbage collection: - - [ ] On startup, find orphaned sandbox directories - - [ ] Clean sandboxes from crashed processes - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] cleanup sandboxes` CLI command - - [ ] **CONC3.2** [Hamza] Implement checkpoint file cleanup: - - [ ] Track checkpoint files in database - - [ ] Retention policy (default: 7 days after plan completion) - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] cleanup checkpoints` CLI command - - [ ] **CONC3.3** [Hamza] Implement session cleanup: - - [ ] Clean sessions with no activity for 30 days - - [ ] Clean associated conversation history - - [ ] **CONC3.4** [Hamza] Add automatic cleanup on startup: - - [ ] Run cleanup for orphaned resources - - [ ] Log what was cleaned - - [ ] **CONC3.5** [Brent] Review resource lifecycle patterns only: - - [ ] Validate cleanup doesn't affect active resources - - [ ] Review concurrent access during cleanup - - [ ] Tests: Cleanup tests - - [ ] **CONC3.6** [Rui] Write Behave scenarios in `features/garbage_collection.feature`: - - [ ] Scenario: Orphaned sandbox cleaned on startup - - [ ] Scenario: Old checkpoints cleaned by retention policy - - [ ] Scenario: Cleanup commands work manually + - [ ] Code: Cleanup abandoned resources + - [ ] **CONC3.1** [Hamza] Implement sandbox garbage collection: + - [ ] On startup, find orphaned sandbox directories + - [ ] Clean sandboxes from crashed processes + - [ ] Add `agents [--data-dir PATH] [--config-path PATH] cleanup sandboxes` CLI command + - [ ] **CONC3.2** [Hamza] Implement checkpoint file cleanup: + - [ ] Track checkpoint files in database + - [ ] Retention policy (default: 7 days after plan completion) + - [ ] Add `agents [--data-dir PATH] [--config-path PATH] cleanup checkpoints` CLI command + - [ ] **CONC3.3** [Hamza] Implement session cleanup: + - [ ] Clean sessions with no activity for 30 days + - [ ] Clean associated conversation history + - [ ] **CONC3.4** [Hamza] Add automatic cleanup on startup: + - [ ] Run cleanup for orphaned resources + - [ ] Log what was cleaned + - [ ] **CONC3.5** [Brent] Review resource lifecycle patterns only: + - [ ] Validate cleanup doesn't affect active resources + - [ ] Review concurrent access during cleanup + - [ ] Tests: Cleanup tests + - [ ] **CONC3.6** [Rui] Write Behave scenarios in `features/garbage_collection.feature`: + - [ ] Scenario: Orphaned sandbox cleaned on startup + - [ ] Scenario: Old checkpoints cleaned by retention policy + - [ ] Scenario: Cleanup commands work manually --- ### Section 15: Definition of Done & Invariants [Days 14-15] - [ ] **Stage DOD1: Definition of Done Enforcement** (Day 14) **[Luis]** - - [ ] Code: Implement DoD validation - - [ ] **DOD1.1** [Luis] Parse DoD must/should/may structure: - - [ ] Parse action's `definition_of_done` field - - [ ] Identify MUST, SHOULD, MAY requirements - - [ ] Generate validation checklist from DoD - - [ ] **DOD1.2** [Luis] Validate DoD after execution: - - [ ] Run DoD checklist after Execute completes - - [ ] MUST requirements block apply if failed - - [ ] SHOULD requirements warn but allow apply - - [ ] MAY requirements are informational only - - [ ] **DOD1.3** [Luis] Display DoD validation results: - - [ ] Show checklist in CLI output - - [ ] Color-code: green (pass), red (fail), yellow (warn) - - [ ] Tests: DoD tests - - [ ] **DOD1.4** [Rui] Write Behave scenarios in `features/definition_of_done.feature`: - - [ ] Scenario: DoD MUST failure blocks apply - - [ ] Scenario: DoD SHOULD failure warns but allows apply - - [ ] Scenario: DoD validation results displayed + - [ ] Code: Implement DoD validation + - [ ] **DOD1.1** [Luis] Parse DoD must/should/may structure: + - [ ] Parse action's `definition_of_done` field + - [ ] Identify MUST, SHOULD, MAY requirements + - [ ] Generate validation checklist from DoD + - [ ] **DOD1.2** [Luis] Validate DoD after execution: + - [ ] Run DoD checklist after Execute completes + - [ ] MUST requirements block apply if failed + - [ ] SHOULD requirements warn but allow apply + - [ ] MAY requirements are informational only + - [ ] **DOD1.3** [Luis] Display DoD validation results: + - [ ] Show checklist in CLI output + - [ ] Color-code: green (pass), red (fail), yellow (warn) + - [ ] Tests: DoD tests + - [ ] **DOD1.4** [Rui] Write Behave scenarios in `features/definition_of_done.feature`: + - [ ] Scenario: DoD MUST failure blocks apply + - [ ] Scenario: DoD SHOULD failure warns but allows apply + - [ ] Scenario: DoD validation results displayed - [ ] **Stage DOD2: Invariant System** (Day 14-15) **[Luis]** - - [ ] Code: Implement user-defined invariants - - [ ] **DOD2.1** [Luis] Define `Invariant` model: - - [ ] Field `invariant_id: str` - - [ ] Field `project_id: str` - - [ ] Field `description: str` - human readable - - [ ] Field `check_command: str | None` - shell command to verify - - [ ] Field `check_code: str | None` - Python code to verify - - [ ] Field `severity: str` - error|warning - - [ ] **DOD2.2** [Luis] Implement `agents [--data-dir PATH] [--config-path PATH] project add-invariant`: - - [ ] Add invariant to project - - [ ] Specify check command or code - - [ ] **DOD2.3** [Luis] Check invariants during execution: - - [ ] Run invariant checks after each major step - - [ ] Fail execution if invariant violated (severity=error) - - [ ] Warn if invariant violated (severity=warning) - - [ ] Tests: Invariant tests - - [ ] **DOD2.4** [Rui] Write Behave scenarios in `features/invariants.feature`: - - [ ] Scenario: Invariant violation blocks execution - - [ ] Scenario: Invariant warning allows continuation - - [ ] Scenario: Invariant with command check works + - [ ] Code: Implement user-defined invariants + - [ ] **DOD2.1** [Luis] Define `Invariant` model: + - [ ] Field `invariant_id: str` + - [ ] Field `project_id: str` + - [ ] Field `description: str` - human readable + - [ ] Field `check_command: str | None` - shell command to verify + - [ ] Field `check_code: str | None` - Python code to verify + - [ ] Field `severity: str` - error|warning + - [ ] **DOD2.2** [Luis] Implement `agents [--data-dir PATH] [--config-path PATH] project add-invariant`: + - [ ] Add invariant to project + - [ ] Specify check command or code + - [ ] **DOD2.3** [Luis] Check invariants during execution: + - [ ] Run invariant checks after each major step + - [ ] Fail execution if invariant violated (severity=error) + - [ ] Warn if invariant violated (severity=warning) + - [ ] Tests: Invariant tests + - [ ] **DOD2.4** [Rui] Write Behave scenarios in `features/invariants.feature`: + - [ ] Scenario: Invariant violation blocks execution + - [ ] Scenario: Invariant warning allows continuation + - [ ] Scenario: Invariant with command check works --- ### Section 16: Context Indexing [Days 15-17] - [ ] **Stage CTX1: Repository Indexing** (Day 15-16) **[Hamza]** - - [ ] Code: Implement repo indexing for large codebases - - [ ] **CTX1.1** [Hamza] Create `IndexingService` in `src/cleveragents/application/services/indexing_service.py`: - - [ ] Method `index_project(project: Project) -> Index`: - - [ ] Scan all files in project resources - - [ ] Apply ignore patterns - - [ ] Build file tree index - - [ ] Detect language per file - - [ ] Method `search(query: str, project_id: str) -> list[SearchResult]`: - - [ ] Full-text search across indexed files - - [ ] Return file paths, line numbers, snippets - - [ ] Method `refresh_index(project_id: str) -> None`: - - [ ] Update index for changed files only - - [ ] **CTX1.2** [Hamza] Implement file tree representation: - - [ ] Parse directory structure - - [ ] Include file sizes, modification times - - [ ] Support efficient subtree queries - - [ ] **CTX1.3** [Hamza] Add language detection: - - [ ] Detect language from file extension - - [ ] Detect language from shebang/magic bytes - - [ ] Store language in index - - [ ] Tests: Indexing tests - - [ ] **CTX1.4** [Rui] Write Behave scenarios in `features/context_indexing.feature`: - - [ ] Scenario: Project indexing creates searchable index - - [ ] Scenario: Search returns relevant results - - [ ] Scenario: Index refresh updates changed files only + - [ ] Code: Implement repo indexing for large codebases + - [ ] **CTX1.1** [Hamza] Create `IndexingService` in `src/cleveragents/application/services/indexing_service.py`: + - [ ] Method `index_project(project: Project) -> Index`: + - [ ] Scan all files in project resources + - [ ] Apply ignore patterns + - [ ] Build file tree index + - [ ] Detect language per file + - [ ] Method `search(query: str, project_id: str) -> list[SearchResult]`: + - [ ] Full-text search across indexed files + - [ ] Return file paths, line numbers, snippets + - [ ] Method `refresh_index(project_id: str) -> None`: + - [ ] Update index for changed files only + - [ ] **CTX1.2** [Hamza] Implement file tree representation: + - [ ] Parse directory structure + - [ ] Include file sizes, modification times + - [ ] Support efficient subtree queries + - [ ] **CTX1.3** [Hamza] Add language detection: + - [ ] Detect language from file extension + - [ ] Detect language from shebang/magic bytes + - [ ] Store language in index + - [ ] Tests: Indexing tests + - [ ] **CTX1.4** [Rui] Write Behave scenarios in `features/context_indexing.feature`: + - [ ] Scenario: Project indexing creates searchable index + - [ ] Scenario: Search returns relevant results + - [ ] Scenario: Index refresh updates changed files only - [ ] **Stage CTX2: Embedding Index** (Day 16-17) **[Hamza]** - - [ ] Code: Optional embedding-based search - - [ ] **CTX2.1** [Hamza] Integrate with VectorStoreService: - - [ ] Chunk files into segments - - [ ] Generate embeddings for each chunk - - [ ] Store in FAISS index - - [ ] **CTX2.2** [Hamza] Implement semantic search: - - [ ] Method `semantic_search(query: str, project_id: str) -> list[SearchResult]` - - [ ] Use query embedding to find similar chunks - - [ ] Return ranked results with relevance scores - - [ ] **CTX2.3** [Hamza] Make embedding index optional: - - [ ] Only build if `CLEVERAGENTS_ENABLE_EMBEDDINGS=true` - - [ ] Fall back to full-text search if not available - - [ ] Tests: Embedding tests - - [ ] **CTX2.4** [Rui] Write Behave scenarios in `features/embedding_search.feature`: - - [ ] Scenario: Semantic search returns conceptually similar results - - [ ] Scenario: System works without embedding index + - [ ] Code: Optional embedding-based search + - [ ] **CTX2.1** [Hamza] Integrate with VectorStoreService: + - [ ] Chunk files into segments + - [ ] Generate embeddings for each chunk + - [ ] Store in FAISS index + - [ ] **CTX2.2** [Hamza] Implement semantic search: + - [ ] Method `semantic_search(query: str, project_id: str) -> list[SearchResult]` + - [ ] Use query embedding to find similar chunks + - [ ] Return ranked results with relevance scores + - [ ] **CTX2.3** [Hamza] Make embedding index optional: + - [ ] Only build if `CLEVERAGENTS_ENABLE_EMBEDDINGS=true` + - [ ] Fall back to full-text search if not available + - [ ] Tests: Embedding tests + - [ ] **CTX2.4** [Rui] Write Behave scenarios in `features/embedding_search.feature`: + - [ ] Scenario: Semantic search returns conceptually similar results + - [ ] Scenario: System works without embedding index --- ### Section 17: Skill Registry [Days 17-18] - [ ] **Stage SKILL1: Skill Catalog** (Day 17-18) **[Aditya]** - - [ ] Code: Implement skill registry for safety validation - - [ ] **SKILL1.1** [Aditya] Create `SkillRegistry` in `src/cleveragents/actor/skills/registry.py`: - - [ ] Method `register_skill(skill: Skill) -> None` - - [ ] Method `get_skill(name: str) -> Skill | None` - - [ ] Method `list_skills() -> list[Skill]` - - [ ] Method `list_skills_by_capability(read_only: bool) -> list[Skill]` - - [ ] **SKILL1.2** [Aditya] Auto-register skills from actor configs: - - [ ] When actor config parsed, extract tool definitions - - [ ] Register each tool as a skill with metadata - - [ ] **SKILL1.3** [Aditya] Validate skill usage against plan requirements: - - [ ] Check skill capabilities against action requirements - - [ ] Fail if incompatible skill used - - [ ] **SKILL1.4** [Aditya] Implement `agents [--data-dir PATH] [--config-path PATH] skills list`: - - [ ] List all registered skills - - [ ] Show capabilities (read_only, checkpointable, etc.) - - [ ] Tests: Skill registry tests - - [ ] **SKILL1.5** [Rui] Write Behave scenarios in `features/skill_registry.feature`: - - [ ] Scenario: Skills auto-registered from actor config - - [ ] Scenario: Skill capability query works - - [ ] Scenario: Incompatible skill usage detected + - [ ] Code: Implement skill registry for safety validation + - [ ] **SKILL1.1** [Aditya] Create `SkillRegistry` in `src/cleveragents/actor/skills/registry.py`: + - [ ] Method `register_skill(skill: Skill) -> None` + - [ ] Method `get_skill(name: str) -> Skill | None` + - [ ] Method `list_skills() -> list[Skill]` + - [ ] Method `list_skills_by_capability(read_only: bool) -> list[Skill]` + - [ ] **SKILL1.2** [Aditya] Auto-register skills from actor configs: + - [ ] When actor config parsed, extract tool definitions + - [ ] Register each tool as a skill with metadata + - [ ] **SKILL1.3** [Aditya] Validate skill usage against plan requirements: + - [ ] Check skill capabilities against action requirements + - [ ] Fail if incompatible skill used + - [ ] **SKILL1.4** [Aditya] Implement `agents [--data-dir PATH] [--config-path PATH] skills list`: + - [ ] List all registered skills + - [ ] Show capabilities (read_only, checkpointable, etc.) + - [ ] Tests: Skill registry tests + - [ ] **SKILL1.5** [Rui] Write Behave scenarios in `features/skill_registry.feature`: + - [ ] Scenario: Skills auto-registered from actor config + - [ ] Scenario: Skill capability query works + - [ ] Scenario: Incompatible skill usage detected --- @@ -9350,23 +9897,23 @@ The following items are deferred or no longer applicable: - [ ] **Deferred: Database Resources** - After source code resources work - [ ] **Deferred: Cloud Infrastructure Resources** - After source code resources work - [ ] **Deferred: Permission System** - Requires server connectivity (server is a separate project) - - [ ] Namespace-level permissions (who can create/edit org actions) - - [ ] Project-level permissions (who can modify resources, apply changes) - - [ ] Plan-level permissions (can this plan write, require approvals) - - [ ] Skill-level permissions (require approval per call or elevated role) + - [ ] Namespace-level permissions (who can create/edit org actions) + - [ ] Project-level permissions (who can modify resources, apply changes) + - [ ] Plan-level permissions (can this plan write, require approvals) + - [ ] Skill-level permissions (require approval per call or elevated role) - [ ] **POST1: Safety Profile Enforcement (Post-30)** - Deferred safety policy system - - [ ] Move `safety_profile` into Action model (from Stage A2 follow-up) - - [ ] Define `SafetyProfile` model with allow/deny skill categories and approval gates - - [ ] Add action-create CLI flags for safety profile: - - [ ] `--require-sandbox` - - [ ] `--require-checkpoints` - - [ ] `--require-apply-approval` - - [ ] `--allow-skill-category ` (repeatable) - - [ ] `--deny-skill-category ` (repeatable) - - [ ] `--max-cost-usd ` - - [ ] `--max-retries ` - - [ ] Enforce safety profile during execution and apply - - [ ] Add Behave + Robot tests for safety profile enforcement + - [ ] Move `safety_profile` into Action model (from Stage A2 follow-up) + - [ ] Define `SafetyProfile` model with allow/deny skill categories and approval gates + - [ ] Add action-create CLI flags for safety profile: + - [ ] `--require-sandbox` + - [ ] `--require-checkpoints` + - [ ] `--require-apply-approval` + - [ ] `--allow-skill-category ` (repeatable) + - [ ] `--deny-skill-category ` (repeatable) + - [ ] `--max-cost-usd ` + - [ ] `--max-retries ` + - [ ] Enforce safety profile during execution and apply + - [ ] Add Behave + Robot tests for safety profile enforcement - [ ] **Removed: Old 67-command structure** - Replaced by new command structure - [ ] **Removed: Configuration migration utilities** - CleverAgents is standalone @@ -9376,80 +9923,87 @@ The following items are deferred or no longer applicable: ### TEAM ROLES AND ASSIGNMENTS -| Developer | Role | Primary Focus Areas | Notes | -|-----------|------|---------------------|-------| -| **Jeff** | CTO/Lead Architect | Critical path items, architectural decisions, complex integrations | Fastest, most expert developer - handles blocking issues | -| **Luis** | Senior Python Architect | Domain models, persistence, algorithms, state machines | Good architecture but can be pedantic - needs clear requirements | -| **Aditya** | Domain Expert (Agents/LLMs) | Actor YAML configs, hierarchical actors, skill execution | Understands topic best but code may need cleanup | -| **Hamza** | Python/RDF Expert | Resources, sandbox, database, general Python | Well-rounded, no agent experience - assign infrastructure | -| **Rui** | Fast Developer | Testing (Behave/Robot), simpler implementations | New to Python - assign testing and straightforward tasks | -| **Brent** | Quality Specialist | Code review, linting, type checking, documentation | Slow but detail-oriented - low contention independent work | -| **Mike/Brian** | Sysadmins | Deployment, infrastructure setup | Minimal coding tasks | +| Developer | Role | Primary Focus Areas | Notes | +| -------------- | --------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------- | +| **Jeff** | CTO/Lead Architect | Critical path items, architectural decisions, complex integrations | Fastest, most expert developer - handles blocking issues | +| **Luis** | Senior Python Architect | Domain models, persistence, algorithms, state machines | Good architecture but can be pedantic - needs clear requirements | +| **Aditya** | Domain Expert (Agents/LLMs) | Actor YAML configs, hierarchical actors, skill execution | Understands topic best but code may need cleanup | +| **Hamza** | Python/RDF Expert | Resources, sandbox, database, general Python | Well-rounded, no agent experience - assign infrastructure | +| **Rui** | Fast Developer | Testing (Behave/Robot), simpler implementations | New to Python - assign testing and straightforward tasks | +| **Brent** | Quality Specialist | Code review, linting, type checking, documentation | Slow but detail-oriented - low contention independent work | +| **Mike/Brian** | Sysadmins | Deployment, infrastructure setup | Minimal coding tasks | ### Week 1 (Days 1-7) - MVP Target (Source Code Only) -| Day | Morning Focus | Owner | Afternoon Focus | Owner | -|-----|---------------|-------|-----------------|-------| -| 1 | A5.1-A5.4 Plan/Action DB Schema | Jeff + Luis | B1.1-B1.6 Project/Resource Models | Hamza | -| 2 | A5.5-A5.9 Plan/Action Repositories | Jeff | B2.1-B2.4 Project CLI Commands | Hamza + Rui (tests) | -| 3 | B3.1-B3.6 Sandbox Protocol + Git | Jeff + Hamza | B3.7-B3.13 Sandbox Manager + Tests | Luis + Rui | -| 4 | C1.1-C1.6 Actor YAML Schema | Aditya | C2.1-C2.8 Actor Compiler | Aditya + Jeff | -| 5 | C3.1-C3.5 Skill Protocol | Jeff | C3.6 Built-in File Skills | Luis + Jeff | -| 6 | C3.7 MCP Adapter | Aditya + Jeff | C4.1-C4.3 Change Tracking | Luis | -| 7 | C4.4-C4.7 Tool Router + Diff | Jeff | C5.1-C5.5 Validation Pipeline | Luis + Rui | + +| Day | Morning Focus | Owner | Afternoon Focus | Owner | +| --- | ---------------------------------- | ------------- | ---------------------------------- | ------------------- | +| 1 | A5.1-A5.4 Plan/Action DB Schema | Jeff + Luis | B1.1-B1.6 Project/Resource Models | Hamza | +| 2 | A5.5-A5.9 Plan/Action Repositories | Jeff | B2.1-B2.4 Project CLI Commands | Hamza + Rui (tests) | +| 3 | B3.1-B3.6 Sandbox Protocol + Git | Jeff + Hamza | B3.7-B3.13 Sandbox Manager + Tests | Luis + Rui | +| 4 | C1.1-C1.6 Actor YAML Schema | Aditya | C2.1-C2.8 Actor Compiler | Aditya + Jeff | +| 5 | C3.1-C3.5 Skill Protocol | Jeff | C3.6 Built-in File Skills | Luis + Jeff | +| 6 | C3.7 MCP Adapter | Aditya + Jeff | C4.1-C4.3 Change Tracking | Luis | +| 7 | C4.4-C4.7 Tool Router + Diff | Jeff | C5.1-C5.5 Validation Pipeline | Luis + Rui | ### Week 2 (Days 8-14) - M3 Complete + Plan-Actor Integration -| Day | Focus | Owner | Deliverable | -|-----|-------|-------|-------------| -| 8 | C6.1-C6.8 Plan-Actor Integration | Jeff + Aditya | Full execute phase working | -| 9 | C7.1-C7.6 Apply Phase + Diff Review | Jeff + Luis | Apply with review gates | -| 10 | D1.1-D1.8 Decision Model | Hamza + Jeff | Decision recording foundation | -| 11 | D2.1-D2.6 Decision Recording | Jeff + Hamza | Decisions captured in Strategize | -| 12 | E1.1-E1.6 Subplan Model | Luis | Subplan spawning design | -| 13 | E2.1-E2.5 Subplan Execution | Jeff + Luis | Sequential subplan execution | -| 14 | End-to-end integration testing | All + Rui | M3 milestone verified | + +| Day | Focus | Owner | Deliverable | +| --- | ----------------------------------- | ------------- | -------------------------------- | +| 8 | C6.1-C6.8 Plan-Actor Integration | Jeff + Aditya | Full execute phase working | +| 9 | C7.1-C7.6 Apply Phase + Diff Review | Jeff + Luis | Apply with review gates | +| 10 | D1.1-D1.8 Decision Model | Hamza + Jeff | Decision recording foundation | +| 11 | D2.1-D2.6 Decision Recording | Jeff + Hamza | Decisions captured in Strategize | +| 12 | E1.1-E1.6 Subplan Model | Luis | Subplan spawning design | +| 13 | E2.1-E2.5 Subplan Execution | Jeff + Luis | Sequential subplan execution | +| 14 | End-to-end integration testing | All + Rui | M3 milestone verified | ### Week 3 (Days 15-21) - M4 Target (Decision Tree + Correction) -| Day | Focus | Owner | Deliverable | -|-----|-------|-------|-------------| -| 15 | D3.1-D3.6 Decision Tree Storage | Hamza | Decision persistence | -| 16 | D4.1-D4.5 Decision CLI Commands | Hamza + Rui | `agents [--data-dir PATH] [--config-path PATH] plan tree`, `agents [--data-dir PATH] [--config-path PATH] plan explain` | -| 17 | D5.1-D5.8 Decision Correction | Jeff | `agents [--data-dir PATH] [--config-path PATH] plan correct` implementation | -| 18 | D5.9-D5.12 Replay Mechanism | Jeff + Luis | Downstream recomputation | -| 19 | E3.1-E3.5 Parallel Subplan Execution | Luis | Concurrent subplans | -| 20 | E4.1-E4.5 Result Merging | Jeff + Luis | Git-style merge for subplans | -| 21 | M4 integration testing | All | Decision correction working | + +| Day | Focus | Owner | Deliverable | +| --- | ------------------------------------ | ----------- | ----------------------------------------------------------------------------------------------------------------------- | +| 15 | D3.1-D3.6 Decision Tree Storage | Hamza | Decision persistence | +| 16 | D4.1-D4.5 Decision CLI Commands | Hamza + Rui | `agents [--data-dir PATH] [--config-path PATH] plan tree`, `agents [--data-dir PATH] [--config-path PATH] plan explain` | +| 17 | D5.1-D5.8 Decision Correction | Jeff | `agents [--data-dir PATH] [--config-path PATH] plan correct` implementation | +| 18 | D5.9-D5.12 Replay Mechanism | Jeff + Luis | Downstream recomputation | +| 19 | E3.1-E3.5 Parallel Subplan Execution | Luis | Concurrent subplans | +| 20 | E4.1-E4.5 Result Merging | Jeff + Luis | Git-style merge for subplans | +| 21 | M4 integration testing | All | Decision correction working | ### Week 4 (Days 22-30) - M6 Target (Large Project Autonomy - LOCAL MODE ONLY) -| Day | Focus | Owner | Deliverable | -|-----|-------|-------|-------------| -| 22-23 | F1.1-F1.8 Context Indexing | Hamza | Large codebase indexing | -| 24-25 | F2.1-F2.6 Hot/Warm/Cold Context | Jeff + Hamza | Three-tier memory | -| 26-27 | Deep Subplan Hierarchies (5+ levels) | Jeff + Luis | Autonomous decomposition | -| 28-29 | F0.1-F0.5 Server Client Interface Stubs + Large Project Tests | Luis + Rui | Client stubs (NOT server impl), 10K file tests | -| 30 | M6 integration testing | All | Large project autonomy verified (Server connectivity DEFERRED) | + +| Day | Focus | Owner | Deliverable | +| ----- | ------------------------------------------------------------- | ------------ | -------------------------------------------------------------- | +| 22-23 | F1.1-F1.8 Context Indexing | Hamza | Large codebase indexing | +| 24-25 | F2.1-F2.6 Hot/Warm/Cold Context | Jeff + Hamza | Three-tier memory | +| 26-27 | Deep Subplan Hierarchies (5+ levels) | Jeff + Luis | Autonomous decomposition | +| 28-29 | F0.1-F0.5 Server Client Interface Stubs + Large Project Tests | Luis + Rui | Client stubs (NOT server impl), 10K file tests | +| 30 | M6 integration testing | All | Large project autonomy verified (Server connectivity DEFERRED) | > **Note**: Server connectivity (F1-F4) is DEFERRED beyond Day 30. Days 26-29 focus on client **stubs only** and large project testing. The server is a separate project. ### Week 5 (Days 31-35) - M7 Target (Server Connectivity - Client Side Only) -| Day | Focus | Owner | Deliverable | -|-----|-------|-------|-------------| -| 31-32 | F1.1-F1.4 Server Client Infrastructure | Luis | HTTP client for server communication | -| 33 | F2.1-F2.6 Plan Sync Client | Luis | Client can sync plans to server | -| 34 | F3.1-F3.3 WebSocket Client | Luis | Client receives real-time updates | -| 35 | F4.1-F4.3 Remote Project Support | Hamza | Client can request server execution | + +| Day | Focus | Owner | Deliverable | +| ----- | -------------------------------------- | ----- | ------------------------------------ | +| 31-32 | F1.1-F1.4 Server Client Infrastructure | Luis | HTTP client for server communication | +| 33 | F2.1-F2.6 Plan Sync Client | Luis | Client can sync plans to server | +| 34 | F3.1-F3.3 WebSocket Client | Luis | Client receives real-time updates | +| 35 | F4.1-F4.3 Remote Project Support | Hamza | Client can request server execution | ### Week 6 (Days 36-40) - M8 Target (Full Feature Set + Polish) -| Day | Focus | Owner | Deliverable | -|-----|-------|-------|-------------| -| 36 | Automation level refinement | Jeff + Luis | G1 automation enhancements | -| 37 | Cost estimation actors | Aditya | G5 estimation working | -| 38 | Error recovery mechanisms | Jeff | G2 checkpointing + rollback | -| 39 | Performance optimization | Luis + Jeff | Benchmarks passing | -| 40 | Final integration + documentation | All | Release candidate ready | + +| Day | Focus | Owner | Deliverable | +| --- | --------------------------------- | ----------- | --------------------------- | +| 36 | Automation level refinement | Jeff + Luis | G1 automation enhancements | +| 37 | Cost estimation actors | Aditya | G5 estimation working | +| 38 | Error recovery mechanisms | Jeff | G2 checkpointing + rollback | +| 39 | Performance optimization | Luis + Jeff | Benchmarks passing | +| 40 | Final integration + documentation | All | Release candidate ready | ### Continuous Tasks (Throughout) **Brent (Quality - Independent, Low Contention)**: + - Review all PRs within 4 hours of submission - Run `nox -s typecheck` on all branches before merge - Run `nox -s lint` and ensure 0 warnings @@ -9458,6 +10012,7 @@ The following items are deferred or no longer applicable: - Security audit: no eval(), no template injection, no secrets in code **Rui (Testing - Parallel with Feature Work)**: + - Write Behave scenarios for each feature (before implementation starts) - Write Robot integration tests for each milestone - Run full test suite daily @@ -9489,13 +10044,13 @@ Day 22-30: F-G (Context + Server) ─────────────── ### Risk Mitigation -| Risk | Mitigation | Owner | -|------|------------|-------| -| Git worktree complexity | Jeff handles sandbox implementation | Jeff | -| Multi-file generation reliability | Extensive testing, fallback mechanisms | Luis + Rui | -| Decision tree correction bugs | Jeff reviews all correction logic | Jeff | -| Large codebase performance | Early profiling, lazy loading | Luis + Hamza | -| Server mode stability | Incremental rollout, feature flags | Jeff + Luis | +| Risk | Mitigation | Owner | +| --------------------------------- | -------------------------------------- | ------------ | +| Git worktree complexity | Jeff handles sandbox implementation | Jeff | +| Multi-file generation reliability | Extensive testing, fallback mechanisms | Luis + Rui | +| Decision tree correction bugs | Jeff reviews all correction logic | Jeff | +| Large codebase performance | Early profiling, lazy loading | Luis + Hamza | +| Server mode stability | Incremental rollout, feature flags | Jeff + Luis | ### Definition of Done (Each Task) @@ -9515,6 +10070,7 @@ Day 22-30: F-G (Context + Server) ─────────────── ### M1: MVP (Day 7) - Minimally Usable for Source Code **End-to-end verification command sequence:** + ```bash # 1. Create an action agents [--data-dir PATH] [--config-path PATH] action create \ @@ -9550,6 +10106,7 @@ cd /path/to/repo && git log -1 # Shows CleverAgents commit ``` **Technical Criteria:** + - [ ] Plan and Action records persist to SQLite database - [ ] Phase transitions (ACTION → STRATEGIZE → EXECUTE → APPLY → APPLIED) work correctly - [ ] Git worktree sandbox creates isolated working directory @@ -9561,6 +10118,7 @@ cd /path/to/repo && git log -1 # Shows CleverAgents commit ### M3: Full Plan Lifecycle with Actors (Day 14) **End-to-end verification:** + ```bash # Create actor YAML file cat > my_actor.yaml < # Should ``` **Technical Criteria:** + - [ ] Actor YAML files parse and validate correctly - [ ] Actors compile to LangGraph StateGraphs - [ ] Inline skill code executes in sandboxed environment @@ -9608,6 +10167,7 @@ agents [--data-dir PATH] [--config-path PATH] plan artifacts # Should ### M4: Decision Tree & Correction (Day 21) **End-to-end verification:** + ```bash # Execute a plan to generate decisions agents [--data-dir PATH] [--config-path PATH] plan use local/complex-action --project local/large-project @@ -9639,21 +10199,23 @@ agents [--data-dir PATH] [--config-path PATH] plan tree ``` **Technical Criteria:** + - [ ] Decisions recorded during Strategize with full context snapshot - [ ] Decision tree persists to database - [ ] `agents [--data-dir PATH] [--config-path PATH] plan tree` displays ASCII tree correctly - [ ] `agents [--data-dir PATH] [--config-path PATH] plan explain` shows all decision details - [ ] Correction in revert mode: - - [ ] Archives old decisions - - [ ] Rolls back sandbox to checkpoint - - [ ] Re-executes from decision point - - [ ] Generates new downstream decisions + - [ ] Archives old decisions + - [ ] Rolls back sandbox to checkpoint + - [ ] Re-executes from decision point + - [ ] Generates new downstream decisions - [ ] Correction in append mode creates fix subplan - [ ] History preserved for comparison ### M5: Subplans & Parallel Execution (Day 25) **End-to-end verification:** + ```bash # Execute plan that spawns multiple subplans agents [--data-dir PATH] [--config-path PATH] plan use local/refactor-action --project local/monorepo @@ -9675,6 +10237,7 @@ agents [--data-dir PATH] [--config-path PATH] plan diff # Shows merge ``` **Technical Criteria:** + - [ ] SUBPLAN_SPAWN decisions created during Strategize - [ ] Subplans actually spawned during Execute - [ ] Sequential subplan execution works (one at a time) @@ -9688,6 +10251,7 @@ agents [--data-dir PATH] [--config-path PATH] plan diff # Shows merge ### M6: Large Project Handling (Day 30) **End-to-end verification:** + ```bash # Index a large project (10,000+ files) agents [--data-dir PATH] [--config-path PATH] project create --name local/large-project @@ -9733,6 +10297,7 @@ agents [--data-dir PATH] [--config-path PATH] plan apply ``` **Technical Criteria:** + - [ ] Projects with 10,000+ files index without timeout - [ ] Context window management works (hot/warm/cold tiers) - [ ] Hierarchical decomposition creates 4+ levels of subplans @@ -9862,16 +10427,19 @@ DAY 30: M6 TARGET ⊕ **Server connectivity (WORKSTREAM F) is deferred beyond the 30-day timeline. The server is a separate project—this implementation covers the client only.** The CleverAgents executable (`agents`) is purely a **client application** that can: + 1. Run in stand-alone local-only mode (no server required) 2. Connect to an independently developed CleverAgents server for multi-user/collaborative features During Days 1-30, the following client stub infrastructure should be created: + - [ ] Server client connection command (`agents [--data-dir PATH] [--config-path PATH] connect ` - stub) - [ ] Abstract interfaces for client-to-server communication - [ ] Resource abstraction that can detect local vs remote resources - [ ] Placeholder client methods that return "Server connectivity not yet implemented" **What is NOT needed by Day 30:** + - Server implementation (the server is a separate project) - Full client-server API implementation - WebSocket client implementation diff --git a/scripts/run-semgrep.sh b/scripts/run-semgrep.sh deleted file mode 100755 index 81247299b..000000000 --- a/scripts/run-semgrep.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/sh -# Run semgrep if installed, skip gracefully if not -if command -v semgrep >/dev/null 2>&1; then - semgrep --config=.semgrep.yml --error --quiet src/ -else - echo "semgrep not installed, skipping (install with: pip install semgrep)" -fi