From a7d702e85d148b148232c5ea579dac1f1561ebcd Mon Sep 17 00:00:00 2001 From: Jeffrey Phillips Freeman Date: Wed, 11 Feb 2026 20:25:33 -0500 Subject: [PATCH] Docs: Updated implementation plan with new plan plus added asv requirements to commits --- implementation_plan.md | 5986 +++++++--------------------------------- 1 file changed, 962 insertions(+), 5024 deletions(-) diff --git a/implementation_plan.md b/implementation_plan.md index 422cba2b5..a8d1c17fb 100644 --- a/implementation_plan.md +++ b/implementation_plan.md @@ -1298,20 +1298,49 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target - [X] Add argument parsing for action parameters (`--arg name:type:required|optional:description`) - [X] Location: `src/cleveragents/domain/models/core/action.py` - [X] Tests: Behave scenarios for action model validation (22 scenarios in `features/action_model.feature`) - - [ ] **A2.1** [Luis] Extend Action model with additional fields (follow-up): - - [ ] Field `estimation_actor: str | None` - optional actor for cost/risk estimation - - [ ] Field `review_actor: str | None` - optional actor for code review - - [ ] Field `safety_profile: SafetyProfile | None` - safety constraints (DEFERRED to post-30; see Stage POST1) - - [ ] **A2.2** [Luis] Define `SafetyProfile` model (DEFERRED to post-30; see Stage POST1): - - [ ] Field `allowed_skill_categories: list[str] | None` - whitelist of skill types - - [ ] Field `require_checkpoints: bool` - require checkpointable skills - - [ ] Field `require_sandbox: bool` - require sandbox for all resources - - [ ] Field `require_human_approval: bool` - require approval at Apply - - [ ] Field `max_cost_usd: float | None` - budget cap - - [ ] Field `max_retries: int` - maximum retry attempts - - [ ] **A2.3** [Rui] Write tests for extended action model (DEFERRED to post-30; see Stage POST1): - - [ ] Scenario: Action with estimation_actor validates correctly - - [ ] Scenario: Safety profile enforced during execution + **Parallel Group A2b: Action/Plan Spec Alignment (M1-critical)** + **PARALLEL SUBTRACK A2b.alpha [Jeff]**: Action model alignment + invariants/automation metadata + **PARALLEL SUBTRACK A2b.beta [Luis]**: Plan metadata alignment + action linkage + **PARALLEL SUBTRACK A2b.gamma [Aditya]**: Action YAML schema + examples (config-first) + **SEQUENTIAL MERGE NOTE**: A2b.alpha + A2b.beta must land before A4b CLI wiring. + - [ ] **COMMIT (Owner: Jeff | Group: A2b.alpha) - Commit message: "feat(domain): align action metadata with invariants and automation profiles"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Update `src/cleveragents/domain/models/core/action.py` docstring to state actions are defined via YAML config and registered via CLI (remove "NOT YAML" wording). + - [ ] Code [Jeff]: Add `automation_profile` field (namespaced name string) to `Action` and validate `/` format + `local/` default handling. + - [ ] Code [Jeff]: Add `invariant_actor` (optional actor ref) and `invariants` list (action-scoped) with trimming, de-duplication, and empty-string rejection. + - [ ] Code [Jeff]: Add `definition_of_done_template` to preserve the pre-rendered DoD string before arg substitution; keep `definition_of_done` as rendered output. + - [ ] Code [Jeff]: Extend `ActionArgument` validation to enforce `min_value <= max_value`, regex only for string args, and default value type checks. + - [ ] Code [Jeff]: Add `Action.to_template_context()` (or equivalent) to generate deterministic arg context for templating. + - [ ] Docs [Jeff]: Update or create `docs/reference/action_model.md` with new fields, examples, and invariants/automation profile semantics. + - [ ] Tests (Behave) [Rui]: Add scenarios in `features/action_model.feature` for invariants validation, automation profile parsing, and definition_of_done_template retention. + - [ ] Tests (Robot) [Rui]: Add Robot scenario that loads action YAML and asserts invariants/automation profile fields are surfaced in CLI output (wired in A4b). + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/action_model_bench.py` to benchmark action argument parsing + template rendering. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(domain): align action metadata with invariants and automation profiles"`. + - [ ] **COMMIT (Owner: Luis | Group: A2b.beta) - Commit message: "feat(domain): align plan metadata with action linkage and automation profiles"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Add `action_name` (namespaced name string) and `action_id` (ULID string) to `Plan` for traceability to the originating action. + - [ ] Code [Luis]: Add `automation_profile`, `invariant_actor`, and `invariants` fields to `Plan` with source tags (action/project/plan/global). + - [ ] Code [Luis]: Add `arguments` map (validated JSON-serializable values) and `definition_of_done_template` capture to the plan model for later re-rendering. + - [ ] Code [Luis]: Add execution metadata placeholders: `changeset_id`, `sandbox_refs`, `validation_summary`, and `decision_root_id` (optional until later stages). + - [ ] Code [Luis]: Enforce automation profile immutability after `plan use` and when phase progresses beyond Strategize. + - [ ] Docs [Luis]: Update `docs/reference/plan_model.md` to document new fields, immutability rules, and action linkage. + - [ ] Tests (Behave) [Rui]: Add scenarios in `features/plan_model.feature` for automation profile lock, invariant persistence, and action linkage fields. + - [ ] Tests (Robot) [Rui]: Add Robot scenario to inspect `plan status` output for action linkage and automation profile fields (once CLI aligned). + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_model_bench.py` for plan validation and serialization. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(domain): align plan metadata with action linkage and automation profiles"`. + - [ ] **COMMIT (Owner: Aditya | Group: A2b.gamma) - Commit message: "docs(action): add action YAML schema and examples"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Docs [Aditya]: Author `docs/schema/action.schema.yaml` with versioning, required fields, and explicit type constraints for actors, invariants, and arguments. + - [ ] Docs [Aditya]: Add example action configs under `examples/actions/` (simple, invariant-heavy, multi-project, and estimation-actor examples). + - [ ] Code [Aditya]: Add schema validation helper in `src/cleveragents/action/schema.py` that loads YAML, validates schema version, and returns typed data. + - [ ] Code [Aditya]: Add clear error messages for missing required fields and invalid namespaced names. + - [ ] Tests (Behave) [Rui]: Add scenarios that load each example YAML and assert schema validation passes; add invalid schema cases. + - [ ] Tests (Robot) [Rui]: Add a Robot smoke test that reads example YAML files and reports parse success/failure. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/action_schema_bench.py` for YAML schema validation throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "docs(action): add action YAML schema and examples"`. - [x] **Stage A3: Plan State Machine** (Day 1-2) - COMPLETED 2026-02-05 - [x] Code: Implement plan lifecycle state machine @@ -1339,545 +1368,191 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target - [X] `agents [--data-dir PATH] [--config-path PATH] plan cancel ` - cancel non-terminal plan - [X] Location: `src/cleveragents/cli/commands/action.py`, `src/cleveragents/cli/commands/plan.py` - [X] Tests: Behave tests for action CLI (15 scenarios in `features/action_cli.feature`) - - [ ] Tests: Behave tests for plan lifecycle CLI commands (pending) - - **[Rui]** Write 20 Behave scenarios in `features/plan_lifecycle_cli.feature` covering: - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with valid action and project - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with missing project error - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with invalid action error - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan use` with argument validation - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on strategize-complete plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan execute` on non-strategize plan (error case) - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on execute-complete plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan apply` on non-execute plan (error case) - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan status` output format verification - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by phase - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan list` filtering by state - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on active plan - - [ ] `agents [--data-dir PATH] [--config-path PATH] plan cancel` on already-applied plan (error case) - - [ ] Tests: Robot integration tests for CLI commands (pending) - - **[Rui]** Write Robot test suite `robot/plan_lifecycle_cli.robot` for end-to-end CLI testing + **Parallel Group A4b: Action/Plan CLI Spec Alignment + Tests (M1-critical)** + **PARALLEL SUBTRACK A4b.alpha [Jeff]**: CLI feature alignment + **PARALLEL SUBTRACK A4b.beta [Rui]**: Behave + Robot coverage + - [ ] **COMMIT (Owner: Jeff | Group: A4b.alpha) - Commit message: "feat(cli): support action create from YAML config"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add `--config/-c` to `agents action create`; load YAML via `action/schema.py` and fail fast on schema violations. + - [ ] Code [Jeff]: Implement override precedence (CLI flags override YAML fields; explicit CLI empty string clears YAML value). + - [ ] Code [Jeff]: Normalize namespaced names, actor refs, and argument definitions from YAML into `ActionArgument` objects. + - [ ] Code [Jeff]: Add `--update` guard (if action exists) or explicit error per spec; ensure idempotent update path is clear. + - [ ] Docs [Jeff]: Update CLI reference with YAML-based action creation + override examples and failure messages. + - [ ] Tests (Behave) [Rui]: Add scenarios in `features/action_cli.feature` covering valid config, overrides, invalid schema, and missing required fields. + - [ ] Tests (Robot) [Rui]: Add `robot/action_cli_from_config.robot` with end-to-end CLI flow and output assertions. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/action_cli_config_bench.py` for config parsing + normalization. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(cli): support action create from YAML config"`. + - [ ] **COMMIT (Owner: Jeff | Group: A4b.alpha) - Commit message: "feat(cli): extend plan use with invariants and automation profile"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add `--automation-profile`, `--invariant`, and `--invariant-actor` flags to `agents plan use` and plumb into PlanLifecycleService. + - [ ] Code [Jeff]: Resolve action name -> action_id, validate automation profile existence, and attach plan-scoped invariants. + - [ ] Code [Jeff]: Persist resolved invariants + profile in Plan metadata for later Strategize/Execute steps. + - [ ] Docs [Jeff]: Update `docs/reference/plan_cli.md` with examples for profile + invariants and error cases. + - [ ] Tests (Behave) [Rui]: Add scenarios in `features/plan_lifecycle_cli.feature` covering profile selection, invariant validation, and multiple projects. + - [ ] Tests (Robot) [Rui]: Add Robot tests for plan use with invariants and automation profiles (positive + negative cases). + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_use_cli_bench.py` for argument parsing and validation. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(cli): extend plan use with invariants and automation profile"`. + - [ ] **COMMIT (Owner: Rui | Group: A4b.beta) - Commit message: "test(cli): add plan lifecycle Behave and Robot coverage"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Tests (Behave) [Rui]: Add full coverage for `plan execute`, `plan apply`, `plan status`, `plan list`, `plan cancel` (success + error paths). + - [ ] Tests (Robot) [Rui]: Add `robot/plan_lifecycle_cli.robot` covering end-to-end lifecycle transitions. + - [ ] Docs [Rui]: Update `docs/development/testing.md` with new CLI suites and command mappings. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_cli_smoke_bench.py` for CLI argument parsing overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "test(cli): add plan lifecycle Behave and Robot coverage"`. -- [ ] **Stage A5: Plan Persistence** (Day 1-2) **[Jeff + Luis - Critical Path]** - - **PARALLEL SUBTRACK A5.alpha [Jeff - Day 1 AM]**: Database Schema (A5.1, A5.2) - **PARALLEL SUBTRACK A5.beta [Luis - Day 1 AM]**: SQLAlchemy Models (A5.3, A5.4) - can start with schema design doc - **SEQUENTIAL AFTER alpha+beta [Jeff - Day 1 PM]**: Repository Implementation (A5.5, A5.6) - **SEQUENTIAL AFTER repos [Jeff - Day 2 AM]**: Service Integration (A5.7, A5.8) - **PARALLEL CONTINUOUS [Rui - Day 1-2]**: Test Writing (A5.9, A5.10, A5.11) - - - [ ] Code: Plan database schema and repository - - [ ] **A5.1** [Jeff] Create Alembic migration for `lifecycle_plans` table in `alembic/versions/xxx_add_lifecycle_plans.py`: - - [ ] **A5.1a** [Jeff] Create migration file with `revision` and `down_revision` links: - - [ ] Run `alembic revision -m "add_lifecycle_plans_table"` to generate file - - [ ] Verify revision ID is unique - - [ ] Set `down_revision` to point to previous migration (likely actions table) - - [ ] Commit: "feat(db): add lifecycle_plans migration scaffold" - - [ ] **A5.1b** [Jeff] Define `lifecycle_plans` table schema in upgrade() function: - - [ ] Column `plan_id` TEXT PRIMARY KEY (ULID format, validated at application layer) - - [ ] Column `parent_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - for subplan hierarchy - - [ ] Column `root_plan_id` TEXT NULLABLE FK references lifecycle_plans(plan_id) - always points to topmost plan - - [ ] Column `action_id` TEXT NOT NULL FK references actions(action_id) - the action template used - - [ ] Column `phase` TEXT NOT NULL CHECK(phase IN ('ACTION','STRATEGIZE','EXECUTE','APPLY','APPLIED')) - lifecycle phase - - [ ] Column `state` TEXT NOT NULL - processing state within phase (available/draft/archived for ACTION; queued/processing/errored/complete/cancelled for others) - - [ ] Column `attempt` INTEGER NOT NULL DEFAULT 1 - increments on re-execution after correction - - [ ] Column `automation_level` TEXT NOT NULL DEFAULT 'manual' CHECK(automation_level IN ('manual','review_before_apply','full_automation')) - - [ ] Column `project_ids` TEXT NOT NULL - JSON array of project ULIDs this plan operates on - - [ ] Column `arguments` TEXT NULLABLE - JSON object mapping argument name to provided value - - [ ] Column `strategy_context` TEXT NULLABLE - JSON blob storing Strategize phase outputs (strategy, execution blueprint, resource queries) - - [ ] Column `execution_log` TEXT NULLABLE - JSON array of execution events [{timestamp, event_type, details}] - - [ ] Column `changeset_id` TEXT NULLABLE FK references changesets(changeset_id) - link to generated changes - - [ ] Column `sandbox_refs` TEXT NULLABLE - JSON object mapping resource_id to sandbox_path - - [ ] Column `error_message` TEXT NULLABLE - last error message if state is errored - - [ ] Column `created_at` TEXT NOT NULL - ISO8601 timestamp of plan creation - - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 timestamp of last modification - - [ ] Column `completed_at` TEXT NULLABLE - ISO8601 timestamp when plan reached terminal state - - [ ] Column `created_by` TEXT NULLABLE - user/session identifier who created the plan - - [ ] Commit: "feat(db): define lifecycle_plans table columns" - - [ ] **A5.1c** [Jeff] Create indices for common queries: - - [ ] Index `ix_lifecycle_plans_phase` on `phase` - for phase-based filtering - - [ ] Index `ix_lifecycle_plans_state` on `state` - for state-based filtering - - [ ] Index `ix_lifecycle_plans_parent` on `parent_plan_id` - for subplan lookups - - [ ] Index `ix_lifecycle_plans_root` on `root_plan_id` - for full tree queries - - [ ] Index `ix_lifecycle_plans_created` on `created_at` - for recent plans - - [ ] Index `ix_lifecycle_plans_action` on `action_id` - for action usage lookups - - [ ] Index `ix_lifecycle_plans_project` on `project_ids` - for project-based queries (use json_extract if needed) - - [ ] Commit: "feat(db): add lifecycle_plans indices" - - [ ] **A5.1d** [Jeff] Define foreign key ON DELETE behaviors: - - [ ] parent_plan_id: ON DELETE SET NULL - orphan subplans if parent deleted (preserve for debugging) - - [ ] root_plan_id: ON DELETE SET NULL - same reasoning - - [ ] action_id: ON DELETE RESTRICT - cannot delete action if plans exist using it - - [ ] changeset_id: ON DELETE SET NULL - preserve plan record even if changeset cleaned up - - [ ] Commit: "feat(db): define lifecycle_plans FK constraints" - - [ ] **A5.1e** [Jeff] Write `downgrade()` function to drop table: - - [ ] Drop all indices first - - [ ] Drop the lifecycle_plans table - - [ ] Verify downgrade works with `alembic downgrade -1` - - [ ] Commit: "feat(db): add lifecycle_plans downgrade function" - - [ ] **A5.2** [Jeff] Create Alembic migration for `actions` table in `alembic/versions/xxx_add_actions.py`: - - [ ] **A5.2a** [Jeff] Create migration file: - - [ ] Run `alembic revision -m "add_actions_table"` - - [ ] This migration MUST run BEFORE lifecycle_plans (set down_revision appropriately) - - [ ] Commit: "feat(db): add actions migration scaffold" - - [ ] **A5.2b** [Jeff] Define `actions` table schema: - - [ ] Column `action_id` TEXT PRIMARY KEY - ULID format - - [ ] Column `name` TEXT NOT NULL - full namespaced name (e.g., "local/code-coverage", "myorg/deploy-action") - - [ ] Column `namespace` TEXT NOT NULL - extracted namespace portion for filtering (e.g., "local", "myorg") - - [ ] Column `short_name` TEXT NOT NULL - extracted name portion after namespace (e.g., "code-coverage") - - [ ] Column `description` TEXT NULLABLE - human-readable description - - [ ] Column `definition_of_done` TEXT NOT NULL - explicit testable completion criteria (must/should/may format) - - [ ] Column `strategy_actor` TEXT NOT NULL - namespaced actor reference for Strategize phase (e.g., "local/coverage-strategist") - - [ ] Column `execution_actor` TEXT NOT NULL - namespaced actor reference for Execute phase - - [ ] Column `estimation_actor` TEXT NULLABLE - optional actor for cost/risk estimation (runs after Strategize) - - [ ] Column `review_actor` TEXT NULLABLE - optional actor for code review - - [ ] Column `inputs_schema` TEXT NOT NULL DEFAULT '[]' - JSON array of ActionArgument definitions - - [ ] Column `state` TEXT NOT NULL DEFAULT 'draft' CHECK(state IN ('available','draft','archived')) - - [ ] Column `reusable` BOOLEAN NOT NULL DEFAULT TRUE - if false, action self-deletes after first use - - [ ] Column `read_only` BOOLEAN NOT NULL DEFAULT FALSE - if true, only read-only skills allowed - - [ ] Column `safety_profile` TEXT NULLABLE - JSON object for SafetyProfile constraints (DEFERRED to post-30; see POST1) - - [ ] Column `created_at` TEXT NOT NULL - ISO8601 creation timestamp - - [ ] Column `updated_at` TEXT NOT NULL - ISO8601 last modification timestamp - - [ ] Commit: "feat(db): define actions table columns" - - [ ] **A5.2c** [Jeff] Create indices: - - [ ] UNIQUE index on `name` - enforce unique namespaced names - - [ ] Index `ix_actions_namespace` on `namespace` - for namespace filtering - - [ ] Index `ix_actions_state` on `state` - for state filtering - - [ ] Index `ix_actions_short_name` on `short_name` - for partial name searches - - [ ] Commit: "feat(db): add actions indices" - - [ ] **A5.2d** [Jeff] Write `downgrade()` function: - - [ ] Drop indices and table - - [ ] Verify with `alembic downgrade -1` - - [ ] Commit: "feat(db): add actions downgrade function" - - [ ] **A5.3** [Luis] Create `LifecyclePlanModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **A5.3a** [Luis] Define class structure: - - [ ] Create class `LifecyclePlanModel(Base)` with `__tablename__ = 'lifecycle_plans'` - - [ ] Import necessary SQLAlchemy types: `Column, String, Integer, Boolean, Text, ForeignKey, DateTime` - - [ ] Import relationship types: `relationship, backref` - - [ ] Commit: "feat(models): add LifecyclePlanModel class scaffold" - - [ ] **A5.3b** [Luis] Define all columns matching migration schema: - - [ ] `plan_id = Column(String(26), primary_key=True)` - ULID is 26 chars - - [ ] `parent_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` - - [ ] `root_plan_id = Column(String(26), ForeignKey('lifecycle_plans.plan_id', ondelete='SET NULL'), nullable=True)` - - [ ] `action_id = Column(String(26), ForeignKey('actions.action_id', ondelete='RESTRICT'), nullable=False)` - - [ ] `phase = Column(String(20), nullable=False)` - enum handled at domain layer - - [ ] `state = Column(String(20), nullable=False)` - - [ ] `attempt = Column(Integer, nullable=False, default=1)` - - [ ] `automation_level = Column(String(30), nullable=False, default='manual')` - - [ ] `project_ids = Column(Text, nullable=False)` - JSON string - - [ ] `arguments = Column(Text, nullable=True)` - JSON string - - [ ] `strategy_context = Column(Text, nullable=True)` - large JSON blob - - [ ] `execution_log = Column(Text, nullable=True)` - JSON array - - [ ] `changeset_id = Column(String(26), nullable=True)` - - [ ] `sandbox_refs = Column(Text, nullable=True)` - JSON object - - [ ] `error_message = Column(Text, nullable=True)` - - [ ] `created_at = Column(String(30), nullable=False)` - ISO8601 - - [ ] `updated_at = Column(String(30), nullable=False)` - - [ ] `completed_at = Column(String(30), nullable=True)` - - [ ] `created_by = Column(String(255), nullable=True)` - - [ ] Commit: "feat(models): define LifecyclePlanModel columns" - - [ ] **A5.3c** [Luis] Define relationships: - - [ ] `parent_plan = relationship('LifecyclePlanModel', remote_side=[plan_id], backref='children', foreign_keys=[parent_plan_id])` - - [ ] `action = relationship('ActionModel', backref='plans')` - - [ ] NOTE: root_plan relationship not needed as query pattern is different - - [ ] Commit: "feat(models): define LifecyclePlanModel relationships" - - [ ] **A5.3d** [Luis] Implement `to_domain() -> Plan` method: - - [ ] Import `Plan, PlanPhase, ProcessingState, AutomationLevel` from domain - - [ ] Convert `phase` string to `PlanPhase` enum: `PlanPhase[self.phase]` - - [ ] Convert `state` string to appropriate state enum based on phase - - [ ] Parse `project_ids` JSON: `json.loads(self.project_ids)` with error handling - - [ ] Parse `arguments` JSON if not None: `json.loads(self.arguments) if self.arguments else None` - - [ ] Parse `strategy_context` JSON if not None - - [ ] Parse `execution_log` JSON if not None - - [ ] Parse `sandbox_refs` JSON if not None - - [ ] Convert timestamp strings to `datetime.fromisoformat()` objects - - [ ] Construct and return `Plan(plan_id=self.plan_id, ...)` - - [ ] Add comprehensive docstring explaining the conversion - - [ ] Commit: "feat(models): implement LifecyclePlanModel.to_domain()" - - [ ] **A5.3e** [Luis] Implement classmethod `from_domain(plan: Plan) -> LifecyclePlanModel`: - - [ ] Add `@classmethod` decorator - - [ ] Convert `plan.phase.name` to string for phase column - - [ ] Convert state enum `.name` to string - - [ ] Serialize `project_ids` to JSON: `json.dumps(plan.project_ids)` - - [ ] Serialize `arguments` to JSON if not None - - [ ] Serialize `strategy_context` to JSON if not None (handle nested objects) - - [ ] Serialize `execution_log` to JSON if not None - - [ ] Serialize `sandbox_refs` to JSON if not None - - [ ] Convert datetime objects to `.isoformat()` strings - - [ ] Return constructed `LifecyclePlanModel` instance - - [ ] Commit: "feat(models): implement LifecyclePlanModel.from_domain()" - - [ ] **A5.4** [Luis] Create `ActionModel` SQLAlchemy model in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **A5.4a** [Luis] Define class structure and columns: - - [ ] Create class `ActionModel(Base)` with `__tablename__ = 'actions'` - - [ ] `action_id = Column(String(26), primary_key=True)` - - [ ] `name = Column(String(255), nullable=False, unique=True)` - - [ ] `namespace = Column(String(100), nullable=False)` - - [ ] `short_name = Column(String(150), nullable=False)` - - [ ] `description = Column(Text, nullable=True)` - - [ ] `definition_of_done = Column(Text, nullable=False)` - - [ ] `strategy_actor = Column(String(255), nullable=False)` - - [ ] `execution_actor = Column(String(255), nullable=False)` - - [ ] `estimation_actor = Column(String(255), nullable=True)` - - [ ] `review_actor = Column(String(255), nullable=True)` - - [ ] `inputs_schema = Column(Text, nullable=False, default='[]')` - - [ ] `state = Column(String(20), nullable=False, default='draft')` - - [ ] `reusable = Column(Boolean, nullable=False, default=True)` - - [ ] `read_only = Column(Boolean, nullable=False, default=False)` - - [ ] `safety_profile = Column(Text, nullable=True)` (DEFERRED to post-30; see POST1) - - [ ] `created_at = Column(String(30), nullable=False)` - - [ ] `updated_at = Column(String(30), nullable=False)` - - [ ] Commit: "feat(models): define ActionModel columns" - - [ ] **A5.4b** [Luis] Implement `to_domain() -> Action` method: - - [ ] Import `Action, ActionState, ActionArgument` from domain - - [ ] Convert `state` string to `ActionState` enum - - [ ] Parse `inputs_schema` JSON and convert to `list[ActionArgument]` - - [ ] Parse `safety_profile` JSON if present to `SafetyProfile` or None (DEFERRED to post-30; see POST1) - - [ ] Convert timestamps to datetime objects - - [ ] Construct and return `Action` instance - - [ ] Commit: "feat(models): implement ActionModel.to_domain()" - - [ ] **A5.4c** [Luis] Implement classmethod `from_domain(action: Action) -> ActionModel`: - - [ ] Extract namespace and short_name from action.name using `NamespacedName.parse()` - - [ ] Serialize `inputs_schema` to JSON from list of ActionArgument (call `.model_dump()` on each) - - [ ] Serialize `safety_profile` to JSON if present (DEFERRED to post-30; see POST1) - - [ ] Convert timestamps to ISO8601 strings - - [ ] Return constructed `ActionModel` instance - - [ ] Commit: "feat(models): implement ActionModel.from_domain()" - - [ ] **A5.5** [Jeff] Implement `LifecyclePlanRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **A5.5a** [Jeff] Define class structure: - - [ ] Create class `LifecyclePlanRepository` with proper typing - - [ ] Add `__init__(self, session_factory: Callable[[], Session])` - session factory injection - - [ ] Store `self._session_factory = session_factory` - - [ ] Add class docstring explaining repository pattern usage - - [ ] Commit: "feat(repo): add LifecyclePlanRepository scaffold" - - [ ] **A5.5b** [Jeff] Implement `create(plan: Plan) -> Plan`: - - [ ] Open session using context manager: `with self._session_factory() as session:` - - [ ] Convert domain model: `model = LifecyclePlanModel.from_domain(plan)` - - [ ] Add to session: `session.add(model)` - - [ ] Commit transaction: `session.commit()` - - [ ] Refresh to get any database-generated values: `session.refresh(model)` - - [ ] Convert back and return: `return model.to_domain()` - - [ ] Wrap in try/except for `IntegrityError` - raise custom `DuplicatePlanError` if duplicate ID - - [ ] Add type hints and docstring - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.create()" - - [ ] **A5.5c** [Jeff] Implement `get_by_id(plan_id: str) -> Plan | None`: - - [ ] Query by primary key: `session.query(LifecyclePlanModel).filter_by(plan_id=plan_id).first()` - - [ ] Return `None` if not found - - [ ] Convert to domain model if found - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_id()" - - [ ] **A5.5d** [Jeff] Implement `get_by_phase(phase: PlanPhase, limit: int = 100) -> list[Plan]`: - - [ ] Filter by phase column: `.filter_by(phase=phase.name)` - - [ ] Order by created_at DESC: `.order_by(LifecyclePlanModel.created_at.desc())` - - [ ] Apply limit: `.limit(limit)` - - [ ] Convert all results to domain models using list comprehension - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_phase()" - - [ ] **A5.5e** [Jeff] Implement `get_by_state(state: ProcessingState, limit: int = 100) -> list[Plan]`: - - [ ] Similar pattern to get_by_phase - - [ ] Filter by state column - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_by_state()" - - [ ] **A5.5f** [Jeff] Implement `get_children(parent_plan_id: str) -> list[Plan]`: - - [ ] Filter by parent_plan_id: `.filter_by(parent_plan_id=parent_plan_id)` - - [ ] Order by created_at ASC (oldest first for processing order) - - [ ] Convert all to domain models - - [ ] Used for listing direct subplans - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_children()" - - [ ] **A5.5g** [Jeff] Implement `get_tree(root_plan_id: str) -> list[Plan]`: - - [ ] Use recursive CTE query for all descendants: - ```python - from sqlalchemy import text - cte = text(''' - WITH RECURSIVE plan_tree AS ( - SELECT * FROM lifecycle_plans WHERE plan_id = :root_id - UNION ALL - SELECT lp.* FROM lifecycle_plans lp - INNER JOIN plan_tree pt ON lp.parent_plan_id = pt.plan_id - ) - SELECT * FROM plan_tree ORDER BY created_at ASC - ''') - ``` - - [ ] Execute and map results to domain models - - [ ] Return in tree order (parent before children by creation time) - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.get_tree()" - - [ ] **A5.5h** [Jeff] Implement `update(plan: Plan) -> Plan`: - - [ ] Fetch existing record by plan_id - - [ ] Raise `PlanNotFoundError` if not exists - - [ ] Update all fields from domain model (use a helper to copy attributes) - - [ ] Auto-update `updated_at` timestamp to now - - [ ] Commit transaction - - [ ] Return updated plan (re-query to ensure consistency) - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.update()" - - [ ] **A5.5i** [Jeff] Implement `list_all(limit: int = 100, offset: int = 0) -> list[Plan]`: - - [ ] Query all with pagination: `.offset(offset).limit(limit)` - - [ ] Order by created_at DESC - - [ ] Convert to domain models - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.list_all()" - - [ ] **A5.5j** [Jeff] Implement `count(phase: PlanPhase | None = None, state: ProcessingState | None = None) -> int`: - - [ ] Use `session.query(func.count(LifecyclePlanModel.plan_id))` - - [ ] Apply optional phase filter - - [ ] Apply optional state filter - - [ ] Return `.scalar()` result - - [ ] Commit: "feat(repo): implement LifecyclePlanRepository.count()" - - [ ] **A5.5k** [Jeff] Add `@retry_database` decorator to all methods: - - [ ] Import from `src/cleveragents/core/retry_patterns.py` - - [ ] Configure: 3 retries, exponential backoff (1s, 2s, 4s) - - [ ] Only retry on `OperationalError` (database locked, connection timeout) - - [ ] Do NOT retry on `IntegrityError` (these are application logic errors) - - [ ] Commit: "feat(repo): add retry decorator to LifecyclePlanRepository" - - [ ] **A5.6** [Luis] Implement `ActionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **A5.6a** [Luis] Define class with session factory injection: - - [ ] Create class `ActionRepository` - - [ ] Add `__init__(self, session_factory: Callable[[], Session])` - - [ ] Commit: "feat(repo): add ActionRepository scaffold" - - [ ] **A5.6b** [Luis] Implement `create(action: Action) -> Action`: - - [ ] Same pattern as LifecyclePlanRepository - - [ ] Handle duplicate name error specifically - - [ ] Commit: "feat(repo): implement ActionRepository.create()" - - [ ] **A5.6c** [Luis] Implement `get_by_id(action_id: str) -> Action | None`: - - [ ] Query by primary key - - [ ] Convert to domain or return None - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_id()" - - [ ] **A5.6d** [Luis] Implement `get_by_name(name: str) -> Action | None`: - - [ ] Query by exact namespaced name match: `.filter_by(name=name).first()` - - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action show local/my-action` - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_name()" - - [ ] **A5.6e** [Luis] Implement `get_by_namespace(namespace: str, state: ActionState | None = None) -> list[Action]`: - - [ ] Filter by namespace column - - [ ] Optionally filter by state - - [ ] Order by short_name ASC for consistent display - - [ ] Convert all to domain models - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_namespace()" - - [ ] **A5.6f** [Luis] Implement `get_by_state(state: ActionState) -> list[Action]`: - - [ ] Filter by state column - - [ ] Order by updated_at DESC (most recently modified first) - - [ ] Commit: "feat(repo): implement ActionRepository.get_by_state()" - - [ ] **A5.6g** [Luis] Implement `update(action: Action) -> Action`: - - [ ] Fetch by action_id - - [ ] Update all fields - - [ ] Auto-update updated_at - - [ ] Commit and return - - [ ] Commit: "feat(repo): implement ActionRepository.update()" - - [ ] **A5.6h** [Luis] Implement `list_available(namespace: str | None = None) -> list[Action]`: - - [ ] Filter by state='available' - - [ ] Optionally filter by namespace - - [ ] Order by namespace ASC, short_name ASC - - [ ] Used for `agents [--data-dir PATH] [--config-path PATH] action list` - - [ ] Commit: "feat(repo): implement ActionRepository.list_available()" - - [ ] **A5.6i** [Luis] Implement `delete(action_id: str) -> bool`: - - [ ] First check if any plans reference this action: `plan_repo.count(action_id=action_id)` - - [ ] If plans exist, raise `ActionInUseError` with count of plans - - [ ] Otherwise delete the action - - [ ] Return True if deleted - - [ ] Commit: "feat(repo): implement ActionRepository.delete()" - - [ ] **A5.6j** [Luis] Add retry decorator to all methods: - - [ ] Same pattern as LifecyclePlanRepository - - [ ] Commit: "feat(repo): add retry decorator to ActionRepository" - - [ ] **A5.7** [Jeff] Update `PlanLifecycleService` to use repositories: - - [ ] **A5.7a** [Jeff] Modify `__init__()` to accept repository dependencies: - - [ ] Change signature: `def __init__(self, plan_repository: LifecyclePlanRepository, action_repository: ActionRepository):` - - [ ] Store as instance variables: `self._plan_repo = plan_repository`, `self._action_repo = action_repository` - - [ ] REMOVE the in-memory storage: Delete `self._plans: dict` and `self._actions: dict` - - [ ] Update docstring to reflect dependency injection - - [ ] Commit: "refactor(service): update PlanLifecycleService to inject repositories" - - [ ] **A5.7b** [Jeff] Update `create_action()` to use ActionRepository: - - [ ] Replace `self._actions[action.action_id] = action` with `self._action_repo.create(action)` - - [ ] Handle `DuplicateActionError` by converting to user-friendly error message - - [ ] Return the created action from repository (may have database-modified fields) - - [ ] Commit: "refactor(service): update create_action() to use repository" - - [ ] **A5.7c** [Jeff] Update `get_action()` to use repository: - - [ ] Replace dict lookup with `self._action_repo.get_by_id()` or `get_by_name()` - - [ ] Handle both ID and namespaced name lookups - - [ ] Commit: "refactor(service): update get_action() to use repository" - - [ ] **A5.7d** [Jeff] Update `list_actions()` to use repository: - - [ ] Replace dict.values() iteration with `self._action_repo.list_available()` - - [ ] Add namespace filter parameter - - [ ] Add state filter parameter - - [ ] Commit: "refactor(service): update list_actions() to use repository" - - [ ] **A5.7e** [Jeff] Update `use_action()` to use both repositories: - - [ ] Fetch action from ActionRepository by name - - [ ] Raise `ActionNotFoundError` if not exists - - [ ] Raise `ActionNotAvailableError` if action.state != AVAILABLE - - [ ] Create new Plan domain object with ULID, set action_id reference - - [ ] Persist plan via LifecyclePlanRepository.create() - - [ ] Return the created plan - - [ ] Commit: "refactor(service): update use_action() to use repositories" - - [ ] **A5.7f** [Jeff] Update all plan state transition methods: - - [ ] `start_strategize()`: fetch plan → verify phase → update state → save - - [ ] `complete_strategize()`: fetch → verify → update phase+state → save - - [ ] `fail_strategize()`: fetch → update state to ERRORED → set error_message → save - - [ ] `start_execute()`: same pattern - - [ ] `complete_execute()`: same pattern, store changeset_id - - [ ] `fail_execute()`: same pattern - - [ ] `apply_plan()`: verify Execute phase complete → update to APPLIED → set completed_at → save - - [ ] All methods must re-fetch after save to return current state - - [ ] Commit: "refactor(service): update phase transition methods to use repository" - - [ ] **A5.7g** [Jeff] Update `cancel_plan()` to use repository: - - [ ] Fetch plan - - [ ] Verify not in terminal state (APPLIED or CANCELLED) - - [ ] Set state to CANCELLED, set completed_at - - [ ] Persist via repository - - [ ] Commit: "refactor(service): update cancel_plan() to use repository" - - [ ] **A5.7h** [Jeff] Add transaction handling for multi-step operations: - - [ ] For operations that modify multiple entities (e.g., use_action creates plan + may update action): - - [ ] Use UnitOfWork pattern: start transaction, do all operations, commit atomically - - [ ] If any step fails, rollback all changes - - [ ] Create `UnitOfWork` class if not exists: manages session lifecycle - - [ ] Commit: "feat(service): add transaction handling for multi-step operations" - - [ ] **A5.8** [Luis] Update DI container in `src/cleveragents/application/container.py`: - - [ ] **A5.8a** [Luis] Add `LifecyclePlanRepository` provider: - - [ ] Create factory function that instantiates repository with session factory - - [ ] Register with container - - [ ] Ensure proper scoping (singleton or per-request based on usage pattern) - - [ ] Commit: "feat(di): add LifecyclePlanRepository provider" - - [ ] **A5.8b** [Luis] Add `ActionRepository` provider: - - [ ] Same pattern as plan repository - - [ ] Commit: "feat(di): add ActionRepository provider" - - [ ] **A5.8c** [Luis] Update `PlanLifecycleService` provider to inject repositories: - - [ ] Modify service factory to resolve both repositories - - [ ] Pass to PlanLifecycleService constructor - - [ ] Verify dependency chain is correct - - [ ] Commit: "feat(di): update PlanLifecycleService provider with repositories" - - [ ] Tests: Integration tests for plan/action persistence - - [ ] **A5.9** [Rui] Write Behave scenarios in `features/plan_persistence.feature`: - - [ ] **A5.9a** [Rui] Scenario: Create plan stores record in database - - [ ] Given: An action "local/test-action" exists in database with state=AVAILABLE - - [ ] And: A project "local/test-project" exists - - [ ] When: I call `plan_service.use_action("local/test-action", project_ids=["proj-123"])` - - [ ] Then: A plan record exists in the lifecycle_plans table - - [ ] And: The plan_id is a valid 26-character ULID - - [ ] And: The plan.phase is STRATEGIZE - - [ ] And: The plan.state is QUEUED - - [ ] And: The plan.action_id matches the action - - [ ] Commit: "test(behave): add plan creation persistence scenario" - - [ ] **A5.9b** [Rui] Scenario: Update plan phase persists correctly - - [ ] Given: A plan exists in database with phase=STRATEGIZE, state=QUEUED - - [ ] When: I call `plan_service.complete_strategize(plan_id, strategy_context={...})` - - [ ] And: I call `plan_service.start_execute(plan_id)` - - [ ] Then: The database record shows phase='EXECUTE' - - [ ] And: The database record shows state='PROCESSING' - - [ ] And: The updated_at timestamp has changed - - [ ] Commit: "test(behave): add plan phase update persistence scenario" - - [ ] **A5.9c** [Rui] Scenario: Query plans by phase returns filtered results - - [ ] Given: 3 plans exist: 1 in STRATEGIZE, 1 in EXECUTE, 1 in APPLIED - - [ ] When: I query `plan_repo.get_by_phase(PlanPhase.STRATEGIZE)` - - [ ] Then: Only 1 plan is returned - - [ ] And: Its phase is STRATEGIZE - - [ ] Commit: "test(behave): add plan phase query scenario" - - [ ] **A5.9d** [Rui] Scenario: Query plans by state returns filtered results - - [ ] Given: 3 plans exist: 1 QUEUED, 1 PROCESSING, 1 ERRORED - - [ ] When: I query `plan_repo.get_by_state(ProcessingState.ERRORED)` - - [ ] Then: Only 1 plan is returned - - [ ] And: Its state is ERRORED - - [ ] Commit: "test(behave): add plan state query scenario" - - [ ] **A5.9e** [Rui] Scenario: Get plan tree returns parent and all children - - [ ] Given: A root plan exists with plan_id="root-123" - - [ ] And: A child plan exists with parent_plan_id="root-123" - - [ ] And: A grandchild plan exists with parent_plan_id=child_plan_id - - [ ] When: I query `plan_repo.get_tree("root-123")` - - [ ] Then: 3 plans are returned in order - - [ ] And: First plan is the root - - [ ] And: Second plan is the child - - [ ] And: Third plan is the grandchild - - [ ] Commit: "test(behave): add plan tree query scenario" - - [ ] **A5.9f** [Rui] Scenario: Concurrent plan creation is thread-safe - - [ ] Given: An action exists - - [ ] When: 10 threads simultaneously call `plan_service.use_action()` - - [ ] Then: All 10 plans are created successfully - - [ ] And: All 10 plan_ids are unique - - [ ] And: No database integrity errors occurred - - [ ] Commit: "test(behave): add concurrent plan creation scenario" - - [ ] **A5.10** [Rui] Write Behave scenarios in `features/action_persistence.feature`: - - [ ] **A5.10a** [Rui] Scenario: Create action stores record in database - - [ ] Given: No action named "local/test-action" exists - - [ ] When: I call `action_service.create_action()` with valid parameters - - [ ] Then: An action record exists in the actions table - - [ ] And: The action_id is a valid 26-character ULID - - [ ] And: The namespace column is "local" - - [ ] And: The short_name column is "test-action" - - [ ] Commit: "test(behave): add action creation persistence scenario" - - [ ] **A5.10b** [Rui] Scenario: Get action by namespaced name works - - [ ] Given: An action "local/my-action" exists in database - - [ ] When: I call `action_repo.get_by_name("local/my-action")` - - [ ] Then: The action is returned - - [ ] And: Its name matches "local/my-action" - - [ ] Commit: "test(behave): add action name lookup scenario" - - [ ] **A5.10c** [Rui] Scenario: List available excludes archived actions - - [ ] Given: 3 actions exist: 2 with state=AVAILABLE, 1 with state=ARCHIVED - - [ ] When: I call `action_repo.list_available()` - - [ ] Then: Only 2 actions are returned - - [ ] And: Neither has state=ARCHIVED - - [ ] Commit: "test(behave): add action list available scenario" - - [ ] **A5.10d** [Rui] Scenario: Update action state persists - - [ ] Given: An action exists with state=DRAFT - - [ ] When: I call `action_service.make_available(action_id)` - - [ ] Then: The database record shows state='AVAILABLE' - - [ ] Commit: "test(behave): add action state update scenario" - - [ ] **A5.10e** [Rui] Scenario: Delete action with existing plans fails - - [ ] Given: An action "local/used-action" exists - - [ ] And: A plan exists that references this action - - [ ] When: I call `action_repo.delete(action_id)` - - [ ] Then: An ActionInUseError is raised - - [ ] And: The action still exists in the database - - [ ] Commit: "test(behave): add action delete protection scenario" - - [ ] **A5.11** [Rui] Write Robot test `robot/plan_persistence_e2e.robot`: - - [ ] **A5.11a** [Rui] Test: Full lifecycle persists all transitions - - [ ] Create action via CLI: `agents [--data-dir PATH] [--config-path PATH] action create --name local/e2e-test ...` - - [ ] Make action available: `agents [--data-dir PATH] [--config-path PATH] action available ` - - [ ] Create project: `agents [--data-dir PATH] [--config-path PATH] project create --name local/e2e-project` - - [ ] Use action on project: `agents [--data-dir PATH] [--config-path PATH] plan use local/e2e-test --project local/e2e-project` - - [ ] Execute plan: `agents [--data-dir PATH] [--config-path PATH] plan execute ` - - [ ] Apply plan: `agents [--data-dir PATH] [--config-path PATH] plan apply ` - - [ ] Verify via `agents [--data-dir PATH] [--config-path PATH] plan status ` shows APPLIED phase - - [ ] Query database directly to verify all state transitions recorded - - [ ] Commit: "test(robot): add full lifecycle persistence e2e test" - - [ ] **A5.11b** [Rui] Test: Restart persistence - - [ ] Create plan via CLI - - [ ] Get plan_id from output - - [ ] Simulate process crash (kill the process or restart CLI) - - [ ] Run new CLI command: `agents [--data-dir PATH] [--config-path PATH] plan status ` - - [ ] Verify plan still exists and shows correct state - - [ ] Commit: "test(robot): add restart persistence e2e test" - - [ ] **A5.11c** [Rui] Test: Concurrent CLI access - - [ ] Start two CLI processes accessing same plan - - [ ] One process starts execute, other queries status - - [ ] Verify no data corruption or deadlocks - - [ ] Both processes complete successfully - - [ ] Commit: "test(robot): add concurrent CLI access e2e test" +**Parallel Group A5: Plan Persistence (M1-critical)** + **PARALLEL SUBTRACK A5.alpha [Jeff]**: Alembic migrations for action/plan tables + **PARALLEL SUBTRACK A5.beta [Luis]**: SQLAlchemy models for new tables + **SEQUENTIAL AFTER alpha+beta [Jeff + Luis]**: Repositories + service integration + **PARALLEL CONTINUOUS [Rui]**: Persistence tests added inside each commit + - [ ] **COMMIT (Owner: Jeff | Group: A5.alpha) - Commit message: "feat(db): add actions and action_invariants tables"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add Alembic migration for `actions` table with ULID PK, namespaced_name, actor refs, DoD fields, automation_profile, invariant_actor, timestamps. + - [ ] Code [Jeff]: Add `action_invariants` table with FK to actions, `scope` column, and created_at timestamp. + - [ ] Code [Jeff]: Add unique index on actions.namespaced_name and search index on namespace for list filtering. + - [ ] Code [Jeff]: Ensure downgrade path drops indexes and tables in reverse order. + - [ ] Docs [Jeff]: Update `docs/reference/database_schema.md` with column-level details and constraints. + - [ ] Tests (Behave) [Rui]: Add migration scenario that runs upgrade and asserts tables + indexes exist. + - [ ] Tests (Robot) [Rui]: Add Robot migration smoke test using `nox -s db_migrate` (create session if missing). + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/db_migration_actions_bench.py` for migration runtime baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(db): add actions and action_invariants tables"`. + - [ ] **COMMIT (Owner: Jeff | Group: A5.alpha) - Commit message: "feat(db): add lifecycle_plans and plan_projects tables"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add Alembic migration for `lifecycle_plans` with ULID PK, phase/state enums, action linkage, automation_profile, invariant_actor, and timestamps. + - [ ] Code [Jeff]: Add `plan_projects` table with plan_id, project_name (namespaced), read_only flag, and alias. + - [ ] Code [Jeff]: Add indexes on plan phase/state for filtering and plan_projects.project_name for lookups. + - [ ] Docs [Jeff]: Update schema docs with plan/project link rules and uniqueness constraints. + - [ ] Tests (Behave) [Rui]: Add migration scenario verifying plan/project link table + indexes. + - [ ] Tests (Robot) [Rui]: Add Robot test that inserts a plan/project link row and queries it. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/db_migration_plans_bench.py` for migration runtime baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(db): add lifecycle_plans and plan_projects tables"`. + - [ ] **COMMIT (Owner: Jeff | Group: A5.alpha) - Commit message: "feat(db): add plan arguments and plan invariants tables"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add `plan_arguments` table (plan_id, name, value_json, value_type) and `plan_invariants` table (plan_id, invariant_text, source_scope). + - [ ] Code [Jeff]: Add uniqueness constraint on (plan_id, name) for arguments and (plan_id, invariant_text) for invariants. + - [ ] Docs [Jeff]: Document argument storage, JSON serialization rules, and invariant source scopes. + - [ ] Tests (Behave) [Rui]: Add migration scenario verifying both tables and constraints. + - [ ] Tests (Robot) [Rui]: Add Robot test that inserts a plan invariant and asserts retrieval. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/db_migration_plan_args_bench.py` for migration runtime baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(db): add plan arguments and plan invariants tables"`. + - [ ] **COMMIT (Owner: Luis | Group: A5.beta) - Commit message: "feat(models): add action and lifecycle plan ORM models"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Add SQLAlchemy models for Action, ActionInvariant, LifecyclePlan, PlanProjectLink, PlanArgument, PlanInvariant. + - [ ] Code [Luis]: Implement `to_domain()` and `from_domain()` mappings with ULID validation, enum conversion, and timestamp normalization. + - [ ] Code [Luis]: Add repository-facing helpers for filtering by namespace/phase/state. + - [ ] Docs [Luis]: Update ORM mapping notes in `docs/reference/database_schema.md` with model field mapping table. + - [ ] Tests (Behave) [Rui]: Add scenarios for ORM round-trip serialization and enum conversions. + - [ ] Tests (Robot) [Rui]: Add Robot test that loads a plan and asserts field mapping correctness. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/orm_mapping_bench.py` for Action/Plan mapping throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(models): add action and lifecycle plan ORM models"`. + - [ ] **COMMIT (Owner: Jeff | Group: A5.gamma) - Commit message: "feat(repo): add action and lifecycle plan repositories"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Implement ActionRepository CRUD + list filters by namespace/state/automation profile. + - [ ] Code [Jeff]: Implement PlanRepository CRUD + list filters by phase/state/project; add plan lookup by namespaced name. + - [ ] Code [Jeff]: Add retry decorator to repositories; retry only on `OperationalError`, never on `IntegrityError`. + - [ ] Docs [Jeff]: Document repository interfaces, error types, and pagination guidance. + - [ ] Tests (Behave) [Rui]: Add scenarios for repository create/get/list/update/delete guardrails. + - [ ] Tests (Robot) [Rui]: Add Robot test that exercises repository through service layer. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/repository_query_bench.py` for list + filter performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(repo): add action and lifecycle plan repositories"`. + - [ ] **COMMIT (Owner: Luis | Group: A5.gamma) - Commit message: "feat(service): persist plan lifecycle via repositories"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Update `PlanLifecycleService` to use repositories instead of in-memory dicts. + - [ ] Code [Luis]: Ensure plan creation persists arguments, invariants, automation profile, and project links in a single transaction. + - [ ] Code [Luis]: Add transactional safeguards for multi-step updates (create action + plan, correction updates). + - [ ] Docs [Luis]: Update service docs to reflect persistence and remove in-memory notes. + - [ ] Tests (Behave) [Rui]: Add scenarios for persisted lifecycle transitions and error handling. + - [ ] Tests (Robot) [Rui]: Add end-to-end test that restarts the app and re-reads plan state. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_lifecycle_service_bench.py` for persistence operations. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(service): persist plan lifecycle via repositories"`. + - [ ] **COMMIT (Owner: Luis | Group: A5.gamma) - Commit message: "feat(di): wire lifecycle repos and services"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Register ActionRepository + PlanRepository in `application/container.py` and UnitOfWork. + - [ ] Code [Luis]: Inject repositories into PlanLifecycleService and CLI commands. + - [ ] Code [Luis]: Add container wiring tests to ensure singleton lifetimes are correct. + - [ ] Docs [Luis]: Update DI wiring notes in `docs/architecture/decisions/adr-003.md`. + - [ ] Tests (Behave) [Rui]: Add scenarios that use container wiring for lifecycle commands. + - [ ] Tests (Robot) [Rui]: Add Robot smoke test verifying CLI uses persisted service. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/di_container_bench.py` for container resolution overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(di): wire lifecycle repos and services"`. -- [ ] **Stage A6: Automation Levels Foundation** (Day 4-5) **[Luis]** - - [ ] Code: Implement basic automation level support - - [ ] **A6.1** [Luis] Add `AutomationLevel` enum to `src/cleveragents/domain/models/core/plan.py`: - - [ ] Value `MANUAL` - user triggers each phase transition - - [ ] Value `REVIEW_BEFORE_APPLY` - auto strategize+execute, pause before apply - - [ ] Value `FULL_AUTOMATION` - all phases automatic - - [ ] **A6.2** [Luis] Add automation level configuration to `src/cleveragents/config/settings.py`: - - [ ] Add `default_automation_level: AutomationLevel` setting - - [ ] Add `CLEVERAGENTS_AUTOMATION_LEVEL` environment variable - - [ ] Implement hierarchy: plan-level > session-level > global-level - - [ ] **A6.3** [Luis] Update `PlanLifecycleService` to respect automation levels: - - [ ] Add `automation_level` parameter to `use_action()` method - - [ ] If automation allows, automatically call `execute_plan()` after strategize completes - - [ ] If full automation, automatically call `apply_plan()` after execute completes - - [ ] Add pause/resume capability for review-before-apply mode - - [ ] **A6.4** [Luis] Update CLI commands to support automation levels: - - [ ] Add `--automation-level` flag to `agents [--data-dir PATH] [--config-path PATH] plan use` command - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] config set automation-level ` command - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] plan set-automation-level ` command: - - [ ] Can change automation level for existing plan - - [ ] Only affects future phase transitions - - [ ] Subplans created after change use new level - - [ ] Add `agents [--data-dir PATH] [--config-path PATH] session set automation-level ` command: - - [ ] Set session-level automation (overrides global) - - [ ] Persists for current session only - - [ ] Tests: Automation level tests - - [ ] **A6.5** [Rui] Write Behave scenarios in `features/automation_levels.feature`: - - [ ] Scenario: Manual mode requires explicit execute command - - [ ] Scenario: Review-before-apply auto-executes but pauses at apply - - [ ] Scenario: Full automation runs all phases without user input - - [ ] Scenario: Plan-level automation overrides global setting - - [ ] Scenario: Change automation level mid-plan works correctly + - [ ] **COMMIT (Owner: Rui | Group: A5.tests) - Commit message: "test(persistence): add plan/action persistence suites"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Tests (Behave) [Rui]: Add plan persistence scenarios (create, update phase/state, list filters, plan tree, concurrency). + - [ ] Tests (Behave) [Rui]: Add action persistence scenarios (create, list available, archive guard). + - [ ] Tests (Robot) [Rui]: Add plan persistence E2E (full lifecycle, restart persistence, concurrent CLI access). + - [ ] Docs [Rui]: Update `docs/development/testing.md` with persistence suites and `nox` commands. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/persistence_suites_bench.py` for DB read/write baselines. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "test(persistence): add plan/action persistence suites"`. -**M1 SUCCESS CRITERIA**: -- [ ] Can create an action via CLI and it persists to database -- [ ] Can use an action on a project to create a plan -- [ ] Plan transitions through phases with database persistence -- [ ] Automation levels work (at least manual mode fully functional) +**Parallel Group A5.legacy: Remove legacy plan build/apply path (M1-critical)** + - [ ] **COMMIT (Owner: Jeff | Group: A5.legacy) - Commit message: "refactor(plan): remove legacy plan service and CLI"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Remove `PlanService` usage from CLI (`plan tell/build/apply/new/current/list/cd/continue`). + - [ ] Code [Jeff]: Remove or quarantine legacy `plan_service.py`, `plan_legacy.py`, and legacy CLI helpers; add explicit NotImplementedError where needed. + - [ ] Code [Jeff]: Remove or archive legacy `PlanModel`/`PlanStatus` DB tables if unused by v3; document migration path. + - [ ] Docs [Jeff]: Update CLI docs to list only v3 lifecycle commands and new `plan use/execute/apply` flows. + - [ ] Tests (Behave) [Rui]: Remove/replace legacy scenarios with v3 equivalents and adjust coverage expectations. + - [ ] Tests (Robot) [Rui]: Remove legacy robot suites and add v3 replacements where needed. + - [ ] Tests (ASV) [Rui]: Update asv suite to remove legacy plan benchmarks and add v3 lifecycle baseline benchmark. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "refactor(plan): remove legacy plan service and CLI"`. + +**Parallel Group A6: Automation Profiles Foundation [Jeff + Luis]** (M1-critical; depends on A5 persistence) + **PARALLEL SUBTRACK A6.core [Jeff]**: Profile model + built-ins + schema + **PARALLEL SUBTRACK A6.service [Luis]**: Profile resolution + precedence + **PARALLEL SUBTRACK A6.cli [Rui]**: CLI commands for profiles + **SEQUENTIAL MERGE NOTE**: A6.core must land before A6.service/cli; A6.service must land before gating integration in Section 6. + - [ ] **COMMIT (Owner: Jeff | Group: A6.core) - Commit message: "feat(domain): add automation profile model and built-ins"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add `AutomationProfile` model with threshold fields (phase transitions, decision autonomy, child plan spawn, self-repair, apply gating). + - [ ] Code [Jeff]: Add built-in profiles (`manual`, `review`, `supervised`, `full-auto`, etc.) per spec with constant definitions. + - [ ] Code [Jeff]: Add YAML schema for automation profiles under `docs/schema/automation_profile.schema.yaml` and loader helper. + - [ ] Docs [Jeff]: Add `docs/reference/automation_profiles.md` describing built-ins and threshold semantics. + - [ ] Tests (Behave) [Rui]: Add scenarios for profile validation and built-in defaults. + - [ ] Tests (Robot) [Rui]: Add Robot test that loads each built-in profile and prints summary. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/automation_profile_bench.py` for profile validation. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(domain): add automation profile model and built-ins"`. + - [ ] **COMMIT (Owner: Luis | Group: A6.service) - Commit message: "feat(service): resolve automation profiles with precedence"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Add `AutomationProfileService` to resolve profiles with precedence (plan > action > project > global). + - [ ] Code [Luis]: Add persistence table `automation_profiles` (namespaced name PK) and repository with list/show. + - [ ] Code [Luis]: Add config key `core.automation_profile` and env var override for global default. + - [ ] Docs [Luis]: Update `docs/reference/config.md` with automation profile defaults and override behavior. + - [ ] Tests (Behave) [Rui]: Add scenarios for precedence resolution and missing profile errors. + - [ ] Tests (Robot) [Rui]: Add Robot config smoke test for global profile override. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/automation_profile_resolution_bench.py` for resolution latency. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(service): resolve automation profiles with precedence"`. + - [ ] **COMMIT (Owner: Rui | Group: A6.cli) - Commit message: "feat(cli): add automation-profile commands"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Rui]: Implement `agents automation-profile add/remove/list/show` commands with YAML config input. + - [ ] Code [Rui]: Add `--automation-profile` to `plan use` and output profile in `plan status`. + - [ ] Docs [Rui]: Update CLI reference with automation-profile command examples. + - [ ] Tests (Behave) [Rui]: Add CLI scenarios for profile add/list/show/remove. + - [ ] Tests (Robot) [Rui]: Add Robot CLI tests for automation-profile commands. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/automation_profile_cli_bench.py` for CLI parsing. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "feat(cli): add automation-profile commands"`. + +**M1 SUCCESS CRITERIA (Day 7 MVP - source code only)**: +- Action created from YAML config and persisted (namespaced name, invariants, automation profile). +- Project created and linked to a local git-checkout resource. +- Plan use -> strategize -> execute -> apply completes with sandbox isolation and diff review. +- Tool-based change tracking produces a ChangeSet and applies to the repo after approval. +- `nox` passes with coverage >=97% on the MVP end-to-end path. --- @@ -1885,29 +1560,174 @@ MERGE POINT: Day 30 - M6 Large Project Autonomy Target **Target: Milestone M2 (+10 days)** -**WEEK 1-2 - PARALLEL WITH PLAN LIFECYCLE** +**Parallel Group B1: Resource Registry Core [Hamza + Jeff]** (can start after A5.alpha migrations are available) +- [ ] **COMMIT (Owner: Hamza | Group: B1.core) - Commit message: "feat(domain): add resource type spec and resource model"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Create `src/cleveragents/domain/models/core/resource_type.py` with `ResourceTypeSpec`, `ResourceTypeArgument`, `ResourceKind` (physical/virtual), and `SandboxStrategy` enum. + - [ ] Code [Hamza]: Add resource type fields: `user_addable`, `allowed_parents`, `allowed_children`, `auto_discover`, `handler` reference, and `sandbox_strategy` default. + - [ ] Code [Hamza]: Add `Resource` and `ResourceRef` models with ULID, optional namespaced name, type name, location, description, sandbox strategy, read_only, and metadata. + - [ ] Code [Hamza]: Add validators for namespaced naming, ULID format, and parent/child DAG sanity (no self loops, no duplicate edges, type compatibility). + - [ ] Docs [Hamza]: Add `docs/reference/resource_model.md` with examples for git-checkout and fs-directory resources plus physical/virtual notes. + - [ ] Tests (Behave) [Rui]: Add scenarios validating ULID format, namespace rules, allowed parent/child type checks, and sandbox strategy defaults. + - [ ] Tests (Robot) [Rui]: Add Robot test that loads a ResourceTypeSpec YAML fixture and validates it. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_model_bench.py` for resource validation + DAG checks. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(domain): add resource type spec and resource model"`. +- [ ] **COMMIT (Owner: Hamza | Group: B1.core) - Commit message: "feat(domain): add project model v3 with linked resources"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Add `Project`, `ProjectResourceLink`, `ProjectValidation`, and `ProjectContextPolicy` models using namespaced name as the unique identifier (no ULID per spec). + - [ ] Code [Hamza]: Add fields for `invariants`, `invariant_actor`, `automation_profile`, and `context_views` (strategize/execute/apply/default). + - [ ] Code [Hamza]: Add validation for resource link overrides (read_only flags, alias uniqueness, resource existence). + - [ ] Docs [Hamza]: Add `docs/reference/project_model.md` describing resource linking, validation attachments, and context view policies. + - [ ] Tests (Behave) [Rui]: Add scenarios for project model validation, link overrides, and context view inheritance. + - [ ] Tests (Robot) [Rui]: Add Robot test that creates a Project object and prints serialized output. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/project_model_bench.py` for serialization/validation performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(domain): add project model v3 with linked resources"`. +- [ ] **COMMIT (Owner: Jeff | Group: B1.core) - Commit message: "feat(db): add resource registry tables"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Add Alembic migration for `resource_types`, `resources`, and `resource_edges` tables with indexes on type/name/namespace. + - [ ] Code [Jeff]: Store `resource_kind` (physical/virtual), `sandbox_strategy`, and optional `namespaced_name` in `resources`. + - [ ] Code [Jeff]: Add foreign keys and cascade rules for resource_edges (parent/child) with uniqueness constraint. + - [ ] Docs [Jeff]: Update `docs/reference/database_schema.md` with resource registry tables and constraints. + - [ ] Tests (Behave) [Rui]: Add migration scenarios verifying tables, indices, and edge uniqueness. + - [ ] Tests (Robot) [Rui]: Add Robot migration smoke test using `nox -s db_migrate`. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_registry_migration_bench.py` for migration baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(db): add resource registry tables"`. -``` -WORKSTREAM B PARALLEL STRUCTURE: +**Parallel Group B2: Project Persistence + Services [Hamza + Luis]** (depends on B1 domain models) +- [ ] **COMMIT (Owner: Jeff | Group: B2.persistence) - Commit message: "feat(db): add projects and project links tables"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Add Alembic migration for `projects`, `project_resource_links`, and `project_validations` tables. + - [ ] Code [Jeff]: Use namespaced name as project primary key; enforce unique constraint on `projects.namespaced_name`. + - [ ] Code [Jeff]: Add indexes for `project_resource_links.project_name` and `resource_id` for fast joins. + - [ ] Docs [Jeff]: Document project table schema and link semantics in `docs/reference/database_schema.md`. + - [ ] Tests (Behave) [Rui]: Add migration scenarios verifying project tables and constraints. + - [ ] Tests (Robot) [Rui]: Add Robot test that inserts a project and link row. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/project_migration_bench.py` for migration baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(db): add projects and project links tables"`. +- [ ] **COMMIT (Owner: Hamza | Group: B2.persistence) - Commit message: "feat(repo): add resource repositories"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement `ResourceTypeRepository` CRUD and `ResourceRepository` CRUD with DAG edge helpers. + - [ ] Code [Hamza]: Add methods for tree traversal, child discovery queries, and name/ULID resolution. + - [ ] Code [Hamza]: Add repository guardrails for preventing cycles and duplicate edges. + - [ ] Docs [Hamza]: Document repository interfaces in `docs/reference/repositories.md`. + - [ ] Tests (Behave) [Rui]: Add repository scenarios for create/get/list/tree and cycle rejection. + - [ ] Tests (Robot) [Rui]: Add Robot test exercising tree output ordering. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_repository_bench.py` for tree query performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(repo): add resource repositories"`. +- [ ] **COMMIT (Owner: Hamza | Group: B2.persistence) - Commit message: "feat(repo): add project repositories"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement `ProjectRepository` and `ProjectResourceLinkRepository` with namespace filtering and name-based lookup. + - [ ] Code [Hamza]: Add methods to list project validations and context policies. + - [ ] Docs [Hamza]: Update repository docs with project link examples and validation attachment notes. + - [ ] Tests (Behave) [Rui]: Add scenarios for project create/link/unlink and validation list. + - [ ] Tests (Robot) [Rui]: Add Robot test that links two resources to one project. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/project_repository_bench.py` for link/unlink performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(repo): add project repositories"`. +- [ ] **COMMIT (Owner: Hamza | Group: B2.service) - Commit message: "feat(service): add resource registry service"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement `ResourceRegistryService` for register/remove/show/tree operations with name/ULID resolution. + - [ ] Code [Hamza]: Add auto-discovery hook that delegates to resource handlers (git-checkout for MVP). + - [ ] Code [Hamza]: Add validation that resource type supports parent/child linkage before linking. + - [ ] Docs [Hamza]: Add `docs/reference/resource_registry.md` describing API behavior and error cases. + - [ ] Tests (Behave) [Rui]: Add scenarios for register/remove/show/tree behavior and auto-discovery. + - [ ] Tests (Robot) [Rui]: Add Robot test that registers a git-checkout and inspects child count. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_registry_service_bench.py` for register/show performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(service): add resource registry service"`. +- [ ] **COMMIT (Owner: Luis | Group: B2.service) - Commit message: "feat(service): add project service v3"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement `ProjectService` create/list/show/delete/link/unlink methods using repositories. + - [ ] Code [Luis]: Add validation attachment helpers and context policy setters for project views. + - [ ] Code [Luis]: Enforce read-only resource links and project-level invariant actor defaults. + - [ ] Docs [Luis]: Update `docs/reference/project_service.md` with usage examples and error cases. + - [ ] Tests (Behave) [Rui]: Add scenarios for project create/link/unlink/validation/context policy. + - [ ] Tests (Robot) [Rui]: Add Robot test that creates project and links a resource. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/project_service_bench.py` for link/unlink performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(service): add project service v3"`. -TRACK B.alpha [Hamza - Day 1-2]: Domain Models (B1.1-B1.6) - └── Can start immediately, no dependencies - -TRACK B.beta [Hamza - Day 2-3]: CLI Commands (B2.1-B2.2) - └── Depends on B1 models - -TRACK B.gamma [Hamza + Luis - Day 3-5]: Sandbox Framework (B3.1-B3.8) - └── Depends on B1 Resource model - └── Luis owns Protocol (B3.1-B3.2), Hamza owns Implementations (B3.3-B3.4) - -TRACK B.delta [Hamza - Day 5-6]: Resource Service (B4.1-B4.3) - └── Depends on B3 sandbox - -TRACK B.epsilon [Hamza - Day 6-7]: Persistence (B5.1-B5.5) - └── Depends on B1 models, parallel with B4 +**Parallel Group B3: CLI Commands [Rui]** (depends on B2 services) +- [ ] **COMMIT (Owner: Rui | Group: B3.cli) - Commit message: "feat(cli): add resource type commands"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Rui]: Add `agents resource type add/remove/list/show` commands with YAML config input and schema validation. + - [ ] Code [Rui]: Implement `--update` behavior and error on name conflicts per spec. + - [ ] Docs [Rui]: Update CLI reference with resource type examples and expected output columns. + - [ ] Tests (Behave) [Rui]: Add scenarios for resource type lifecycle and invalid schema handling. + - [ ] Tests (Robot) [Rui]: Add Robot suite `robot/resource_type_cli.robot`. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_type_cli_bench.py` for config parsing overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "feat(cli): add resource type commands"`. +- [ ] **COMMIT (Owner: Rui | Group: B3.cli) - Commit message: "feat(cli): add resource commands"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Rui]: Add `agents resource add/remove/list/show/tree` commands with type-specific flags and name/ULID resolution. + - [ ] Code [Rui]: Implement `resource inspect --tree/--file` per spec for resource introspection. + - [ ] Docs [Rui]: Update CLI reference with resource examples (git-checkout, fs-directory) and output columns. + - [ ] Tests (Behave) [Rui]: Add scenarios for resource registration, list filters, and tree rendering. + - [ ] Tests (Robot) [Rui]: Add Robot suite `robot/resource_cli.robot`. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/resource_cli_bench.py` for command parsing and list output. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "feat(cli): add resource commands"`. +- [ ] **COMMIT (Owner: Rui | Group: B3.cli) - Commit message: "feat(cli): add project commands"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Rui]: Add `agents project create/show/list/delete/link-resource/unlink-resource` commands using namespaced project names. + - [ ] Code [Rui]: Add `agents project validation add/remove/list` and `project context set/show` commands (context views per phase). + - [ ] Docs [Rui]: Update CLI reference with project examples and validation output. + - [ ] Tests (Behave) [Rui]: Add scenarios for project create/link/validation/context policies. + - [ ] Tests (Robot) [Rui]: Add Robot suite `robot/project_cli.robot`. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/project_cli_bench.py` for command parsing and list output. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "feat(cli): add project commands"`. -TESTING [Rui - Continuous]: Write tests BEFORE implementation -``` +**Parallel Group B4: Sandboxing [Luis + Jeff]** (depends on resource registry + project links) +- [ ] **COMMIT (Owner: Luis | Group: B4.sandbox) - Commit message: "feat(sandbox): add sandbox strategy interface and manager"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add `SandboxStrategy` protocol, `SandboxRef`, `SandboxManager`, and `SandboxRegistry` with per-resource sandboxes. + - [ ] Code [Luis]: Implement lazy sandbox creation, cleanup hooks, and plan-scoped retention policy stubs. + - [ ] Code [Luis]: Add sandbox path rewriting helper for tool execution and MCP adapters. + - [ ] Docs [Luis]: Add `docs/reference/sandbox.md` describing lifecycle, APIs, and path rewriting rules. + - [ ] Tests (Behave) [Rui]: Add scenarios for sandbox manager creation, cleanup, and path rewrite behavior. + - [ ] Tests (Robot) [Rui]: Add Robot test that creates a sandbox and verifies filesystem isolation. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/sandbox_manager_bench.py` for sandbox creation overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(sandbox): add sandbox strategy interface and manager"`. +- [ ] **COMMIT (Owner: Luis | Group: B4.sandbox) - Commit message: "feat(sandbox): implement git_worktree strategy"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement git worktree creation, checkout, and cleanup for git-checkout resources. + - [ ] Code [Luis]: Add safe fallback for repositories without clean worktrees and clear error messages. + - [ ] Code [Luis]: Record sandbox metadata (worktree path, branch, base commit) for rollback. + - [ ] Docs [Luis]: Update sandbox doc with git_worktree usage and rollback behavior. + - [ ] Tests (Behave) [Rui]: Add scenarios for git worktree sandbox creation and rollback. + - [ ] Tests (Robot) [Rui]: Add Robot test that modifies sandbox and verifies original repo unchanged. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/git_worktree_bench.py` for sandbox creation time. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(sandbox): implement git_worktree strategy"`. +- [ ] **COMMIT (Owner: Hamza | Group: B4.sandbox) - Commit message: "feat(resource): add git-checkout handler and discovery"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Add git-checkout handler that validates repo path, branch, and read_only flags. + - [ ] Code [Hamza]: Implement child resource discovery for fs-directory children (schema-only for now) and record ULID-only children. + - [ ] Code [Hamza]: Add sandbox strategy mapping for git-checkout and path normalization helpers. + - [ ] Docs [Hamza]: Document git-checkout handler behavior in `docs/reference/resources_git.md`. + - [ ] Tests (Behave) [Rui]: Add scenarios for handler validation and discovery counts. + - [ ] Tests (Robot) [Rui]: Add Robot test registering a git repo and asserting discovered children. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/git_discovery_bench.py` for discovery cost. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(resource): add git-checkout handler and discovery"`. +- [ ] **COMMIT (Owner: Luis | Group: B4.sandbox) - Commit message: "feat(sandbox): add copy_on_write strategy stub"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add copy_on_write strategy skeleton with TODOs for large-project optimization. + - [ ] Code [Luis]: Raise explicit NotImplementedError with guidance on when it will be available. + - [ ] Docs [Luis]: Document that copy_on_write is stubbed for post-M1 work. + - [ ] Tests (Behave) [Rui]: Add scenario that selecting copy_on_write raises NotImplementedError with clear message. + - [ ] Tests (Robot) [Rui]: Add Robot test verifying stub error output. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/sandbox_stub_bench.py` (baseline no-op). + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(sandbox): add copy_on_write strategy stub"`. - [ ] **Stage B1: Project Data Model** (Day 1-2) **[Hamza - Python Expert, RDF Background]** @@ -3476,1343 +3296,273 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation **WEEK 2 - CRITICAL FOR MVP** -- [ ] **Stage C1: Actor YAML Schema Formalization** (Day 5-6) **[Aditya - Domain Expert]** - - **SEQUENTIAL ORDER**: C1.1 (Enums) → C1.2 (Tool/Route models) → C1.3 (Context model) → C1.4 (ActorConfigSchema) → C1.5 (Examples) → C1.6 (Docs) → C1.7 (Tests) - - - [ ] Code: Formalize actor YAML schema - - [ ] **C1.1** [Aditya] Define core enums in `src/cleveragents/actor/schema.py`: - - [ ] **C1.1a** [Aditya] Create file with `ActorType` enum: - ```python - class ActorType(str, Enum): - """Type of actor determining execution behavior.""" - LLM = "llm" # Single LLM with system prompt - TOOL = "tool" # Collection of callable tools - GRAPH = "graph" # Multi-node StateGraph with routing - ``` - - [ ] Commit: "feat(actor): define ActorType enum" - - [ ] **C1.1b** [Aditya] Define `NodeType` enum: - ```python - class NodeType(str, Enum): - """Type of node in a graph actor.""" - AGENT = "agent" # LLM agent node - TOOL = "tool" # Tool execution node - CONDITIONAL = "conditional" # Routing/conditional node - SUBGRAPH = "subgraph" # Nested actor reference - ``` - - [ ] Commit: "feat(actor): define NodeType enum" - - [ ] **C1.1c** [Aditya] Define `ContextView` enum: - ```python - class ContextView(str, Enum): - """Role-based context filtering for actors.""" - STRATEGIST = "strategist" # High-level architecture, READMEs - EXECUTOR = "executor" # Precise code sections for edits - REVIEWER = "reviewer" # Diffs, tests, style guides - FULL = "full" # Complete context (default) - ``` - - [ ] Commit: "feat(actor): define ContextView enum" - - [ ] **C1.2** [Aditya] Define tool and route models: - - [ ] **C1.2a** [Aditya] Define `ToolParameter` model: - ```python - class ToolParameter(BaseModel): - """Parameter definition for inline tool.""" - name: str = Field(..., description="Parameter name") - type: str = Field(..., description="JSON Schema type (string, integer, object, etc.)") - description: str = Field(..., description="What this parameter is for") - required: bool = Field(default=True) - default: Any = Field(default=None) - enum: list[str] | None = Field(default=None, description="Allowed values") - ``` - - [ ] Commit: "feat(actor): define ToolParameter model" - - [ ] **C1.2b** [Aditya] Define `ToolDefinition` model: - ```python - class ToolDefinition(BaseModel): - """Inline tool/skill definition in actor YAML.""" - name: str = Field(..., min_length=1, max_length=100, description="Tool identifier") - description: str = Field(..., description="What the tool does (shown to LLM)") - parameters: list[ToolParameter] = Field(default_factory=list) - returns: str = Field(default="Any", description="Return type documentation") - code: str = Field(..., description="Python code to execute") - timeout_seconds: int = Field(default=30, ge=1, le=300) - - @field_validator('code') - @classmethod - def validate_code_syntax(cls, v: str) -> str: - """Validate Python syntax without executing.""" - try: - compile(v, '', 'exec') - except SyntaxError as e: - raise ValueError(f"Invalid Python syntax: {e}") - return v - ``` - - [ ] Commit: "feat(actor): define ToolDefinition model with code validation" - - [ ] **C1.2c** [Aditya] Define `EdgeDefinition` model: - ```python - class EdgeDefinition(BaseModel): - """Edge in actor graph topology.""" - source: str = Field(..., description="Source node name") - target: str = Field(..., description="Target node name") - condition: str | None = Field(default=None, description="Python expression for conditional routing") - label: str | None = Field(default=None, description="Edge label for visualization") - ``` - - [ ] Commit: "feat(actor): define EdgeDefinition model" - - [ ] **C1.2d** [Aditya] Define `NodeDefinition` model: - ```python - class NodeDefinition(BaseModel): - """Node in actor graph.""" - name: str = Field(..., description="Unique node identifier") - type: NodeType = Field(..., description="Type of node") - # For agent nodes: - model: str | None = Field(default=None, description="Model name for agent nodes") - system_prompt: str | None = Field(default=None) - tools: list[str] | None = Field(default=None, description="Tool names available to this agent") - # For tool nodes: - tool: str | None = Field(default=None, description="Tool name to execute") - # For subgraph nodes: - actor: str | None = Field(default=None, description="Actor reference (e.g., local/other-actor)") - ``` - - [ ] Commit: "feat(actor): define NodeDefinition model" - - [ ] **C1.2e** [Aditya] Define `RouteDefinition` model: - ```python - class RouteDefinition(BaseModel): - """Complete graph topology definition.""" - nodes: list[NodeDefinition] = Field(..., min_length=1) - edges: list[EdgeDefinition] = Field(default_factory=list) - entry_point: str = Field(..., description="Name of starting node") - - @model_validator(mode='after') - def validate_topology(self) -> Self: - """Validate graph is well-formed.""" - node_names = {n.name for n in self.nodes} - if self.entry_point not in node_names: - raise ValueError(f"entry_point '{self.entry_point}' not in nodes") - for edge in self.edges: - if edge.source not in node_names: - raise ValueError(f"Edge source '{edge.source}' not in nodes") - if edge.target not in node_names: - raise ValueError(f"Edge target '{edge.target}' not in nodes") - return self - ``` - - [ ] Commit: "feat(actor): define RouteDefinition with topology validation" - - [ ] **C1.3** [Aditya] Define context/memory configuration: - - [ ] **C1.3a** [Aditya] Define `MemoryConfig` model: - ```python - class MemoryConfig(BaseModel): - """Memory/conversation history settings.""" - enabled: bool = Field(default=True, description="Whether to maintain history") - max_turns: int = Field(default=20, ge=1, le=100, description="Max conversation turns") - summarization_threshold: int = Field(default=15, description="Turns before summarizing") - include_system_messages: bool = Field(default=True) - ``` - - [ ] Commit: "feat(actor): define MemoryConfig model" - - [ ] **C1.3b** [Aditya] Define `ContextConfigSchema` model: - ```python - class ContextConfigSchema(BaseModel): - """Context window configuration for actor.""" - context_window_fraction: float = Field(default=0.8, ge=0.1, le=1.0, - description="Fraction of model's context window to use") - context_view: ContextView = Field(default=ContextView.FULL, - description="Role-based context filtering") - include_file_patterns: list[str] = Field(default_factory=list, - description="Glob patterns for files to always include") - exclude_file_patterns: list[str] = Field(default_factory=list, - description="Glob patterns for files to never include") - max_file_size_kb: int = Field(default=100, description="Max file size to include") - ``` - - [ ] Commit: "feat(actor): define ContextConfigSchema model" - - [ ] **C1.4** [Aditya] Define main `ActorConfigSchema`: - - [ ] **C1.4a** [Aditya] Create comprehensive model: - ```python - class ActorConfigSchema(BaseModel): - """Complete actor configuration from YAML.""" - # Identity - version: str = Field(default="3", description="Config schema version") - name: str = Field(..., description="Actor name (without namespace)") - namespace: str = Field(default="local") - description: str | None = Field(default=None) - tags: list[str] = Field(default_factory=list) - - # Type and provider - type: ActorType = Field(..., description="Actor type") - model: str | None = Field(default=None, description="LLM model name") - provider: str | None = Field(default=None, description="Provider: openai, anthropic, etc.") - - # LLM configuration - system_prompt: str | None = Field(default=None) - temperature: float = Field(default=0.7, ge=0.0, le=2.0) - max_tokens: int | None = Field(default=None) - - # Tools/skills - tools: list[ToolDefinition] = Field(default_factory=list, - description="Inline tool definitions") - builtin_tools: list[str] = Field(default_factory=list, - description="Names of built-in tools to include") - mcp_servers: list[str] = Field(default_factory=list, - description="MCP server identifiers to connect") - - # Graph topology (for type=GRAPH) - routes: RouteDefinition | None = Field(default=None) - - # Memory and context - memory: MemoryConfig = Field(default_factory=MemoryConfig) - context: ContextConfigSchema = Field(default_factory=ContextConfigSchema) - - # Execution - timeout_seconds: int = Field(default=300, description="Total execution timeout") - max_iterations: int = Field(default=50, description="Max LLM calls per invocation") - - @model_validator(mode='after') - def validate_type_requirements(self) -> Self: - """Validate fields based on actor type.""" - if self.type == ActorType.LLM: - if not self.model: - raise ValueError("LLM actors require 'model' field") - if self.type == ActorType.GRAPH: - if not self.routes: - raise ValueError("GRAPH actors require 'routes' field") - return self - - model_config = ConfigDict(extra='forbid') # Reject unknown fields - ``` - - [ ] Commit: "feat(actor): define ActorConfigSchema with validation" - - [ ] **C1.4b** [Aditya] Add YAML loading helper: - ```python - @classmethod - def from_yaml(cls, path: Path | str) -> "ActorConfigSchema": - """Load and validate actor config from YAML file.""" - import yaml - with open(path) as f: - data = yaml.safe_load(f) - return cls.model_validate(data) - - def to_yaml(self) -> str: - """Serialize config to YAML string.""" - import yaml - return yaml.dump(self.model_dump(exclude_none=True), sort_keys=False) - ``` - - [ ] Commit: "feat(actor): add YAML serialization helpers" - - [ ] **C1.5** [Aditya] Create comprehensive example actors in `examples/actors/`: - - [ ] **C1.5a** [Aditya] Create `simple_llm_actor.yaml`: - ```yaml - version: "3" - name: simple-assistant - namespace: local - description: Basic LLM assistant with no tools - type: llm - model: gpt-4-turbo - provider: openai - system_prompt: | - You are a helpful coding assistant. Answer questions - concisely and provide code examples when appropriate. - temperature: 0.7 - memory: - enabled: true - max_turns: 10 - ``` - - [ ] Commit: "docs(examples): add simple_llm_actor.yaml" - - [ ] **C1.5b** [Aditya] Create `tool_actor.yaml`: - ```yaml - version: "3" - name: file-reader - namespace: local - description: Actor that can read and search files - type: llm - model: gpt-4-turbo - system_prompt: | - You can read and search files to answer questions. - tools: - - name: read_file - description: Read contents of a file - parameters: - - name: path - type: string - description: Path to file - code: | - result = context.get_file(input_data["path"]) - - name: search_files - description: Search for pattern in files - parameters: - - name: pattern - type: string - description: Regex pattern to search - code: | - result = context.search_files("**/*", input_data["pattern"]) - builtin_tools: - - list_directory - ``` - - [ ] Commit: "docs(examples): add tool_actor.yaml" - - [ ] **C1.5c** [Aditya] Create `graph_actor.yaml`: - ```yaml - version: "3" - name: research-writer - namespace: local - description: Multi-step research and writing workflow - type: graph - routes: - entry_point: planner - nodes: - - name: planner - type: agent - model: gpt-4-turbo - system_prompt: Break down the writing task into research topics. - - name: researcher - type: agent - model: gpt-4-turbo - system_prompt: Research the assigned topic thoroughly. - tools: [search_files, read_file] - - name: writer - type: agent - model: gpt-4-turbo - system_prompt: Write content based on research findings. - - name: router - type: conditional - edges: - - source: planner - target: router - - source: router - target: researcher - condition: "state.needs_research" - - source: router - target: writer - condition: "not state.needs_research" - - source: researcher - target: router - ``` - - [ ] Commit: "docs(examples): add graph_actor.yaml" - - [ ] **C1.5d** [Aditya] Create `hierarchical_actor.yaml`: - ```yaml - version: "3" - name: code-reviewer - namespace: local - description: Hierarchical actor that delegates to specialists - type: graph - routes: - entry_point: coordinator - nodes: - - name: coordinator - type: agent - model: gpt-4-turbo - system_prompt: | - Coordinate code review by delegating to specialists. - - name: security-check - type: subgraph - actor: local/security-analyzer - - name: style-check - type: subgraph - actor: local/style-checker - - name: aggregator - type: agent - model: gpt-4-turbo - system_prompt: Combine specialist feedback into final review. - edges: - - source: coordinator - target: security-check - - source: coordinator - target: style-check - - source: security-check - target: aggregator - - source: style-check - target: aggregator - ``` - - [ ] Commit: "docs(examples): add hierarchical_actor.yaml" - - [ ] **C1.5e** [Aditya] Create `strategy_actor.yaml`: - ```yaml - version: "3" - name: default-strategist - namespace: cleveragents - description: Default strategist for Strategize phase - type: llm - model: gpt-4-turbo - temperature: 0.3 # Lower for more consistent strategy - system_prompt: | - You are a technical strategist. Given a task description and codebase context, - create a detailed plan with specific steps. - - Your output must include: - 1. High-level approach explanation - 2. Ordered list of steps with file paths - 3. Dependencies between steps - 4. Risk assessment - 5. Decisions with alternatives considered - - Format decisions as: - DECISION: - CHOSEN: - ALTERNATIVES: - RATIONALE: - context: - context_view: strategist - include_file_patterns: - - "README.md" - - "**/README.md" - - "docs/**/*.md" - ``` - - [ ] Commit: "docs(examples): add strategy_actor.yaml" - - [ ] **C1.5f** [Aditya] Create `execution_actor.yaml`: - ```yaml - version: "3" - name: default-executor - namespace: cleveragents - description: Default executor for Execute phase - type: llm - model: gpt-4-turbo - temperature: 0.2 # Low for precise code generation - system_prompt: | - You are a code executor. Given a strategy and file context, - implement the required changes using the provided tools. - - RULES: - - Use tools to read files before editing - - Use edit_file for targeted changes, write_file for new files - - Always verify changes compile/parse correctly - - Document each change with clear commit messages - tools: - - name: edit_file - description: Make targeted edits to an existing file - parameters: - - name: path - type: string - - name: edits - type: array - code: | - result = context.edit_file(input_data["path"], input_data["edits"]) - builtin_tools: - - read_file - - write_file - - delete_file - - list_directory - - search_files - context: - context_view: executor - max_iterations: 100 # Allow more iterations for complex tasks - ``` - - [ ] Commit: "docs(examples): add execution_actor.yaml" - - [ ] **C1.6** [Aditya] Write comprehensive documentation: - - [ ] **C1.6a** [Aditya] Create `docs/reference/actor_configuration.md`: - - [ ] Full YAML schema reference with all fields - - [ ] Type-specific requirements (LLM vs GRAPH) - - [ ] Tool definition syntax and examples - - [ ] Memory and context configuration - - [ ] Commit: "docs: add actor configuration reference" - - [ ] **C1.6b** [Aditya] Add examples section: - - [ ] Example for each actor type - - [ ] Common patterns (research, review, generation) - - [ ] Anti-patterns to avoid - - [ ] Commit: "docs: add actor configuration examples" - - [ ] **C1.6c** [Aditya] Add migration guide: - - [ ] Changes from v2 format - - [ ] Automated migration script (if needed) - - [ ] Commit: "docs: add actor config migration guide" - - [ ] Tests: Behave scenarios for actor schema validation - - [ ] **C1.7** [Rui] Write Behave scenarios in `features/actor_schema.feature`: - - [ ] **C1.7a** [Rui] Valid config scenarios: - - [ ] Scenario: Simple LLM actor config validates successfully - - [ ] Scenario: Tool actor with inline code validates - - [ ] Scenario: Graph actor with complete topology validates - - [ ] Scenario: Actor with MCP servers configured validates - - [ ] Commit: "test(behave): add valid actor config scenarios" - - [ ] **C1.7b** [Rui] Invalid config scenarios: - - [ ] Scenario: LLM actor without model field fails - - [ ] Scenario: Graph actor without routes fails - - [ ] Scenario: Tool with invalid Python syntax fails - - [ ] Scenario: Graph with missing entry_point fails - - [ ] Scenario: Edge referencing non-existent node fails - - [ ] Commit: "test(behave): add invalid actor config scenarios" - - [ ] **C1.7c** [Rui] YAML loading scenarios: - - [ ] Scenario: Load actor config from YAML file - - [ ] Scenario: Invalid YAML syntax produces clear error - - [ ] Scenario: Unknown fields in YAML are rejected - - [ ] Commit: "test(behave): add YAML loading scenarios" +**Parallel Group C0: Tool Registry + Validation System [Jeff + Luis]** (start Day 5; precedes C1/C3) + **PARALLEL SUBTRACK C0.domain [Jeff]**: Tool + Validation domain models + schemas + **PARALLEL SUBTRACK C0.registry [Luis]**: Tool registry persistence + repositories + **PARALLEL SUBTRACK C0.cli [Rui]**: CLI commands for tools/validations + **SEQUENTIAL MERGE NOTE**: C0.domain must land before C0.registry/cli; C0.registry before C3 context wiring. + - [ ] **COMMIT (Owner: Jeff | Group: C0.domain) - Commit message: "feat(tool): add tool and validation domain models"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Add `Tool` model with namespaced name, description, source type, input/output JSON schema, and capability metadata (read/write/checkpointable). + - [ ] Code [Jeff]: Add `ResourceBinding` model with slot definitions and binding modes (context, static, parameter). + - [ ] Code [Jeff]: Add `Validation` model as Tool subtype with `mode`, `wraps`, and `transform` fields; enforce read-only constraints. + - [ ] Code [Jeff]: Add enums for ToolSource, ToolType (tool/validation), and ValidationMode. + - [ ] Docs [Jeff]: Add `docs/reference/tool_model.md` and `docs/reference/validation_model.md` with examples. + - [ ] Tests (Behave) [Rui]: Add `features/tool_model.feature` for schema validation, resource binding rules, and validation constraints. + - [ ] Tests (Robot) [Rui]: Add `robot/tool_model.robot` smoke tests for model creation. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/tool_model_bench.py` for schema validation throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(tool): add tool and validation domain models"`. + - [ ] **COMMIT (Owner: Luis | Group: C0.registry) - Commit message: "feat(tool): add tool registry persistence"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Luis]: Add DB tables for `tools`, `tool_bindings`, and `validation_attachments` with indexes on namespaced name and type. + - [ ] Code [Luis]: Implement ToolRepository + ValidationAttachmentRepository with list/show filters. + - [ ] Code [Luis]: Add ToolRegistryService for register/update/remove/list/show with name conflict checks. + - [ ] Docs [Luis]: Update `docs/reference/database_schema.md` with tool/validation tables. + - [ ] Tests (Behave) [Rui]: Add `features/tool_registry.feature` for register/update/remove and validation-only constraints. + - [ ] Tests (Robot) [Rui]: Add `robot/tool_registry.robot` for list/show smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/tool_registry_bench.py` for registry list performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(tool): add tool registry persistence"`. + - [ ] **COMMIT (Owner: Jeff | Group: C0.binding) - Commit message: "feat(tool): add resource binding resolution"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Jeff]: Implement binding resolution for contextual, static, and parameter bindings with type compatibility checks. + - [ ] Code [Jeff]: Add resolution helpers for resource name/ULID lookup and project-scoped filtering. + - [ ] Docs [Jeff]: Add `docs/reference/tool_bindings.md` with resolution order and examples. + - [ ] Tests (Behave) [Rui]: Add binding resolution scenarios (context vs static vs parameter). + - [ ] Tests (Robot) [Rui]: Add Robot test resolving a bound resource by name. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/binding_resolution_bench.py` for resolution latency. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(tool): add resource binding resolution"`. + - [ ] **COMMIT (Owner: Rui | Group: C0.cli) - Commit message: "feat(cli): add tool and validation commands"** (COMMIT TASK: only check after every subtask below is complete, `nox` and coverage succeed, and the commit is created) + - [ ] Code [Rui]: Implement `agents tool add/remove/list/show` with YAML config input and `--type` filter. + - [ ] Code [Rui]: Implement `agents validation add/attach/detach` commands and enforce validation-only name use. + - [ ] Docs [Rui]: Update CLI reference with tool/validation commands and output format. + - [ ] Tests (Behave) [Rui]: Add CLI scenarios for tool/validation registration and attachment. + - [ ] Tests (Robot) [Rui]: Add Robot CLI suites for tool and validation commands. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/tool_cli_bench.py` for CLI parsing overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "feat(cli): add tool and validation commands"`. -- [ ] **Stage C2: Actor Loading & Compilation** (Day 6-8) **[Aditya]** - - **SEQUENTIAL ORDER**: C2.1 (Config parser) → C2.2 (CompiledActor) → C2.3 (LLM compiler) → C2.4 (Tool compiler) → C2.5 (Graph compiler) → C2.6 (Reference resolution) → C2.7 (Registry) → C2.8 (Tests) - - - [ ] Code: Enhance actor loading and compilation to LangGraph - - [ ] **C2.1** [Aditya] Create config parser in `src/cleveragents/actor/config.py`: - - [ ] **C2.1a** [Aditya] Define `ActorConfigParser` class: - ```python - class ActorConfigParser: - """Parse and validate actor configurations.""" - - def __init__(self, registry: "ActorRegistry"): - self._registry = registry - - def parse_file(self, path: Path) -> ActorConfigSchema: - """Parse actor config from YAML file.""" - return ActorConfigSchema.from_yaml(path) - - def parse_string(self, content: str) -> ActorConfigSchema: - """Parse actor config from YAML string.""" - import yaml - data = yaml.safe_load(content) - return ActorConfigSchema.model_validate(data) - ``` - - [ ] Commit: "feat(actor): add ActorConfigParser scaffold" - - [ ] **C2.1b** [Aditya] Add tools section validation: - - [ ] Validate each tool has unique name - - [ ] Validate tool code compiles (syntax check) - - [ ] Validate tool parameters have valid JSON Schema types - - [ ] Commit: "feat(actor): add tools section validation" - - [ ] **C2.1c** [Aditya] Add routes section validation: - - [ ] Validate all node names are unique - - [ ] Validate entry_point exists in nodes - - [ ] Validate all edge sources/targets exist - - [ ] Check for unreachable nodes (warning) - - [ ] Commit: "feat(actor): add routes section validation" - - [ ] **C2.1d** [Aditya] Add actor reference validation: - - [ ] For subgraph nodes, validate `actor` field is present - - [ ] Validate referenced actor exists in registry - - [ ] Build dependency graph for circular reference detection - - [ ] Commit: "feat(actor): add actor reference validation" - - [ ] **C2.2** [Aditya] Define `CompiledActor` in `src/cleveragents/actor/compiled.py`: - - [ ] **C2.2a** [Aditya] Create CompiledActor dataclass: - ```python - @dataclass - class CompiledActor: - """A compiled actor ready for execution.""" - config: ActorConfigSchema - graph: StateGraph # LangGraph StateGraph - runnable: CompiledStateGraph # Compiled version for execution - tools: dict[str, Callable] # Name -> callable tool functions - referenced_actors: list[str] # Actor names this depends on - compiled_at: datetime - - def invoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: - """Execute the actor graph with input.""" - return self.runnable.invoke(input_data, config) - - async def ainvoke(self, input_data: dict, config: RunnableConfig | None = None) -> dict: - """Execute the actor graph asynchronously.""" - return await self.runnable.ainvoke(input_data, config) - ``` - - [ ] Commit: "feat(actor): define CompiledActor dataclass" - - [ ] **C2.3** [Aditya] Create `ActorCompiler` in `src/cleveragents/actor/compiler.py`: - - [ ] **C2.3a** [Aditya] Define compiler class scaffold: - ```python - class ActorCompiler: - """Compile actor configs into executable LangGraph graphs.""" - - def __init__( - self, - model_factory: ModelFactory, - skill_registry: SkillRegistry, - mcp_manager: MCPServerManager | None = None - ): - self._model_factory = model_factory - self._skill_registry = skill_registry - self._mcp_manager = mcp_manager - self._compiled_cache: dict[str, CompiledActor] = {} - ``` - - [ ] Commit: "feat(actor): add ActorCompiler scaffold" - - [ ] **C2.3b** [Aditya] Implement main `compile()` method: - ```python - def compile(self, config: ActorConfigSchema) -> CompiledActor: - """Compile actor config into executable graph.""" - # Check cache - cache_key = f"{config.namespace}/{config.name}" - if cache_key in self._compiled_cache: - return self._compiled_cache[cache_key] - - # Compile based on type - match config.type: - case ActorType.LLM: - graph = self._compile_llm_actor(config) - case ActorType.TOOL: - graph = self._compile_tool_actor(config) - case ActorType.GRAPH: - graph = self._compile_graph_actor(config) - - # Build tools dict - tools = self._build_tools(config) - - # Create CompiledActor - compiled = CompiledActor( - config=config, - graph=graph, - runnable=graph.compile(), - tools=tools, - referenced_actors=self._get_referenced_actors(config), - compiled_at=datetime.utcnow() - ) - - # Cache and return - self._compiled_cache[cache_key] = compiled - return compiled - ``` - - [ ] Commit: "feat(actor): implement ActorCompiler.compile()" - - [ ] **C2.4** [Aditya] Implement LLM actor compilation: - - [ ] **C2.4a** [Aditya] Implement `_compile_llm_actor()`: - ```python - def _compile_llm_actor(self, config: ActorConfigSchema) -> StateGraph: - """Compile simple LLM actor into single-node graph.""" - from langgraph.graph import StateGraph, END - from langchain_core.messages import HumanMessage, SystemMessage - - # Create model - model = self._model_factory.create( - model_name=config.model, - provider=config.provider, - temperature=config.temperature, - max_tokens=config.max_tokens - ) - - # Bind tools if any - tools = self._build_tools(config) - if tools: - model = model.bind_tools(list(tools.values())) - - # Define state - class State(TypedDict): - messages: list[BaseMessage] - context: dict - - # Define agent node - def agent(state: State) -> State: - messages = state["messages"] - if config.system_prompt: - messages = [SystemMessage(content=config.system_prompt)] + messages - response = model.invoke(messages) - return {"messages": [response]} - - # Build graph - graph = StateGraph(State) - graph.add_node("agent", agent) - graph.set_entry_point("agent") - graph.add_edge("agent", END) - - return graph - ``` - - [ ] Commit: "feat(actor): implement LLM actor compilation" - - [ ] **C2.4b** [Aditya] Add memory support to LLM actors: - - [ ] If config.memory.enabled, wrap with memory checkpointer - - [ ] Configure message trimming based on max_turns - - [ ] Commit: "feat(actor): add memory support to LLM actors" - - [ ] **C2.5** [Aditya] Implement tool actor compilation: - - [ ] **C2.5a** [Aditya] Implement `_compile_tool_actor()`: - - [ ] Create ReAct-style agent with tools - - [ ] Configure tool calling loop - - [ ] Add tool nodes for each defined tool - - [ ] Commit: "feat(actor): implement tool actor compilation" - - [ ] **C2.5b** [Aditya] Implement tool node generation: - ```python - def _build_tools(self, config: ActorConfigSchema) -> dict[str, Callable]: - """Build callable tools from config.""" - tools = {} - - # Inline tools from YAML - for tool_def in config.tools: - tools[tool_def.name] = self._create_inline_tool(tool_def) - - # Built-in tools - for tool_name in config.builtin_tools: - tool = self._skill_registry.get_skill(tool_name) - if tool: - tools[tool_name] = tool.to_langchain_tool() - - # MCP tools - if config.mcp_servers and self._mcp_manager: - for server_id in config.mcp_servers: - mcp_tools = self._mcp_manager.get_tools(server_id) - tools.update(mcp_tools) - - return tools - ``` - - [ ] Commit: "feat(actor): implement tool building from config" - - [ ] **C2.6** [Aditya] Implement graph actor compilation: - - [ ] **C2.6a** [Aditya] Implement `_compile_graph_actor()`: - ```python - def _compile_graph_actor(self, config: ActorConfigSchema) -> StateGraph: - """Compile multi-node graph actor.""" - routes = config.routes - - # Define state type dynamically based on nodes - State = self._build_state_type(routes) - - # Create graph - graph = StateGraph(State) - - # Add nodes - for node_def in routes.nodes: - node_func = self._create_node(node_def, config) - graph.add_node(node_def.name, node_func) - - # Set entry point - graph.set_entry_point(routes.entry_point) - - # Add edges - for edge in routes.edges: - if edge.condition: - # Conditional edge - condition_func = self._parse_condition(edge.condition) - graph.add_conditional_edges( - edge.source, - condition_func, - {True: edge.target} - ) - else: - # Direct edge - graph.add_edge(edge.source, edge.target) - - return graph - ``` - - [ ] Commit: "feat(actor): implement graph actor compilation" - - [ ] **C2.6b** [Aditya] Implement node creation by type: - - [ ] Agent nodes: create LLM with optional tools - - [ ] Tool nodes: create tool execution wrapper - - [ ] Conditional nodes: create routing logic - - [ ] Subgraph nodes: compile and embed referenced actor - - [ ] Commit: "feat(actor): implement node creation by type" - - [ ] **C2.7** [Aditya] Implement actor reference resolution: - - [ ] **C2.7a** [Aditya] Add circular reference detection: - ```python - def _check_circular_references( - self, - config: ActorConfigSchema, - visited: set[str] | None = None - ) -> None: - """Detect circular actor references.""" - visited = visited or set() - actor_name = f"{config.namespace}/{config.name}" - - if actor_name in visited: - raise CircularReferenceError( - f"Circular reference detected: {' -> '.join(visited)} -> {actor_name}" - ) - - visited.add(actor_name) - - for ref in self._get_referenced_actors(config): - ref_config = self._registry.get_config(ref) - if ref_config: - self._check_circular_references(ref_config, visited.copy()) - ``` - - [ ] Commit: "feat(actor): add circular reference detection" - - [ ] **C2.7b** [Aditya] Implement recursive compilation: - - [ ] When compiling subgraph node, recursively compile referenced actor - - [ ] Cache compiled actors to avoid recompilation - - [ ] Pass context appropriately to subgraphs - - [ ] Commit: "feat(actor): implement recursive actor compilation" - - [ ] **C2.8** [Aditya] Update `ActorRegistry` to support compilation: - - [ ] **C2.8a** [Aditya] Add `get_compiled()` method: - ```python - def get_compiled(self, name: str) -> CompiledActor: - """Get compiled actor by name, compiling if needed.""" - # Check compiled cache - if name in self._compiled_cache: - cached = self._compiled_cache[name] - # Check if config changed - current_config = self.get_config(name) - if current_config and self._config_unchanged(name, current_config): - return cached - - # Load config - config = self.get_config(name) - if not config: - raise ActorNotFoundError(f"Actor '{name}' not found") - - # Compile - compiled = self._compiler.compile(config) - self._compiled_cache[name] = compiled - - return compiled - ``` - - [ ] Commit: "feat(actor): add ActorRegistry.get_compiled()" - - [ ] **C2.8b** [Aditya] Add cache invalidation: - - [ ] Monitor actor YAML files for changes - - [ ] Clear cache entry when config file modified - - [ ] Clear dependent actors when base actor changes - - [ ] Commit: "feat(actor): add compiled actor cache invalidation" - - [ ] Tests: Behave scenarios for actor compilation - - [ ] **C2.9** [Rui] Write Behave scenarios in `features/actor_compilation.feature`: - - [ ] **C2.9a** [Rui] LLM actor compilation scenarios: - - [ ] Scenario: Compile simple LLM actor creates valid graph - - [ ] Given actor config with type=llm and model=gpt-4 - - [ ] When I compile the actor - - [ ] Then CompiledActor is returned - - [ ] And graph has single agent node - - [ ] And runnable can be invoked - - [ ] Scenario: LLM actor with tools binds tools correctly - - [ ] Commit: "test(behave): add LLM actor compilation scenarios" - - [ ] **C2.9b** [Rui] Tool actor compilation scenarios: - - [ ] Scenario: Compile tool actor with inline code works - - [ ] Given actor config with inline tool definitions - - [ ] When I compile the actor - - [ ] Then tools dict contains the defined tools - - [ ] And tools are callable - - [ ] Scenario: Built-in tools are included - - [ ] Commit: "test(behave): add tool actor compilation scenarios" - - [ ] **C2.9c** [Rui] Graph actor compilation scenarios: - - [ ] Scenario: Compile graph actor creates correct topology - - [ ] Given actor config with routes defining 3 nodes - - [ ] When I compile the actor - - [ ] Then graph has 3 nodes - - [ ] And edges match route definition - - [ ] And entry_point is set correctly - - [ ] Scenario: Conditional edges work correctly - - [ ] Commit: "test(behave): add graph actor compilation scenarios" - - [ ] **C2.9d** [Rui] Reference resolution scenarios: - - [ ] Scenario: Actor referencing other actor compiles recursively - - [ ] Given actor A references actor B as subgraph - - [ ] When I compile actor A - - [ ] Then actor B is also compiled - - [ ] And actor B graph is embedded in actor A - - [ ] Scenario: Circular reference detected and errors - - [ ] Given actor A references B and B references A - - [ ] When I try to compile actor A - - [ ] Then CircularReferenceError is raised - - [ ] And error message shows the cycle - - [ ] Commit: "test(behave): add reference resolution scenarios" - - [ ] **C2.9e** [Rui] Error scenarios: - - [ ] Scenario: Invalid actor config produces clear error - - [ ] Scenario: Missing referenced actor produces clear error - - [ ] Scenario: Invalid tool code produces clear error - - [ ] Commit: "test(behave): add compilation error scenarios" +**Parallel Group C1: Actor Schema & Examples [Aditya + Jeff]** (start Day 5; C2 depends on this) +- [ ] **COMMIT (Owner: Aditya | Group: C1.schema) - Commit message: "feat(actor): add actor yaml schema models"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Aditya]: Add schema models (ActorType, NodeType, ContextView, ToolDefinition, RouteDefinition, ActorConfigSchema) with strict validation. + - [ ] Code [Aditya]: Add tool-node schema fields that reference Tool Registry names, including validation nodes. + - [ ] Code [Aditya]: Add YAML load/serialize helpers and schema version guard. + - [ ] Docs [Aditya]: Add `docs/reference/actors_schema.md` with field definitions, tool node semantics, and graph constraints. + - [ ] Tests (Behave) [Rui]: Add `features/actor_schema.feature` scenarios for validation and topology errors. + - [ ] Tests (Robot) [Rui]: Add `robot/actor_schema.robot` YAML load smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/actor_schema_bench.py` for YAML validation cost. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "feat(actor): add actor yaml schema models"`. +- [ ] **COMMIT (Owner: Aditya | Group: C1.examples) - Commit message: "docs(actor): add actor yaml examples"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Docs [Aditya]: Add `docs/reference/actors_examples.md` with strategist, executor, reviewer, tool-only, validation-node, and graph YAML examples. + - [ ] Tests (Behave) [Rui]: Add `features/actor_examples.feature` to ensure all examples validate. + - [ ] Tests (Robot) [Rui]: Add `robot/actor_examples.robot` to load each example. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/actor_examples_load_bench.py` for YAML load throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "docs(actor): add actor yaml examples"`. -- [ ] **Stage C3: Skill Execution Framework** (Day 5-6) **[Jeff + Aditya - Critical Path]** - - [ ] Code: Implement skill execution framework - - [ ] **C3.1** [Jeff] Define `Skill` protocol in `src/cleveragents/actor/skills/protocol.py`: - - [ ] **C3.1a** Create abstract base using `typing.Protocol`: - ```python - class Skill(Protocol): - @property - def name(self) -> str: ... - @property - def description(self) -> str: ... - @property - def parameters(self) -> dict[str, Any]: ... # JSON Schema - @property - def metadata(self) -> SkillMetadata: ... - - async def execute( - self, - input_data: dict[str, Any], - context: SkillContext - ) -> SkillResult: ... - ``` - - [ ] **C3.1b** Define `SkillResult` dataclass: - - [ ] Field `success: bool` - whether execution succeeded - - [ ] Field `result: Any` - return value if successful - - [ ] Field `error: str | None` - error message if failed - - [ ] Field `changes: list[Change]` - changes made to resources - - [ ] Field `duration_ms: int` - execution time - - [ ] **C3.1c** Add docstrings explaining contract for skill implementers - - [ ] **C3.2** [Jeff] Define `SkillMetadata` Pydantic model in `src/cleveragents/actor/skills/metadata.py`: - - [ ] **C3.2a** Core capability fields: - - [ ] Field `read_only: bool = False` - only performs read operations - - [ ] Field `writes: bool = False` - can modify resources - - [ ] Field `write_scope: list[str] = []` - glob patterns for writable paths - - [ ] Field `idempotent: bool = False` - repeated calls produce same result - - [ ] Field `checkpointable: bool = False` - supports checkpoint/rollback - - [ ] Field `side_effects: list[str] = []` - external side effects (e.g., "network", "subprocess") - - [ ] **C3.2b** Safety and control fields: - - [ ] Field `human_approval_required: bool = False` - requires user confirmation - - [ ] Field `rate_limit: RateLimit | None = None` - calls per minute/hour - - [ ] Field `cost_profile: CostProfile | None = None` - estimated cost per call - - [ ] Field `timeout_seconds: int = 30` - maximum execution time - - [ ] **C3.2c** Define `RateLimit` and `CostProfile` models - - [ ] **C3.2d** Add validation to ensure `writes=True` if `write_scope` is non-empty - - [ ] **C3.3** [Jeff] Create `SkillContext` in `src/cleveragents/actor/skills/context.py`: - - [ ] **C3.3a** Define context fields: - - [ ] Field `plan_id: str` - current plan ULID - - [ ] Field `plan: Plan` - full plan object for reference - - [ ] Field `project: Project` - target project - - [ ] Field `resources: list[Resource]` - available resources - - [ ] Field `sandbox_manager: SandboxManager` - for sandbox access - - [ ] Field `changeset: ChangeSet` - accumulating changes - - [ ] Field `invocation_tracker: SkillInvocationTracker` - tracking calls - - [ ] Field `logger: logging.Logger` - skill-specific logger - - [ ] **C3.3b** Implement convenience methods: - - [ ] Method `get_file(path: str, resource: str | None = None) -> str`: - - [ ] Resolve path to sandboxed location - - [ ] Read and return file contents - - [ ] Raise FileNotFoundError if not exists - - [ ] Method `write_file(path: str, content: str, resource: str | None = None) -> Change`: - - [ ] Resolve path to sandboxed location - - [ ] Validate path against deny-list - - [ ] Create parent directories if needed - - [ ] Write content - - [ ] Create and record Change - - [ ] Return Change for tracking - - [ ] Method `edit_file(path: str, edits: list[Edit], resource: str | None = None) -> Change`: - - [ ] Read current content - - [ ] Apply edits sequentially - - [ ] Write modified content - - [ ] Record Change with edits - - [ ] Method `delete_file(path: str, resource: str | None = None) -> Change`: - - [ ] Validate file exists - - [ ] Delete file - - [ ] Record Change - - [ ] Method `list_files(pattern: str, resource: str | None = None) -> list[str]`: - - [ ] Resolve pattern to sandbox - - [ ] Return matching paths - - [ ] Method `search_files(pattern: str, content_pattern: str, resource: str | None = None) -> list[SearchResult]`: - - [ ] Search file contents with regex - - [ ] Return matches with file, line, context - - [ ] **C3.3c** Implement subplan spawning: - - [ ] Method `spawn_subplan(action: str, target_resources: list[str] | None = None, arguments: dict | None = None) -> str`: - - [ ] Validate action exists - - [ ] Create child plan with parent_plan_id = self.plan_id - - [ ] Queue subplan for execution - - [ ] Return subplan_id for tracking - - [ ] Record as `subplan_spawn` decision type - - [ ] **C3.3d** Implement read-only check enforcement: - - [ ] If plan.action.read_only is True, block all write operations - - [ ] Raise `ReadOnlyViolationError` if write attempted - - [ ] **C3.4** [Aditya] Implement `InlineSkillExecutor` in `src/cleveragents/actor/skills/inline_executor.py`: - - [ ] **C3.4a** Create class for executing inline Python code from actor YAML: - ```python - class InlineSkillExecutor: - def __init__(self, code: str, timeout: int = 30): - self.code = code - self.timeout = timeout - ``` - - [ ] **C3.4b** Create sandboxed execution environment: - - [ ] Restricted `__builtins__`: - - [ ] ALLOWED: `len`, `range`, `str`, `int`, `float`, `list`, `dict`, `set`, `tuple`, `bool`, `None`, `True`, `False`, `print`, `isinstance`, `hasattr`, `getattr`, `enumerate`, `zip`, `map`, `filter`, `sorted`, `reversed`, `any`, `all`, `min`, `max`, `sum`, `abs`, `round` - - [ ] BLOCKED: `open`, `exec`, `eval`, `compile`, `__import__`, `globals`, `locals`, `vars`, `dir`, `input` - - [ ] Inject `context: SkillContext` variable - - [ ] Inject `input_data: dict` variable - - [ ] Inject standard library modules: `re`, `json`, `datetime`, `collections`, `itertools`, `functools` - - [ ] **C3.4c** Execute code and capture result: - - [ ] Use `exec()` with restricted globals/locals - - [ ] Capture `result` variable as return value - - [ ] If no `result` variable, return None - - [ ] Wrap in asyncio.wait_for for timeout - - [ ] **C3.4d** Handle errors gracefully: - - [ ] Catch all exceptions during execution - - [ ] Convert to SkillResult with error message - - [ ] Include stack trace in error for debugging - - [ ] Log error with skill name and input - - [ ] **C3.4e** Add timeout support: - - [ ] Default 30 seconds - - [ ] Configurable via skill metadata - - [ ] Raise TimeoutError if exceeded - - [ ] **C3.5** [Aditya] Implement subplan spawning in skill context: - - [ ] **C3.5a** Update SkillContext.spawn_subplan to create real subplans: - - [ ] Call PlanLifecycleService.use_action() with parent_plan_id - - [ ] Set subplan's root_plan_id to parent's root_plan_id (or parent's id if root) - - [ ] Set subplan's automation_level from parent - - [ ] Return subplan_id - - [ ] **C3.5b** Add subplan tracking to parent plan: - - [ ] Store spawned subplan IDs in plan's execution_log - - [ ] Support querying all subplans of a plan - - [ ] **C3.5c** Add subplan completion handling: - - [ ] Parent plan can check subplan status - - [ ] Parent plan can collect subplan results - - [ ] Support waiting for subplan completion - - [ ] **C3.6** [Jeff + Luis] Implement built-in resource skills in `src/cleveragents/actor/skills/builtin/`: - - [ ] **C3.6a** [Jeff] Create base skill class in `__init__.py`: - ```python - class BuiltinSkill(ABC): - @abstractmethod - async def execute(self, input_data: dict, context: SkillContext) -> SkillResult: ... - - def _validate_path(self, path: str, context: SkillContext) -> str: - """Resolve and validate path against sandbox.""" - ... - ``` - - [ ] **C3.6b** [Jeff] File operation skills in `file_ops.py`: - - [ ] **ReadFileSkill**: - - [ ] Parameters: `path: str` (required) - - [ ] Metadata: `read_only=True` - - [ ] Implementation: resolve path, read via sandbox, return content - - [ ] Error handling: FileNotFoundError, PermissionError - - [ ] **WriteFileSkill**: - - [ ] Parameters: `path: str`, `content: str` - - [ ] Metadata: `writes=True, write_scope=['**/*']` - - [ ] Implementation: - - [ ] Validate path not in deny-list - - [ ] Create parent directories if needed - - [ ] Determine if create or modify operation - - [ ] Write content to sandbox - - [ ] Create Change record with operation type - - [ ] Record change in context.changeset - - [ ] Return: Change object with path and operation - - [ ] **EditFileSkill** (most complex - critical for coding): - - [ ] Parameters: `path: str`, `edits: list[Edit]` - - [ ] Metadata: `writes=True, idempotent=False` - - [ ] Implementation: - - [ ] Read original file content - - [ ] For each Edit in edits: - - [ ] If type=SEARCH_REPLACE: find `search` text, replace with `replace` - - [ ] If type=LINE_RANGE: replace lines start_line:end_line with content - - [ ] If type=INSERT_AFTER: insert content after matching line - - [ ] If type=INSERT_BEFORE: insert content before matching line - - [ ] If type=DELETE_LINES: remove lines start_line:end_line - - [ ] Track all changes made - - [ ] Write modified content - - [ ] Create Change with edits list - - [ ] Error handling: SearchTextNotFoundError, InvalidLineRangeError - - [ ] **DeleteFileSkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: delete file, create DELETE Change - - [ ] **MoveFileSkill**: - - [ ] Parameters: `source: str`, `destination: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: move file, create MOVE Change with new_path - - [ ] **CopyFileSkill**: - - [ ] Parameters: `source: str`, `destination: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: copy file, create CREATE Change - - [ ] **C3.6c** [Luis] Directory operation skills in `dir_ops.py`: - - [ ] **CreateDirectorySkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `writes=True` - - [ ] Implementation: create directory (and parents), record Change - - [ ] **ListDirectorySkill**: - - [ ] Parameters: `path: str`, `pattern: str = "*"`, `recursive: bool = False` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: list matching files/dirs, return paths - - [ ] **DeleteDirectorySkill**: - - [ ] Parameters: `path: str`, `recursive: bool = False` - - [ ] Metadata: `writes=True` - - [ ] Implementation: delete directory, record DELETE Changes for all contents - - [ ] **C3.6d** [Luis] Search skills in `search_ops.py`: - - [ ] **SearchFilesSkill**: - - [ ] Parameters: `pattern: str` (glob), `content_pattern: str` (regex), `max_results: int = 100` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: - - [ ] Find files matching glob pattern - - [ ] Search each file for content_pattern - - [ ] Return list of SearchResult(file, line_number, line_content, context) - - [ ] **FindDefinitionSkill** (uses tree-sitter for AST): - - [ ] Parameters: `symbol: str`, `language: str | None = None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: - - [ ] Parse files with tree-sitter - - [ ] Find function/class/variable definitions - - [ ] Return list of Location(file, line, column, snippet) - - [ ] **FindReferencesSkill**: - - [ ] Parameters: `symbol: str` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: find all usages of symbol - - [ ] **GetFileInfoSkill**: - - [ ] Parameters: `path: str` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: return FileInfo(size, mtime, language, line_count) - - [ ] **C3.6e** [Hamza] Git operation skills in `git_ops.py`: - - [ ] **GitStatusSkill**: - - [ ] Parameters: (none) - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git status --porcelain`, parse output - - [ ] **GitDiffSkill**: - - [ ] Parameters: `path: str | None = None`, `staged: bool = False` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git diff [--staged] [path]` - - [ ] **GitLogSkill**: - - [ ] Parameters: `count: int = 10`, `path: str | None = None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git log --oneline -n {count}` - - [ ] **GitBlameSkill**: - - [ ] Parameters: `path: str`, `start_line: int | None`, `end_line: int | None` - - [ ] Metadata: `read_only=True` - - [ ] Implementation: run `git blame -L {start},{end} {path}` - - [ ] **C3.6f** [Jeff] All built-in skills must implement: - - [ ] Full SkillMetadata with accurate capability flags - - [ ] Proper error handling with descriptive messages - - [ ] Logging of all operations for debugging - - [ ] Path validation against sandbox and deny-lists - - [ ] Change recording for all write operations - - [ ] Async execution support - - [ ] **C3.6g** [Jeff] Create skill registration in `src/cleveragents/actor/skills/registry.py`: - - [ ] `BuiltinSkillRegistry` singleton with all built-in skills - - [ ] Method `get_skill(name: str) -> Skill | None` - - [ ] Method `list_skills() -> list[Skill]` - - [ ] Method `list_by_capability(read_only: bool = None, writes: bool = None) -> list[Skill]` - - [ ] Register all C3.6 skills on import - - [ ] **C3.7** [Aditya] Implement MCP skill adapter in `src/cleveragents/actor/skills/mcp_adapter.py`: - - [ ] **C3.7a** `MCPServerConnection` class: - - [ ] Connect to MCP server via stdio or SSE transport - - [ ] List available tools from server - - [ ] Call tools with JSON-RPC - - [ ] Handle server lifecycle (start/stop) - - [ ] **C3.7b** `MCPSkillAdapter` class: - - [ ] Wrap MCP tool as CleverAgents Skill - - [ ] Infer SkillMetadata from MCP tool schema - - [ ] Intercept calls for sandbox path rewriting - - [ ] Record changes when MCP tool modifies resources - - [ ] **C3.7c** Actor YAML integration: - - [ ] Parse `mcp_servers` config in actor definition - - [ ] Auto-register MCP tools as skills in actor context - - [ ] Environment variable substitution for secrets - - [ ] Tests: Integration tests for skill execution - - [ ] **C3.8** [Rui] Write Behave scenarios in `features/skill_execution.feature`: - - [ ] Scenario: Execute inline Python skill with context - - [ ] Scenario: Skill can read files from sandbox - - [ ] Scenario: Skill can write files to sandbox - - [ ] Scenario: Skill with invalid code produces error - - [ ] Scenario: Skill timeout prevents infinite loops - - [ ] Scenario: spawn_subplan creates child plan - - [ ] **C3.9** [Rui] Write Behave scenarios in `features/builtin_skills.feature`: - - [ ] Scenario: WriteFileSkill creates file and records Change - - [ ] Scenario: EditFileSkill applies search/replace edit - - [ ] Scenario: DeleteFileSkill removes file and records Change - - [ ] Scenario: MoveFileSkill renames file and records Change - - [ ] Scenario: ListDirectorySkill returns matching files - - [ ] Scenario: SearchFilesSkill finds content matches - - [ ] Scenario: Skill respects deny-list patterns (.git/, node_modules/) - - [ ] Scenario: Skill enforces sandbox boundaries - - [ ] **C3.10** [Rui] Write Behave scenarios in `features/mcp_integration.feature`: - - [ ] Scenario: Connect to MCP server and list tools - - [ ] Scenario: MCP tool becomes available as skill - - [ ] Scenario: MCP tool call is intercepted for sandbox paths - - [ ] Scenario: MCP tool writes are recorded in ChangeSet +**Parallel Group C2: Actor Loading & Compilation [Aditya + Jeff]** (depends on C1) +- [ ] **COMMIT (Owner: Aditya | Group: C2.loader) - Commit message: "feat(actor): add actor registry and loader"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Aditya]: Implement actor loader/registry with namespaced lookup and cache invalidation. + - [ ] Code [Aditya]: Add registry integration with Tool Registry so tool nodes resolve at load time. + - [ ] Docs [Aditya]: Add `docs/reference/actors_loading.md` with discovery rules and namespaces. + - [ ] Tests (Behave) [Rui]: Add `features/actor_loading.feature` for discovery, duplicates, and namespace lookup. + - [ ] Tests (Robot) [Rui]: Add `robot/actor_loading.robot` for loader smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/actor_loading_bench.py` for registry load performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "feat(actor): add actor registry and loader"`. +- [ ] **COMMIT (Owner: Jeff | Group: C2.compiler) - Commit message: "feat(actor): compile actor configs to LangGraph"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement ActorCompiler that builds LangGraph for LLM, TOOL, and GRAPH actors with tool node wiring. + - [ ] Code [Jeff]: Resolve tool node references through Tool Registry and validate required bindings before compile. + - [ ] Docs [Jeff]: Add `docs/reference/actors_compilation.md` covering compile outputs and error modes. + - [ ] Tests (Behave) [Rui]: Add `features/actor_compilation.feature` for LLM/GRAPH compilation and tool node wiring. + - [ ] Tests (Robot) [Rui]: Add `robot/actor_compilation.robot` smoke test compiling all examples. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/actor_compilation_bench.py` for compilation speed. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(actor): compile actor configs to LangGraph"`. +- [ ] **COMMIT (Owner: Jeff | Group: C2.refs) - Commit message: "feat(actor): resolve actor references and subgraphs"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement reference resolution, cycle detection, and subgraph wiring for actor refs. + - [ ] Code [Jeff]: Ensure cross-namespace reference resolution follows `[server:]namespace/name` rules. + - [ ] Docs [Jeff]: Update `docs/reference/actors_compilation.md` with reference semantics. + - [ ] Tests (Behave) [Rui]: Add `features/actor_reference_resolution.feature` for missing/recursive refs. + - [ ] Tests (Robot) [Rui]: Add `robot/actor_reference_resolution.robot` for subgraph wiring. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/actor_reference_bench.py` for reference resolution performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(actor): resolve actor references and subgraphs"`. -- [ ] **Stage C4: Tool-Based Change Tracking** (Day 8-10) **[Luis - Architectural]** - - [ ] Code: Implement tool-based change tracking (NOT output parsing) - - [ ] **CRITICAL ARCHITECTURE**: ChangeSet is built from skill/tool invocations, NOT by parsing LLM text output - - [ ] **C4.1** [Luis] Update `src/cleveragents/domain/models/core/change.py`: - - [ ] Enhance `Change` model with: - - [ ] Field `operation: OperationType` - create/modify/delete/move - - [ ] Field `path: str` - target resource path - - [ ] Field `new_path: str | None` - for move operations - - [ ] Field `content: str | None` - full content (for create) - - [ ] Field `edits: list[Edit] | None` - targeted edits (for modify) - - [ ] Field `patch: str | None` - unified diff (generated, not parsed) - - [ ] Field `language: str | None` - detected language - - [ ] Field `skill_invocation_id: str` - which skill call produced this - - [ ] Field `timestamp: datetime` - when change was made - - [ ] Field `validation_result: ValidationResult | None` - - [ ] Define `Edit` model for targeted edits: - - [ ] Field `type: EditType` - search_replace, line_range, insert_after, etc. - - [ ] Field `search: str | None` - text to find (for search_replace) - - [ ] Field `replace: str | None` - replacement text - - [ ] Field `start_line: int | None` - for line-based edits - - [ ] Field `end_line: int | None` - for line-based edits - - [ ] Field `content: str | None` - content to insert - - [ ] Enhance `ChangeSet` model with: - - [ ] Field `changes: list[Change]` - all resource changes - - [ ] Field `warnings: list[str]` - non-blocking issues - - [ ] Field `skill_invocations: list[SkillInvocation]` - full invocation history - - [ ] Field `generated_by_actor: str` - actor that produced this - - [ ] Field `validation: ChangeSetValidation` - overall validation - - [ ] Method `add_change(change: Change)` - append change from skill - - [ ] Method `get_change(path: str) -> Change | None` - find by path - - [ ] Method `get_changes_by_skill(skill_id: str) -> list[Change]` - changes from specific skill - - [ ] Method `file_paths() -> list[str]` - all affected paths - - [ ] Method `to_diff() -> str` - unified diff of all changes - - [ ] Method `rollback_to(change_id: str)` - remove changes after point - - [ ] **C4.2** [Luis] Implement `SkillInvocationTracker` in `src/cleveragents/actor/skills/tracker.py`: - - [ ] **C4.2a** `SkillInvocation` model: - - [ ] Field `id: str` - unique invocation ID - - [ ] Field `skill_name: str` - which skill was called - - [ ] Field `parameters: dict` - input parameters - - [ ] Field `result: Any` - skill return value - - [ ] Field `changes: list[Change]` - resource changes produced - - [ ] Field `timestamp: datetime` - when invoked - - [ ] Field `duration_ms: int` - execution time - - [ ] Field `error: str | None` - if skill failed - - [ ] **C4.2b** `SkillInvocationTracker` class: - - [ ] Method `start_invocation(skill: Skill, params: dict) -> str` - begin tracking - - [ ] Method `record_change(invocation_id: str, change: Change)` - record change - - [ ] Method `complete_invocation(invocation_id: str, result: Any)` - finish tracking - - [ ] Method `fail_invocation(invocation_id: str, error: Exception)` - record failure - - [ ] Method `get_invocations() -> list[SkillInvocation]` - full history - - [ ] Method `build_changeset() -> ChangeSet` - assemble from invocations - - [ ] **C4.3** [Luis] Implement `ToolCallRouter` in `src/cleveragents/actor/skills/router.py`: - - [ ] **C4.3a** Parse LLM tool calls (NOT text output): - - [ ] Handle OpenAI-style tool_calls from response - - [ ] Handle Anthropic-style tool_use blocks - - [ ] Handle LangChain AgentAction format - - [ ] **C4.3b** Route tool calls to skills: - - [ ] Look up skill by name in registry - - [ ] Validate parameters against skill schema - - [ ] Check capability metadata (read_only, writes, etc.) - - [ ] Enforce permission restrictions - - [ ] **C4.3c** Execute with tracking: - - [ ] Start invocation tracking - - [ ] Execute skill in sandbox context - - [ ] Record changes produced by skill - - [ ] Complete invocation tracking - - [ ] Return result to LLM - - [ ] **C4.4** [Luis] Implement resource path validation in `src/cleveragents/actor/skills/path_validator.py`: - - [ ] Validate paths against sandbox boundaries - - [ ] Enforce deny-list patterns (.git/, node_modules/, __pycache__/, etc.) - - [ ] Auto-create parent directories for new file paths - - [ ] Resolve relative paths to absolute sandbox paths - - [ ] Detect path traversal attempts (../) - - [ ] **C4.5** [Luis] Create diff generation in `src/cleveragents/agents/diff_generator.py`: - - [ ] Method `generate_unified_diff(change: Change, sandbox: Sandbox) -> str`: - - [ ] Compare sandbox state with original - - [ ] Generate unified diff format - - [ ] Include file headers with paths - - [ ] Method `generate_changeset_diff(changeset: ChangeSet, sandbox: Sandbox) -> str`: - - [ ] Combine all change diffs - - [ ] Add summary header (files created, modified, deleted) - - [ ] Include statistics (lines added/removed) - - [ ] Method `generate_edit_preview(edit: Edit, original: str) -> str`: - - [ ] Show what an edit will change - - [ ] Highlight search/replace matches - - [ ] Tests: Tool-based change tracking tests - - [ ] **C4.6** [Rui] Write Behave scenarios in `features/change_tracking.feature`: - - [ ] Scenario: Skill invocation creates Change record - - [ ] Scenario: Multiple skill calls accumulate in ChangeSet - - [ ] Scenario: ChangeSet correctly tracks skill invocation history - - [ ] Scenario: Failed skill invocation is recorded with error - - [ ] Scenario: Generate unified diff from ChangeSet - - [ ] Scenario: Rollback to specific change point - - [ ] **C4.7** [Rui] Write Behave scenarios in `features/tool_call_routing.feature`: - - [ ] Scenario: Route OpenAI-style tool call to skill - - [ ] Scenario: Route Anthropic-style tool_use to skill - - [ ] Scenario: Validate parameters against skill schema - - [ ] Scenario: Reject tool call for read_only skill trying to write - - [ ] Scenario: Path validation rejects traversal attempt - - [ ] Scenario: Path validation auto-creates directories +**Parallel Group C3: Skill Protocol & Context [Jeff]** (critical path; depends on C1) +- [ ] **COMMIT (Owner: Jeff | Group: C3.protocol) - Commit message: "feat(skill): add skill protocol and metadata"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Define Skill protocol interface, SkillMetadata, SkillResult, and SkillError types. + - [ ] Code [Jeff]: Add `SkillDefinition` model that references Tool Registry names and optional inline tool definitions. + - [ ] Docs [Jeff]: Add `docs/reference/skills_protocol.md` describing metadata, tool composition, and JSON schema rules. + - [ ] Tests (Behave) [Rui]: Add `features/skill_protocol.feature` for metadata validation and error capture. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_protocol.robot` smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/skill_protocol_bench.py` for validation throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(skill): add skill protocol and metadata"`. +- [ ] **COMMIT (Owner: Jeff | Group: C3.context) - Commit message: "feat(skill): add skill context and registry"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement SkillContext (plan/resource access, sandbox path, change tracker) and SkillRegistry. + - [ ] Code [Jeff]: Wire SkillRegistry to Tool Registry for tool resolution and validation node inclusion. + - [ ] Docs [Jeff]: Add `docs/reference/skills_context.md` with context fields and helper methods. + - [ ] Tests (Behave) [Rui]: Add `features/skill_context.feature` for sandboxed access and registry resolution. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_context.robot` for registry smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/skill_context_bench.py` for registry resolution overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(skill): add skill context and registry"`. +- [ ] **COMMIT (Owner: Jeff | Group: C3.inline) - Commit message: "feat(skill): add inline tool executor"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement inline tool execution with timeouts and restricted environment. + - [ ] Code [Jeff]: Ensure inline tools conform to Tool Registry schema and return structured results. + - [ ] Docs [Jeff]: Add `docs/reference/skills_inline.md` with safety constraints. + - [ ] Tests (Behave) [Rui]: Add `features/skill_inline.feature` for execution and timeout handling. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_inline.robot` for inline tool smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/inline_tool_bench.py` for execution overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(skill): add inline tool executor"`. -- [ ] **Stage C5: Validation Pipeline** (Day 9-11) **[Luis]** - - [ ] Code: Implement real validation gates - - [ ] **C5.1** [Luis] Create `ValidationPipeline` in `src/cleveragents/application/services/validation_service.py`: - - [ ] Method `validate_changeset(changeset: ChangeSet, project: Project) -> ValidationResult`: - - [ ] Run all applicable validators - - [ ] Aggregate results - - [ ] Return pass/fail with details - - [ ] Method `validate_syntax(change: Change) -> ValidationResult`: - - [ ] Detect language from extension - - [ ] Run language-specific syntax check - - [ ] Method `validate_lint(change: Change, config: ValidationConfig) -> ValidationResult`: - - [ ] Run lint command from project config - - [ ] Parse lint output for errors - - [ ] Method `validate_tests(project: Project, config: ValidationConfig) -> ValidationResult`: - - [ ] Run test command from project config - - [ ] Parse test results - - [ ] Method `validate_build(project: Project, config: ValidationConfig) -> ValidationResult`: - - [ ] Run build command from project config - - [ ] Check for build errors - - [ ] **C5.2** [Luis] Implement language-specific validators: - - [ ] Python: `python -m py_compile ` - - [ ] JavaScript/TypeScript: `node --check ` or syntax parse - - [ ] JSON: `json.loads()` validation - - [ ] YAML: `yaml.safe_load()` validation - - [ ] **C5.3** [Luis] Implement validation failure handling: - - [ ] If validation fails, attempt repair loop: - - [ ] Send errors to LLM with request to fix - - [ ] Parse fixed output - - [ ] Re-validate - - [ ] Max 3 repair attempts - - [ ] If repair fails, mark changeset as errored - - [ ] Preserve original output for debugging - - [ ] **C5.4** [Luis] Remove stub validation from existing code: - - [ ] Replace "output length > 10" check with real validation - - [ ] Remove "PASS" stub validation - - [ ] Tests: Validation tests - - [ ] **C5.5** [Rui] Write Behave scenarios in `features/validation_pipeline.feature`: - - [ ] Scenario: Valid Python file passes syntax validation - - [ ] Scenario: Invalid Python file fails with clear error - - [ ] Scenario: Lint errors detected and reported - - [ ] Scenario: Test failures detected and reported - - [ ] Scenario: Validation repair loop fixes simple errors - - [ ] Scenario: Validation repair gives up after max attempts +**Parallel Group C4: Built-in Skills [Jeff + Luis]** (depends on C3) +- [ ] **COMMIT (Owner: Jeff | Group: C4.file) - Commit message: "feat(skill): add file operation skills"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement ReadFile, WriteFile, EditFile, and DeleteFile tools with read_only enforcement. + - [ ] Code [Jeff]: Register tools in Tool Registry with resource bindings for fs/git resources. + - [ ] Docs [Jeff]: Add `docs/reference/skills_file.md` with examples and error cases. + - [ ] Tests (Behave) [Rui]: Add `features/skill_file_ops.feature` for read/write/edit/delete flows. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_file_ops.robot` for file ops integration. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/file_tool_bench.py` for read/write throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(skill): add file operation skills"`. +- [ ] **COMMIT (Owner: Jeff | Group: C4.search) - Commit message: "feat(skill): add directory and search skills"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement ListDir, Glob, and Grep tools with ignore patterns and size limits. + - [ ] Code [Jeff]: Register tools in Tool Registry with resource bindings and sandbox awareness. + - [ ] Docs [Jeff]: Add `docs/reference/skills_search.md` with examples. + - [ ] Tests (Behave) [Rui]: Add `features/skill_search.feature` for listing/globbing/searching. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_search.robot` for search integration. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/search_tool_bench.py` for search performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(skill): add directory and search skills"`. +- [ ] **COMMIT (Owner: Luis | Group: C4.git) - Commit message: "feat(skill): add git operation skills"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement read-only git tools (status, diff, log, show) for sandboxed repos. + - [ ] Code [Luis]: Register git tools in Tool Registry with read-only capability metadata. + - [ ] Docs [Luis]: Add `docs/reference/skills_git.md` clarifying no destructive ops in MVP. + - [ ] Tests (Behave) [Rui]: Add `features/skill_git.feature` for git tool outputs. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_git.robot` for git tool integration. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/git_tool_bench.py` for diff/log performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(skill): add git operation skills"`. -- [ ] **Stage C6: Built-in Provider Actors** (Day 10-11) **[Aditya]** - - [ ] Code: Create built-in actors for each provider - - [ ] **C6.1** [Aditya] Generate built-in actor configs in `src/cleveragents/actor/builtins.py`: - - [ ] `openai/gpt-4` - GPT-4 wrapper - - [ ] `openai/gpt-4-turbo` - GPT-4 Turbo wrapper - - [ ] `openai/gpt-3.5-turbo` - GPT-3.5 wrapper - - [ ] `anthropic/claude-3-opus` - Claude 3 Opus wrapper - - [ ] `anthropic/claude-3-sonnet` - Claude 3 Sonnet wrapper - - [ ] `anthropic/claude-3-haiku` - Claude 3 Haiku wrapper - - [ ] `google/gemini-pro` - Gemini Pro wrapper - - [ ] `google/gemini-ultra` - Gemini Ultra wrapper - - [ ] **C6.2** [Aditya] Ensure built-in actors work as strategy/execution actors: - - [ ] Add appropriate system prompts for each role - - [ ] Configure temperature defaults (lower for execution) - - [ ] Test with plan lifecycle - - [ ] **C6.3** [Aditya] Implement provider capability detection: - - [ ] Check which API keys are configured - - [ ] Only register actors for available providers - - [ ] Clear error message for unavailable providers - - [ ] Tests: Verify built-in actors work in plan lifecycle - - [ ] **C6.4** [Rui] Write Behave scenarios in `features/builtin_actors.feature`: - - [ ] Scenario: Built-in OpenAI actor loads correctly - - [ ] Scenario: Built-in Anthropic actor loads correctly - - [ ] Scenario: Built-in actor can be used as strategy actor - - [ ] Scenario: Missing API key produces clear error +**Parallel Group C5: Tool Routing & Change Tracking [Luis + Jeff]** (depends on C3/C4) +- [ ] **COMMIT (Owner: Luis | Group: C5.model) - Commit message: "feat(change): add ChangeSet models and invocation tracker"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add Change/ChangeSet/ToolInvocation models and SkillInvocationTracker. + - [ ] Code [Luis]: Ensure ChangeSet stores resource references, sandbox paths, and tool metadata. + - [ ] Docs [Luis]: Add `docs/reference/change_tracking.md` describing tool-to-change mapping. + - [ ] Tests (Behave) [Rui]: Add `features/change_tracking.feature` for ChangeSet aggregation. + - [ ] Tests (Robot) [Rui]: Add `robot/change_tracking.robot` for tracker smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/change_tracking_bench.py` for invocation tracking overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(change): add ChangeSet models and invocation tracker"`. +- [ ] **COMMIT (Owner: Jeff | Group: C5.router) - Commit message: "feat(change): add tool router for providers"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement ToolCallRouter for OpenAI/Anthropic/LangChain tool schemas with deterministic IDs. + - [ ] Code [Jeff]: Add mapping for tool/validation names and argument schemas based on Tool Registry metadata. + - [ ] Docs [Jeff]: Add `docs/reference/tool_router.md` with provider-specific mappings. + - [ ] Tests (Behave) [Rui]: Add `features/tool_router.feature` for schema mapping. + - [ ] Tests (Robot) [Rui]: Add `robot/tool_router.robot` for routing smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/tool_router_bench.py` for routing performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(change): add tool router for providers"`. +- [ ] **COMMIT (Owner: Luis | Group: C5.diff) - Commit message: "feat(change): add diff review artifacts"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement DiffBuilder and ReviewArtifact models for CLI review. + - [ ] Code [Luis]: Add support for multi-resource diffs and per-resource grouping. + - [ ] Docs [Luis]: Add `docs/reference/diff_review.md` with output format. + - [ ] Tests (Behave) [Rui]: Add `features/diff_review.feature` for diff generation. + - [ ] Tests (Robot) [Rui]: Add `robot/diff_review.robot` for review artifacts. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/diff_review_bench.py` for diff building performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(change): add diff review artifacts"`. -- [ ] **Stage C7: Plan-Actor Integration** (Day 11-14) **[Aditya + Luis]** - - [ ] Code: Connect actors to plan lifecycle - - [ ] **C7.1** [Aditya] Update `PlanLifecycleService.execute_strategize()`: - - [ ] Load strategy_actor from action - - [ ] Compile actor to LangGraph - - [ ] Build strategy context: - - [ ] Project resources - - [ ] Plan description and arguments - - [ ] Definition of done - - [ ] Invoke actor graph with context - - [ ] Parse strategy output - - [ ] Store strategy in plan - - [ ] **C7.2** [Aditya] Implement dependency closure computation: - - [ ] Method `compute_closure(target: str, project: Project) -> ResourceClosure`: - - [ ] Find direct imports/includes - - [ ] Find symbol dependencies - - [ ] Find test dependencies - - [ ] Find build references - - [ ] Integrate with strategy actor context - - [ ] **C7.3** [Luis] Update `PlanLifecycleService.execute_execution()`: - - [ ] Load execution_actor from action - - [ ] Compile actor to LangGraph - - [ ] Build execution context: - - [ ] Strategy output - - [ ] Resource service for sandbox access - - [ ] Bounded dependency closure - - [ ] Invoke actor graph with context - - [ ] Parse output as ChangeSet - - [ ] Run validation pipeline - - [ ] Handle subplan spawning - - [ ] **C7.4** [Luis] Update `PlanLifecycleService.apply_plan()`: - - [ ] Verify execution completed successfully - - [ ] Commit all sandboxes - - [ ] Apply ChangeSet to resources - - [ ] Record applied artifacts - - [ ] Clean up sandboxes - - [ ] **C7.4a** [Luis] Implement ATOMIC apply: - - [ ] Apply must be all-or-nothing: - - [ ] Write all changes to temp files first - - [ ] Validate all writes succeeded - - [ ] Atomic rename/swap to final locations - - [ ] If any step fails, rollback all changes - - [ ] In git mode: - - [ ] All changes in single commit - - [ ] If commit fails, no partial changes applied - - [ ] Handle partial failure: - - [ ] Preserve sandbox for inspection - - [ ] Clear error message about what failed - - [ ] Allow retry after manual fix - - [ ] **C7.5** [Aditya] Implement hierarchical task decomposition: - - [ ] Strategy actor can emit subplan decisions - - [ ] Each subplan gets bounded context - - [ ] Parallel or sequential execution modes - - [ ] **C7.6** [Luis] Connect output parser to execution flow: - - [ ] After actor produces output, parse to ChangeSet - - [ ] Validate ChangeSet - - [ ] Store ChangeSet in plan - - [ ] **C7.7** [Luis] Implement diff review artifact storage: - - [ ] Store generated diff in plan metadata - - [ ] Create `DiffArtifact` model: - - [ ] Field `diff_id: str` - ULID - - [ ] Field `plan_id: str` - parent plan - - [ ] Field `unified_diff: str` - full unified diff - - [ ] Field `file_summaries: list[FileSummary]` - per-file summary - - [ ] Field `risk_markers: list[str]` - touched auth code, migrations, etc. - - [ ] Field `created_at: datetime` - - [ ] Display diff in `agents [--data-dir PATH] [--config-path PATH] plan diff` command - - [ ] Display diff before apply in review-before-apply mode - - [ ] Tests: End-to-end tests for full plan lifecycle with actors - - [ ] **C7.7** [Rui] Write Behave scenarios in `features/plan_actor_integration.feature`: - - [ ] Scenario: Full lifecycle with LLM actors (mocked) - - [ ] Scenario: Strategy actor receives correct context - - [ ] Scenario: Execution actor receives strategy output - - [ ] Scenario: ChangeSet applied correctly - - [ ] Scenario: Validation failure triggers repair loop - - [ ] **C7.8** [Rui] Write Robot integration test `robot/plan_actor_integration.robot`: - - [ ] Test: Full lifecycle with real sandbox - - [ ] Test: Multi-file generation and application - - [ ] Tests: Dependency closure computation accuracy - - [ ] **C7.9** [Rui] Write Behave scenarios in `features/dependency_closure.feature`: - - [ ] Scenario: Python imports detected correctly - - [ ] Scenario: Test file dependencies included +**Parallel Group C6: Validation Pipeline [Luis + Jeff]** (depends on C5 and project validation config) +- [ ] **COMMIT (Owner: Luis | Group: C6.pipeline) - Commit message: "feat(validation): add validation pipeline and results model"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement ValidationCommand, ValidationResult, and ValidationPipeline using Validation attachments from Tool Registry. + - [ ] Code [Luis]: Run validations at end of Execute phase only; do not re-run during Apply per spec. + - [ ] Code [Luis]: Enforce required vs informational validation modes and fix-then-revalidate loop hooks. + - [ ] Docs [Luis]: Add `docs/reference/validation_pipeline.md` with ordering, timeouts, and failure handling. + - [ ] Tests (Behave) [Rui]: Add `features/validation_pipeline.feature` for pass/fail paths and required/informational modes. + - [ ] Tests (Robot) [Rui]: Add `robot/validation_pipeline.robot` for pipeline smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/validation_pipeline_bench.py` for pipeline runtime. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(validation): add validation pipeline and results model"`. +- [ ] **COMMIT (Owner: Jeff | Group: C6.gating) - Commit message: "feat(validation): integrate validation with apply gating"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Block apply on required validation failure; surface validation artifacts for review. + - [ ] Code [Jeff]: Ensure informational validation failures do not block apply but are logged in plan status. + - [ ] Docs [Jeff]: Update `docs/reference/plan_actor_integration.md` with validation gating behavior. + - [ ] Tests (Behave) [Rui]: Add `features/validation_gating.feature` for apply blocking. + - [ ] Tests (Robot) [Rui]: Add `robot/validation_gating.robot` for end-to-end gating. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/validation_gating_bench.py` for gating overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(validation): integrate validation with apply gating"`. + +**Parallel Group C7: MCP Adapter [Aditya]** (depends on C3) +- [ ] **COMMIT (Owner: Aditya | Group: C7.mcp) - Commit message: "feat(skill): add MCP adapter for external tools"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Aditya]: Implement MCP client adapter conforming to Tool interface with connection config. + - [ ] Code [Aditya]: Register MCP tools in Tool Registry with dynamic discovery from MCP server. + - [ ] Docs [Aditya]: Add `docs/reference/skills_mcp.md` with server connection examples. + - [ ] Tests (Behave) [Rui]: Add `features/skill_mcp.feature` for MCP tool calls. + - [ ] Tests (Robot) [Rui]: Add `robot/skill_mcp.robot` for MCP adapter smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/mcp_adapter_bench.py` for tool invocation latency. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "feat(skill): add MCP adapter for external tools"`. + +**Parallel Group C8: Built-in Provider Actors [Aditya]** (depends on C1/C2) +- [ ] **COMMIT (Owner: Aditya | Group: C8.providers) - Commit message: "feat(actor): add built-in provider actors"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Aditya]: Add built-in actor configs for `openai/`, `anthropic/`, and `openrouter/` (plus `google/` if configured). + - [ ] Code [Aditya]: Add built-in actors for invariant reconciliation and estimation roles (using provider defaults). + - [ ] Docs [Aditya]: Add `docs/reference/provider_actors.md` with provider defaults. + - [ ] Tests (Behave) [Rui]: Add `features/provider_actors.feature` for built-in actor loading. + - [ ] Tests (Robot) [Rui]: Add `robot/provider_actors.robot` for registry visibility. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/provider_actor_load_bench.py` for registry load cost. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "feat(actor): add built-in provider actors"`. + +**Parallel Group C9: Plan-Actor Integration [Jeff + Luis]** (depends on C2/C5/C6) +- [ ] **COMMIT (Owner: Jeff | Group: C9.execute) - Commit message: "feat(plan): execute strategize and execute phases via actors"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Connect PlanLifecycleService to actor execution for Strategize and Execute phases. + - [ ] Code [Jeff]: Ensure Strategize is read-only and records decisions without modifying resources. + - [ ] Code [Jeff]: Ensure Execute uses sandbox resources and tool calls routed through Tool Router + ChangeSet. + - [ ] Docs [Jeff]: Add `docs/reference/plan_actor_integration.md` with phase flow. + - [ ] Tests (Behave) [Rui]: Add `features/plan_actor_integration.feature` for strategy/execute flows. + - [ ] Tests (Robot) [Rui]: Add `robot/plan_actor_integration.robot` for end-to-end actor execution. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_actor_integration_bench.py` for execution overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(plan): execute strategize and execute phases via actors"`. +- [ ] **COMMIT (Owner: Jeff | Group: C9.apply) - Commit message: "feat(plan): integrate change review and apply flow"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Wire ChangeSet review artifacts into `plan diff` and review-before-apply flow. + - [ ] Code [Jeff]: Ensure Apply merges sandbox into real resources only after required validations pass. + - [ ] Docs [Jeff]: Update CLI docs for `plan diff` and `plan apply` review output. + - [ ] Tests (Behave) [Rui]: Add `features/plan_review_apply.feature` for review gate behavior. + - [ ] Tests (Robot) [Rui]: Add `robot/plan_review_apply.robot` for review-before-apply path. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/plan_apply_bench.py` for apply throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(plan): integrate change review and apply flow"`. **M3 SUCCESS CRITERIA**: - [ ] Can define actors in YAML with skills @@ -4833,1762 +3583,127 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation **Target: Milestone M4 (+21 days)** -- [ ] **Stage D1: Decision Data Model** (Day 15-16) **[Hamza - Well Rounded]** - - **SEQUENTIAL ORDER**: D1.1 (Enums) → D1.2 (ContextSnapshot) → D1.3 (Decision model) → D1.4 (Helpers) → D1.5 (Tests) - - - [ ] Code: Create Decision domain model - - [ ] **D1.1** [Hamza] Define `DecisionType` enum in `src/cleveragents/domain/models/core/decision.py`: - - [ ] **D1.1a** [Hamza] Create file with DecisionType enum: - ```python - from enum import Enum - - class DecisionType(str, Enum): - """Classification of decision points in plan execution.""" - - # Root decisions - PROMPT_DEFINITION = "prompt_definition" # Initial plan prompt - - # Strategy phase decisions - STRATEGY_CHOICE = "strategy_choice" # High-level approach - IMPLEMENTATION_CHOICE = "implementation_choice" # How to implement - RESOURCE_SELECTION = "resource_selection" # Which resources to use - - # Execution phase decisions - SUBPLAN_SPAWN = "subplan_spawn" # Decision to create subplan - TOOL_INVOCATION = "tool_invocation" # Which tool/skill to use - - # Error handling decisions - ERROR_RECOVERY = "error_recovery" # How to handle failure - VALIDATION_RESPONSE = "validation_response" # Response to validation failure - - # User interaction decisions - USER_INTERVENTION = "user_intervention" # User provided guidance - ``` - - [ ] Commit: "feat(domain): define DecisionType enum" - - [ ] **D1.1b** [Hamza] Add helper method for decision classification: - ```python - @classmethod - def is_strategy_decision(cls, decision_type: "DecisionType") -> bool: - """Check if this is a strategy phase decision.""" - return decision_type in { - cls.PROMPT_DEFINITION, cls.STRATEGY_CHOICE, - cls.IMPLEMENTATION_CHOICE, cls.RESOURCE_SELECTION - } - - @classmethod - def is_execution_decision(cls, decision_type: "DecisionType") -> bool: - """Check if this is an execution phase decision.""" - return decision_type in {cls.SUBPLAN_SPAWN, cls.TOOL_INVOCATION} - ``` - - [ ] Commit: "feat(domain): add DecisionType helper methods" - - [ ] **D1.2** [Hamza] Define `ContextSnapshot` model: - - [ ] **D1.2a** [Hamza] Create ContextSnapshot dataclass: - ```python - @dataclass(frozen=True) - class ContextSnapshot: - """Snapshot of context at decision point for replay.""" - - snapshot_id: str # ULID - hot_context_hash: str # SHA-256 hash of hot context content - hot_context_ref: str # Storage reference (file path or blob ID) - relevant_resources: tuple[str, ...] # Resource IDs in scope - actor_state_ref: str | None # LangGraph checkpoint ID - file_versions: dict[str, str] # path -> git commit or hash - created_at: datetime - ``` - - [ ] Commit: "feat(domain): define ContextSnapshot dataclass" - - [ ] **D1.2b** [Hamza] Add factory method: - ```python - @classmethod - def capture( - cls, - hot_context: str, - resources: list[str], - actor_state: str | None = None, - file_versions: dict[str, str] | None = None - ) -> "ContextSnapshot": - """Capture a snapshot of current context.""" - import hashlib - import ulid - - return cls( - snapshot_id=ulid.new().str, - hot_context_hash=hashlib.sha256(hot_context.encode()).hexdigest(), - hot_context_ref="", # Set by storage layer - relevant_resources=tuple(resources), - actor_state_ref=actor_state, - file_versions=file_versions or {}, - created_at=datetime.utcnow() - ) - ``` - - [ ] Commit: "feat(domain): add ContextSnapshot.capture() factory" - - [ ] **D1.3** [Hamza] Define `Decision` Pydantic model: - - [ ] **D1.3a** [Hamza] Create Decision class with identity fields: - ```python - class Decision(BaseModel): - """A recorded decision point in plan execution.""" - - model_config = ConfigDict(frozen=True) - - # Identity - decision_id: str = Field(..., description="ULID identifier") - plan_id: str = Field(..., description="Parent plan ULID") - - # Tree structure - parent_decision_id: str | None = Field( - default=None, description="Parent in decision tree" - ) - sequence_number: int = Field( - ..., ge=0, description="Order within plan (0=root)" - ) - ``` - - [ ] Commit: "feat(domain): add Decision model identity fields" - - [ ] **D1.3b** [Hamza] Add decision content fields: - ```python - # Decision content - decision_type: DecisionType = Field(..., description="Classification") - question: str = Field(..., min_length=1, description="What was decided") - chosen_option: str = Field(..., min_length=1, description="The choice made") - alternatives_considered: list[str] = Field( - default_factory=list, description="Other options evaluated" - ) - confidence_score: float | None = Field( - default=None, ge=0.0, le=1.0, description="AI confidence 0.0-1.0" - ) - rationale: str = Field(default="", description="Why this choice") - actor_reasoning: str | None = Field( - default=None, description="Raw LLM chain-of-thought" - ) - ``` - - [ ] Commit: "feat(domain): add Decision content fields" - - [ ] **D1.3c** [Hamza] Add context and relationship fields: - ```python - # Context for replay - context_snapshot: ContextSnapshot = Field( - ..., description="Snapshot at decision time" - ) - checkpoint_id: str | None = Field( - default=None, description="Sandbox checkpoint for rollback" - ) - - # Downstream relationships (populated during execution) - downstream_decision_ids: list[str] = Field( - default_factory=list, description="Decisions that depend on this" - ) - downstream_plan_ids: list[str] = Field( - default_factory=list, description="Subplans spawned from this" - ) - artifacts_produced: list[str] = Field( - default_factory=list, description="Artifact IDs created" - ) - ``` - - [ ] Commit: "feat(domain): add Decision context and relationship fields" - - [ ] **D1.3d** [Hamza] Add correction tracking fields: - ```python - # Correction tracking - is_correction: bool = Field( - default=False, description="Is this a corrected decision" - ) - corrects_decision_id: str | None = Field( - default=None, description="Original decision this corrects" - ) - superseded_by: str | None = Field( - default=None, description="Decision that replaced this one" - ) - - # Timestamps - created_at: datetime = Field(default_factory=datetime.utcnow) - ``` - - [ ] Commit: "feat(domain): add Decision correction tracking fields" - - [ ] **D1.3e** [Hamza] Add validators: - ```python - @field_validator('decision_id', 'plan_id') - @classmethod - def validate_ulid(cls, v: str) -> str: - """Validate ULID format.""" - if len(v) != 26 or not v.isalnum(): - raise ValueError(f"Invalid ULID format: {v}") - return v - - @model_validator(mode='after') - def validate_correction_consistency(self) -> Self: - """Ensure correction fields are consistent.""" - if self.is_correction and not self.corrects_decision_id: - raise ValueError("Correction must specify corrects_decision_id") - if self.corrects_decision_id and not self.is_correction: - raise ValueError("corrects_decision_id requires is_correction=True") - return self - ``` - - [ ] Commit: "feat(domain): add Decision validators" - - [ ] **D1.4** [Hamza] Add Decision helper methods: - - [ ] **D1.4a** [Hamza] Add computed properties: - ```python - @property - def is_root(self) -> bool: - """Check if this is the root decision (no parent).""" - return self.parent_decision_id is None - - @property - def is_superseded(self) -> bool: - """Check if this decision has been replaced.""" - return self.superseded_by is not None - - @property - def has_downstream_work(self) -> bool: - """Check if this decision spawned work.""" - return bool(self.downstream_decision_ids or self.downstream_plan_ids) - - @property - def summary(self) -> str: - """Short summary for display.""" - q = self.question[:50] + "..." if len(self.question) > 50 else self.question - return f"[{self.decision_type.value}] {q}" - ``` - - [ ] Commit: "feat(domain): add Decision computed properties" - - [ ] **D1.4b** [Hamza] Add mutation methods (return new instance): - ```python - def with_downstream_decision(self, decision_id: str) -> "Decision": - """Return new Decision with added downstream decision.""" - return self.model_copy(update={ - "downstream_decision_ids": [*self.downstream_decision_ids, decision_id] - }) - - def with_downstream_plan(self, plan_id: str) -> "Decision": - """Return new Decision with added downstream plan.""" - return self.model_copy(update={ - "downstream_plan_ids": [*self.downstream_plan_ids, plan_id] - }) - - def with_artifact(self, artifact_id: str) -> "Decision": - """Return new Decision with added artifact.""" - return self.model_copy(update={ - "artifacts_produced": [*self.artifacts_produced, artifact_id] - }) - - def mark_superseded(self, by_decision_id: str) -> "Decision": - """Return new Decision marked as superseded.""" - return self.model_copy(update={"superseded_by": by_decision_id}) - ``` - - [ ] Commit: "feat(domain): add Decision mutation methods" - - [ ] Tests: Behave scenarios for decision model - - [ ] **D1.5** [Rui] Write Behave scenarios in `features/decision_model.feature`: - - [ ] **D1.5a** [Rui] Creation scenarios: - - [ ] Scenario: Create decision with all required fields - - [ ] Given valid decision_id, plan_id, question, chosen_option, context_snapshot - - [ ] When I create a Decision with these fields - - [ ] Then the Decision is created successfully - - [ ] And sequence_number defaults to provided value - - [ ] Scenario: Create root decision (no parent) - - [ ] When I create a Decision with parent_decision_id=None - - [ ] Then is_root property returns True - - [ ] Scenario: Create child decision - - [ ] When I create a Decision with parent_decision_id set - - [ ] Then is_root property returns False - - [ ] Commit: "test(behave): add decision creation scenarios" - - [ ] **D1.5b** [Rui] Validation scenarios: - - [ ] Scenario: Invalid ULID format rejected - - [ ] When I create a Decision with decision_id="invalid" - - [ ] Then validation error is raised - - [ ] Scenario: Confidence score must be 0.0-1.0 - - [ ] When I create a Decision with confidence_score=1.5 - - [ ] Then validation error is raised - - [ ] Scenario: Correction without corrects_decision_id fails - - [ ] When I create a Decision with is_correction=True and corrects_decision_id=None - - [ ] Then validation error mentions correction consistency - - [ ] Commit: "test(behave): add decision validation scenarios" - - [ ] **D1.5c** [Rui] DecisionType scenarios: - - [ ] Scenario: Each decision type validates correctly - - [ ] For each DecisionType enum value - - [ ] When I create a Decision with that type - - [ ] Then decision is created successfully - - [ ] Scenario: is_strategy_decision helper works - - [ ] Given DecisionType.STRATEGY_CHOICE - - [ ] Then DecisionType.is_strategy_decision() returns True - - [ ] Given DecisionType.TOOL_INVOCATION - - [ ] Then DecisionType.is_strategy_decision() returns False - - [ ] Commit: "test(behave): add DecisionType scenarios" - - [ ] **D1.5d** [Rui] Context snapshot scenarios: - - [ ] Scenario: ContextSnapshot.capture() creates valid snapshot - - [ ] Given hot_context string and resource list - - [ ] When I call ContextSnapshot.capture() - - [ ] Then snapshot has valid ULID - - [ ] And hot_context_hash is SHA-256 of content - - [ ] Scenario: ContextSnapshot is immutable - - [ ] Given a ContextSnapshot instance - - [ ] When I try to modify a field - - [ ] Then FrozenInstanceError is raised - - [ ] Commit: "test(behave): add ContextSnapshot scenarios" +**Parallel Group D1: Decision Domain [Hamza + Rui]** (foundation for D2-D5) +- [ ] **COMMIT (Owner: Hamza | Group: D1.domain) - Commit message: "feat(domain): add decision model and context snapshots"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Add `DecisionType`, `ContextSnapshot`, and `Decision` models with correction fields and helpers. + - [ ] Code [Hamza]: Include required fields: question, chosen option, alternatives, confidence score, rationale, dependencies, and context hash. + - [ ] Docs [Hamza]: Add `docs/reference/decision_model.md` with examples and schema notes. + - [ ] Tests (Behave) [Rui]: Add `features/decision_model.feature` for validation and helpers. + - [ ] Tests (Robot) [Rui]: Add `robot/decision_model.robot` smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_model_bench.py` for decision validation throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(domain): add decision model and context snapshots"`. -- [ ] **Stage D2: Decision Recording** (Day 16-18) **[Hamza]** - - **SEQUENTIAL ORDER**: D2.1 (Service scaffold) → D2.2 (record_decision) → D2.3 (tree queries) → D2.4 (context capture) → D2.5 (strategy integration) → D2.6 (downstream updates) → D2.7 (Tests) - - - [ ] Code: Record decisions during Strategize - - [ ] **D2.1** [Hamza] Create `DecisionService` scaffold in `src/cleveragents/application/services/decision_service.py`: - - [ ] **D2.1a** [Hamza] Define service class with dependencies: - ```python - class DecisionService: - """Service for recording and querying decisions.""" - - def __init__( - self, - decision_repo: DecisionRepository, - snapshot_store: ContextSnapshotStore, - plan_repo: LifecyclePlanRepository - ): - self._decision_repo = decision_repo - self._snapshot_store = snapshot_store - self._plan_repo = plan_repo - self._sequence_counters: dict[str, int] = {} # plan_id -> next sequence - ``` - - [ ] Commit: "feat(service): add DecisionService scaffold" - - [ ] **D2.1b** [Hamza] Define ContextSnapshotStore protocol: - ```python - class ContextSnapshotStore(Protocol): - """Protocol for storing context snapshots.""" - - def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: - """Store snapshot content and return with ref set.""" - ... - - def retrieve(self, snapshot_id: str) -> tuple[ContextSnapshot, str]: - """Retrieve snapshot and its content.""" - ... - - def retrieve_by_hash(self, hash: str) -> tuple[ContextSnapshot, str] | None: - """Retrieve by content hash (for deduplication).""" - ... - ``` - - [ ] Commit: "feat(service): define ContextSnapshotStore protocol" - - [ ] **D2.2** [Hamza] Implement `record_decision()` method: - - [ ] **D2.2a** [Hamza] Core implementation: - ```python - def record_decision( - self, - plan_id: str, - decision_type: DecisionType, - question: str, - chosen_option: str, - hot_context: str, - resources: list[str], - parent_decision_id: str | None = None, - alternatives: list[str] | None = None, - confidence: float | None = None, - rationale: str = "", - actor_reasoning: str | None = None, - checkpoint_id: str | None = None - ) -> Decision: - """Record a new decision for a plan.""" - import ulid - - # Generate IDs - decision_id = ulid.new().str - - # Get next sequence number for this plan - seq = self._get_next_sequence(plan_id) - - # Capture context snapshot - snapshot = self._capture_snapshot(hot_context, resources, checkpoint_id) - - # Create decision - decision = Decision( - decision_id=decision_id, - plan_id=plan_id, - parent_decision_id=parent_decision_id, - sequence_number=seq, - decision_type=decision_type, - question=question, - chosen_option=chosen_option, - alternatives_considered=alternatives or [], - confidence_score=confidence, - rationale=rationale, - actor_reasoning=actor_reasoning, - context_snapshot=snapshot, - checkpoint_id=checkpoint_id - ) - - # Persist - self._decision_repo.create(decision) - - # Update parent's downstream if applicable - if parent_decision_id: - self._add_downstream_decision(parent_decision_id, decision_id) - - logger.info(f"Recorded decision {decision_id}: {decision.summary}") - return decision - ``` - - [ ] Commit: "feat(service): implement record_decision()" - - [ ] **D2.2b** [Hamza] Add sequence number management: - ```python - def _get_next_sequence(self, plan_id: str) -> int: - """Get next sequence number for a plan.""" - if plan_id not in self._sequence_counters: - # Load max sequence from existing decisions - existing = self._decision_repo.get_max_sequence(plan_id) - self._sequence_counters[plan_id] = (existing or -1) + 1 - - seq = self._sequence_counters[plan_id] - self._sequence_counters[plan_id] += 1 - return seq - ``` - - [ ] Commit: "feat(service): add sequence number management" - - [ ] **D2.3** [Hamza] Implement tree query methods: - - [ ] **D2.3a** [Hamza] Implement `get_decision_tree()`: - ```python - def get_decision_tree(self, plan_id: str) -> list[Decision]: - """Get all decisions for a plan in tree order.""" - decisions = self._decision_repo.get_by_plan(plan_id) - - # Sort by sequence number to get chronological order - return sorted(decisions, key=lambda d: d.sequence_number) - - def get_decision_tree_nested(self, plan_id: str) -> DecisionTree: - """Get decisions as nested tree structure.""" - decisions = self.get_decision_tree(plan_id) - return self._build_tree(decisions) - - def _build_tree(self, decisions: list[Decision]) -> DecisionTree: - """Build tree from flat list of decisions.""" - by_id = {d.decision_id: d for d in decisions} - roots = [] - - for d in decisions: - if d.parent_decision_id is None: - roots.append(DecisionNode(decision=d, children=[])) - else: - # Find parent and add as child - # Implementation details... - - return DecisionTree(roots=roots, total_count=len(decisions)) - ``` - - [ ] Commit: "feat(service): implement decision tree queries" - - [ ] **D2.3b** [Hamza] Implement `get_decision()` and `get_children()`: - ```python - def get_decision(self, decision_id: str) -> Decision | None: - """Get a single decision by ID.""" - return self._decision_repo.get_by_id(decision_id) - - def get_children(self, decision_id: str) -> list[Decision]: - """Get all direct children of a decision.""" - return self._decision_repo.get_children(decision_id) - - def get_ancestors(self, decision_id: str) -> list[Decision]: - """Get all ancestors from decision to root.""" - ancestors = [] - current = self.get_decision(decision_id) - - while current and current.parent_decision_id: - parent = self.get_decision(current.parent_decision_id) - if parent: - ancestors.append(parent) - current = parent - - return ancestors - ``` - - [ ] Commit: "feat(service): implement get_decision and get_children" - - [ ] **D2.4** [Hamza] Implement context snapshot capture: - - [ ] **D2.4a** [Hamza] Implement `_capture_snapshot()`: - ```python - def _capture_snapshot( - self, - hot_context: str, - resources: list[str], - checkpoint_id: str | None = None - ) -> ContextSnapshot: - """Capture and store a context snapshot.""" - # Create snapshot object - snapshot = ContextSnapshot.capture( - hot_context=hot_context, - resources=resources, - actor_state=checkpoint_id - ) - - # Check for duplicate by hash (deduplication) - existing = self._snapshot_store.retrieve_by_hash(snapshot.hot_context_hash) - if existing: - logger.debug(f"Reusing existing snapshot with hash {snapshot.hot_context_hash[:8]}") - return existing[0] - - # Store new snapshot - stored = self._snapshot_store.store(snapshot, hot_context) - return stored - ``` - - [ ] Commit: "feat(service): implement context snapshot capture" - - [ ] **D2.4b** [Hamza] Implement FileContextSnapshotStore: - ```python - class FileContextSnapshotStore: - """Store snapshots in filesystem.""" - - def __init__(self, base_dir: Path): - self._base_dir = base_dir - self._base_dir.mkdir(parents=True, exist_ok=True) - - def store(self, snapshot: ContextSnapshot, content: str) -> ContextSnapshot: - """Store snapshot content to file.""" - file_path = self._base_dir / f"{snapshot.snapshot_id}.json" - - data = { - "snapshot": snapshot.__dict__, - "content": content - } - file_path.write_text(json.dumps(data)) - - # Update snapshot with ref - return dataclasses.replace( - snapshot, - hot_context_ref=str(file_path) - ) - ``` - - [ ] Commit: "feat(service): implement FileContextSnapshotStore" - - [ ] **D2.5** [Hamza] Integrate decision recording into strategy actor: - - [ ] **D2.5a** [Hamza] Create DecisionRecordingCallback: - ```python - class DecisionRecordingCallback: - """LangGraph callback to record decisions during execution.""" - - def __init__(self, decision_service: DecisionService, plan_id: str): - self._service = decision_service - self._plan_id = plan_id - self._current_parent: str | None = None - - def on_strategy_decision( - self, - question: str, - chosen: str, - alternatives: list[str], - confidence: float | None, - rationale: str, - context: str - ) -> Decision: - """Called when strategy actor makes a decision.""" - decision = self._service.record_decision( - plan_id=self._plan_id, - decision_type=DecisionType.STRATEGY_CHOICE, - question=question, - chosen_option=chosen, - hot_context=context, - resources=[], # Populated from plan - parent_decision_id=self._current_parent, - alternatives=alternatives, - confidence=confidence, - rationale=rationale - ) - return decision - ``` - - [ ] Commit: "feat(service): add DecisionRecordingCallback" - - [ ] **D2.5b** [Hamza] Record root PROMPT_DEFINITION decision: - ```python - def record_prompt_definition( - self, - plan_id: str, - prompt: str, - context: str - ) -> Decision: - """Record the initial prompt as root decision.""" - return self.record_decision( - plan_id=plan_id, - decision_type=DecisionType.PROMPT_DEFINITION, - question="What should be done?", - chosen_option=prompt, - hot_context=context, - resources=[], - parent_decision_id=None, - rationale="User provided prompt" - ) - ``` - - [ ] Commit: "feat(service): add record_prompt_definition()" - - [ ] **D2.6** [Hamza] Implement downstream relationship updates: - - [ ] **D2.6a** [Hamza] Add methods to update downstream fields: - ```python - def _add_downstream_decision(self, parent_id: str, child_id: str) -> None: - """Add child to parent's downstream_decision_ids.""" - parent = self._decision_repo.get_by_id(parent_id) - if parent: - updated = parent.with_downstream_decision(child_id) - self._decision_repo.update(updated) - - def add_downstream_plan(self, decision_id: str, plan_id: str) -> None: - """Record that a decision spawned a subplan.""" - decision = self._decision_repo.get_by_id(decision_id) - if decision: - updated = decision.with_downstream_plan(plan_id) - self._decision_repo.update(updated) - logger.info(f"Linked subplan {plan_id} to decision {decision_id}") - - def add_artifact(self, decision_id: str, artifact_id: str) -> None: - """Record that a decision produced an artifact.""" - decision = self._decision_repo.get_by_id(decision_id) - if decision: - updated = decision.with_artifact(artifact_id) - self._decision_repo.update(updated) - ``` - - [ ] Commit: "feat(service): implement downstream relationship updates" - - [ ] **D2.6b** [Hamza] Implement `mark_superseded()`: - ```python - def mark_superseded(self, decision_id: str, by_decision_id: str) -> None: - """Mark a decision as superseded by another.""" - decision = self._decision_repo.get_by_id(decision_id) - if not decision: - raise DecisionNotFoundError(decision_id) - - if decision.superseded_by: - raise AlreadySupersededError( - f"Decision {decision_id} already superseded by {decision.superseded_by}" - ) - - updated = decision.mark_superseded(by_decision_id) - self._decision_repo.update(updated) - logger.info(f"Marked decision {decision_id} as superseded by {by_decision_id}") - ``` - - [ ] Commit: "feat(service): implement mark_superseded()" - - [ ] Tests: Verify decision tree is built during Strategize - - [ ] **D2.7** [Rui] Write Behave scenarios in `features/decision_recording.feature`: - - [ ] **D2.7a** [Rui] Basic recording scenarios: - - [ ] Scenario: Record first decision creates root - - [ ] Given a plan "plan-123" with no decisions - - [ ] When I call record_decision with decision_type=PROMPT_DEFINITION - - [ ] Then a Decision is created with sequence_number=0 - - [ ] And parent_decision_id is None - - [ ] And is_root returns True - - [ ] Scenario: Record subsequent decisions increment sequence - - [ ] Given a plan with 2 existing decisions - - [ ] When I record another decision - - [ ] Then sequence_number is 2 - - [ ] Commit: "test(behave): add basic decision recording scenarios" - - [ ] **D2.7b** [Rui] Tree structure scenarios: - - [ ] Scenario: Child decision links to parent - - [ ] Given root decision D1 exists - - [ ] When I record decision D2 with parent_decision_id=D1.id - - [ ] Then D1.downstream_decision_ids contains D2.id - - [ ] Scenario: Get decision tree returns correct order - - [ ] Given decisions D1, D2, D3 with sequences 0, 1, 2 - - [ ] When I call get_decision_tree(plan_id) - - [ ] Then decisions are returned in sequence order - - [ ] Scenario: Get children returns direct children only - - [ ] Given D1 -> D2 -> D3 (D2 child of D1, D3 child of D2) - - [ ] When I call get_children(D1.id) - - [ ] Then only D2 is returned (not D3) - - [ ] Commit: "test(behave): add decision tree structure scenarios" - - [ ] **D2.7c** [Rui] Context snapshot scenarios: - - [ ] Scenario: Context snapshot captured with decision - - [ ] Given hot context "file contents..." - - [ ] When I record a decision - - [ ] Then decision.context_snapshot is not None - - [ ] And context_snapshot.hot_context_hash is valid SHA-256 - - [ ] Scenario: Duplicate context reuses existing snapshot - - [ ] Given decision D1 with context hash "abc123" - - [ ] When I record D2 with identical context - - [ ] Then D2.context_snapshot.snapshot_id differs from D1 - - [ ] But content is only stored once (deduplication) - - [ ] Commit: "test(behave): add context snapshot scenarios" - - [ ] **D2.7d** [Rui] Downstream relationship scenarios: - - [ ] Scenario: Subplan spawn updates downstream_plan_ids - - [ ] Given decision D1 of type SUBPLAN_SPAWN - - [ ] When subplan SP1 is created from D1 - - [ ] And add_downstream_plan(D1.id, SP1.id) is called - - [ ] Then D1.downstream_plan_ids contains SP1.id - - [ ] Scenario: Artifact production updates artifacts_produced - - [ ] Given decision D1 produces artifact A1 - - [ ] When add_artifact(D1.id, A1.id) is called - - [ ] Then D1.artifacts_produced contains A1.id - - [ ] Commit: "test(behave): add downstream relationship scenarios" +**Parallel Group D2: Decision Recording Service [Hamza + Luis]** (depends on D1) +- [ ] **COMMIT (Owner: Hamza | Group: D2.service) - Commit message: "feat(service): add decision recording and snapshot store"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement `DecisionService` with `record_decision`, sequence numbers, tree queries, and downstream linking. + - [ ] Code [Hamza]: Add `ContextSnapshotStore` interface with a file-backed MVP implementation and hash dedupe. + - [ ] Code [Luis]: Integrate decision recording into strategize/execute phases (prompt/strategy/subplan/tool decisions). + - [ ] Docs [Hamza]: Add `docs/reference/decision_service.md` covering recording and snapshots. + - [ ] Tests (Behave) [Rui]: Add `features/decision_recording.feature` scenarios. + - [ ] Tests (Robot) [Rui]: Add `robot/decision_recording.robot` integration smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_recording_bench.py` for record throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(service): add decision recording and snapshot store"`. -- [ ] **Stage D3: Decision CLI & Viewing** (Day 16-17) **[Hamza]** - - **SEQUENTIAL ORDER**: D3.1 (tree command) → D3.2 (explain command) → D3.3 (JSON output) → D3.4 (guidance-file) → D3.5 (Tests) - - - [ ] Code: Decision viewing commands - - [ ] **D3.1** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan tree [plan_id]`: - - [ ] **D3.1a** [Hamza] Create command in `src/cleveragents/cli/commands/plan.py`: - ```python - @plan.command("tree") - @click.argument("plan_id", required=False) - @click.option("--format", "output_format", - type=click.Choice(["tree", "json", "flat"]), default="tree") - @click.option("--show-superseded", is_flag=True, - help="Include superseded decisions") - def show_tree(plan_id: str | None, output_format: str, show_superseded: bool): - """Display the decision tree for a plan.""" - ``` - - [ ] Commit: "feat(cli): add plan tree command signature" - - [ ] **D3.1b** [Hamza] Implement plan resolution: - ```python - # If no plan_id, use current/most recent plan - if not plan_id: - plan = plan_service.get_current_plan() - if not plan: - console.print("[red]No active plan. Specify a plan ID.[/red]") - raise SystemExit(1) - plan_id = plan.plan_id - - # Fetch decision tree - decisions = decision_service.get_decision_tree(plan_id) - if not decisions: - console.print(f"[yellow]No decisions recorded for plan {plan_id}[/yellow]") - return - ``` - - [ ] Commit: "feat(cli): implement plan resolution for tree command" - - [ ] **D3.1c** [Hamza] Implement tree rendering with Rich: - ```python - def _render_decision_tree(decisions: list[Decision], show_superseded: bool): - """Render decision tree using Rich Tree.""" - from rich.tree import Tree - from rich.text import Text - - # Build tree structure - root_decisions = [d for d in decisions if d.is_root] - by_parent: dict[str, list[Decision]] = {} - for d in decisions: - if d.parent_decision_id: - by_parent.setdefault(d.parent_decision_id, []).append(d) - - # Create Rich tree - tree = Tree("[bold]Decision Tree[/bold]") - - def add_node(parent_tree, decision: Decision): - # Format decision display - type_color = _get_type_color(decision.decision_type) - label = Text() - label.append(f"[{decision.decision_type.value}] ", style=type_color) - label.append(f'"{decision.question[:40]}..."' if len(decision.question) > 40 else f'"{decision.question}"') - - if decision.confidence_score: - label.append(f" (conf: {decision.confidence_score:.2f})", style="dim") - - # Mark superseded - if decision.superseded_by: - if not show_superseded: - return - label.stylize("strike dim") - label.append(" [SUPERSEDED]", style="yellow") - - # Mark corrections - if decision.is_correction: - label.append(" [CORRECTION]", style="green") - - # Add subplan links - for subplan_id in decision.downstream_plan_ids: - label.append(f" → {subplan_id[:8]}", style="cyan") - - node = parent_tree.add(label) - - # Add children recursively - for child in by_parent.get(decision.decision_id, []): - add_node(node, child) - - for root in root_decisions: - add_node(tree, root) - - console.print(tree) - ``` - - [ ] Commit: "feat(cli): implement tree rendering with Rich" - - [ ] **D3.1d** [Hamza] Add type-specific coloring: - ```python - def _get_type_color(decision_type: DecisionType) -> str: - """Get color for decision type.""" - colors = { - DecisionType.PROMPT_DEFINITION: "bold white", - DecisionType.STRATEGY_CHOICE: "blue", - DecisionType.IMPLEMENTATION_CHOICE: "cyan", - DecisionType.RESOURCE_SELECTION: "magenta", - DecisionType.SUBPLAN_SPAWN: "green", - DecisionType.TOOL_INVOCATION: "yellow", - DecisionType.ERROR_RECOVERY: "red", - DecisionType.VALIDATION_RESPONSE: "orange3", - DecisionType.USER_INTERVENTION: "bold yellow", - } - return colors.get(decision_type, "white") - ``` - - [ ] Commit: "feat(cli): add decision type coloring" - - [ ] **D3.2** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan explain `: - - [ ] **D3.2a** [Hamza] Create command signature: - ```python - @plan.command("explain") - @click.argument("decision_id") - @click.option("--show-context", is_flag=True, help="Show full context snapshot") - @click.option("--show-reasoning", is_flag=True, help="Show raw LLM reasoning") - def explain_decision(decision_id: str, show_context: bool, show_reasoning: bool): - """Show detailed explanation of a specific decision.""" - ``` - - [ ] Commit: "feat(cli): add plan explain command signature" - - [ ] **D3.2b** [Hamza] Implement detailed display: - ```python - decision = decision_service.get_decision(decision_id) - if not decision: - console.print(f"[red]Decision {decision_id} not found[/red]") - raise SystemExit(1) - - # Create panels for display - from rich.panel import Panel - from rich.table import Table - - # Header panel - header = Panel( - f"[bold]Decision: {decision.decision_id}[/bold]\n" - f"Type: {decision.decision_type.value}\n" - f"Plan: {decision.plan_id}", - title="Decision Details" - ) - console.print(header) - - # Question and answer - console.print(f"\n[bold]Question:[/bold] {decision.question}") - console.print(f"\n[bold green]Chosen:[/bold green] {decision.chosen_option}") - - # Alternatives - if decision.alternatives_considered: - console.print("\n[bold]Alternatives Considered:[/bold]") - for alt in decision.alternatives_considered: - console.print(f" • {alt}") - - # Confidence and rationale - if decision.confidence_score is not None: - bar = "█" * int(decision.confidence_score * 10) + "░" * (10 - int(decision.confidence_score * 10)) - console.print(f"\n[bold]Confidence:[/bold] {decision.confidence_score:.2f} [{bar}]") - - if decision.rationale: - console.print(f"\n[bold]Rationale:[/bold] {decision.rationale}") - ``` - - [ ] Commit: "feat(cli): implement explain decision display" - - [ ] **D3.2c** [Hamza] Show upstream/downstream relationships: - ```python - # Upstream (ancestors) - ancestors = decision_service.get_ancestors(decision_id) - if ancestors: - console.print("\n[bold]Decision Path (what led here):[/bold]") - for i, anc in enumerate(reversed(ancestors)): - indent = " " * i - console.print(f"{indent}↳ [{anc.decision_type.value}] {anc.question[:50]}") - - # Downstream impact - children = decision_service.get_children(decision_id) - if children or decision.downstream_plan_ids: - console.print("\n[bold]Downstream Impact:[/bold]") - if children: - console.print(f" • {len(children)} child decisions") - if decision.downstream_plan_ids: - console.print(f" • {len(decision.downstream_plan_ids)} subplans spawned:") - for sp_id in decision.downstream_plan_ids: - console.print(f" - {sp_id}") - if decision.artifacts_produced: - console.print(f" • {len(decision.artifacts_produced)} artifacts produced") - ``` - - [ ] Commit: "feat(cli): add upstream/downstream display" - - [ ] **D3.2d** [Hamza] Add context and reasoning display: - ```python - # Context snapshot - if show_context: - console.print("\n[bold]Context Snapshot:[/bold]") - console.print(f" Hash: {decision.context_snapshot.hot_context_hash[:16]}...") - console.print(f" Resources: {', '.join(decision.context_snapshot.relevant_resources)}") - - # Optionally show full content - try: - _, content = snapshot_store.retrieve(decision.context_snapshot.snapshot_id) - console.print(Panel(content[:1000] + "..." if len(content) > 1000 else content, - title="Context Content")) - except Exception as e: - console.print(f" [dim]Content not available: {e}[/dim]") - - # Raw LLM reasoning - if show_reasoning and decision.actor_reasoning: - console.print(Panel(decision.actor_reasoning, title="LLM Reasoning")) - ``` - - [ ] Commit: "feat(cli): add context and reasoning display" - - [ ] **D3.3** [Hamza] Implement JSON output: - - [ ] **D3.3a** [Hamza] Add JSON format to tree command: - ```python - if output_format == "json": - # Build JSON structure - def decision_to_dict(d: Decision) -> dict: - return { - "decision_id": d.decision_id, - "type": d.decision_type.value, - "question": d.question, - "chosen_option": d.chosen_option, - "alternatives": d.alternatives_considered, - "confidence": d.confidence_score, - "rationale": d.rationale, - "parent_id": d.parent_decision_id, - "sequence": d.sequence_number, - "is_correction": d.is_correction, - "superseded_by": d.superseded_by, - "downstream_decisions": d.downstream_decision_ids, - "downstream_plans": d.downstream_plan_ids, - "created_at": d.created_at.isoformat() - } - - tree_json = { - "plan_id": plan_id, - "decision_count": len(decisions), - "decisions": [decision_to_dict(d) for d in decisions] - } - - print(json.dumps(tree_json, indent=2)) - ``` - - [ ] Commit: "feat(cli): add JSON output for plan tree" - - [ ] **D3.3b** [Hamza] Add flat format (for scripting): - ```python - if output_format == "flat": - # Tab-separated values for easy parsing - print("ID\tTYPE\tSEQ\tPARENT\tQUESTION") - for d in decisions: - print(f"{d.decision_id}\t{d.decision_type.value}\t{d.sequence_number}\t" - f"{d.parent_decision_id or '-'}\t{d.question[:50]}") - ``` - - [ ] Commit: "feat(cli): add flat output for plan tree" - - [ ] **D3.4** [Hamza] Implement `--guidance-file` option: - - [ ] **D3.4a** [Hamza] Add option to plan correct command: - ```python - @plan.command("correct") - @click.argument("decision_id") - @click.option("--mode", type=click.Choice(["revert", "append"]), required=True) - @click.option("--guidance", "-g", help="Correction guidance text") - @click.option("--guidance-file", "-f", type=click.File('r'), - help="Read guidance from file (use - for stdin)") - @click.option("--dry-run", is_flag=True, help="Show impact without executing") - def correct_decision(decision_id, mode, guidance, guidance_file, dry_run): - """Correct a decision and re-execute affected work.""" - ``` - - [ ] Commit: "feat(cli): add guidance-file option to correct command" - - [ ] **D3.4b** [Hamza] Handle guidance source priority: - ```python - # Get guidance from appropriate source - if guidance_file: - guidance_text = guidance_file.read() - elif guidance: - guidance_text = guidance - else: - console.print("[red]Either --guidance or --guidance-file is required[/red]") - raise SystemExit(1) - - if not guidance_text.strip(): - console.print("[red]Guidance cannot be empty[/red]") - raise SystemExit(1) - ``` - - [ ] Commit: "feat(cli): implement guidance source handling" - - [ ] Tests: Behave scenarios for decision CLI - - [ ] **D3.5** [Rui] Write Behave scenarios in `features/decision_cli.feature`: - - [ ] **D3.5a** [Rui] Tree command scenarios: - - [ ] Scenario: Display decision tree for plan - - [ ] Given plan with 5 decisions in tree structure - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id}` - - [ ] Then output shows tree with all decisions - - [ ] And decisions are color-coded by type - - [ ] Scenario: Tree command with JSON format - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --format=json` - - [ ] Then output is valid JSON - - [ ] And JSON contains all decision fields - - [ ] Scenario: Tree hides superseded by default - - [ ] Given decision D1 superseded by D1' - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree` - - [ ] Then D1 is not shown - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree --show-superseded` - - [ ] Then D1 is shown with strikethrough - - [ ] Commit: "test(behave): add tree command scenarios" - - [ ] **D3.5b** [Rui] Explain command scenarios: - - [ ] Scenario: Explain shows full decision details - - [ ] Given decision D1 with all fields populated - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` - - [ ] Then output shows question, chosen option, alternatives - - [ ] And output shows confidence and rationale - - [ ] Scenario: Explain shows upstream path - - [ ] Given decision D3 with ancestors D1 -> D2 -> D3 - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D3.id}` - - [ ] Then output shows "Decision Path" section - - [ ] And D1 and D2 are listed as ancestors - - [ ] Scenario: Explain shows downstream impact - - [ ] Given decision D1 with 2 child decisions and 1 subplan - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan explain {D1.id}` - - [ ] Then output shows "Downstream Impact" section - - [ ] And shows "2 child decisions" and "1 subplan" - - [ ] Commit: "test(behave): add explain command scenarios" - - [ ] **D3.5c** [Rui] Guidance file scenarios: - - [ ] Scenario: Read guidance from file - - [ ] Given guidance file with text "Fix the authentication bug" - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file guidance.txt` - - [ ] Then correction uses the file content as guidance - - [ ] Scenario: Read guidance from stdin - - [ ] When I run `echo "Fix bug" | agents [--data-dir PATH] [--config-path PATH] plan correct {id} --mode=append --guidance-file=-` - - [ ] Then correction uses stdin content as guidance - - [ ] Commit: "test(behave): add guidance file scenarios" +**Parallel Group D3: Decision CLI & Viewing [Hamza + Rui]** (depends on D1/D2) +- [ ] **COMMIT (Owner: Hamza | Group: D3.cli) - Commit message: "feat(cli): add plan tree and explain commands"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement `plan tree` and `plan explain` with rich/json/flat formats and `--show-superseded`/`--show-context`. + - [ ] Code [Hamza]: Add `--show-reasoning` to include confidence and alternatives in explain output per spec. + - [ ] Docs [Hamza]: Update CLI reference for decision viewing commands. + - [ ] Tests (Behave) [Rui]: Add tree/explain scenarios including superseded handling. + - [ ] Tests (Robot) [Rui]: Add `robot/decision_cli.robot` smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_cli_bench.py` for tree rendering overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(cli): add plan tree and explain commands"`. -- [ ] **Stage D4: Decision Correction Mechanism** (Day 17-19) **[Jeff - CRITICAL FOR 30-DAY GOAL]** - - **IMPORTANCE**: This is the core mechanism that enables large project autonomy. Without decision correction, any mistake requires restarting from scratch. With it, users can guide the system to correct specific decisions and only recompute affected work. - - **SEQUENTIAL ORDER**: D4.1 (Service) → D4.2 (Sandbox Checkpoints) → D4.3 (Re-execution) → D4.4-D4.5 (CLI) - - - [ ] Code: Implement decision correction (core to large project autonomy) - - [ ] **D4.1** [Jeff] Implement correction service in `src/cleveragents/application/services/correction_service.py`: - - [ ] **D4.1a** [Jeff] Create service scaffold and types: - - [ ] Import necessary domain models (Decision, Plan, DecisionType) - - [ ] Import repositories (DecisionRepository, LifecyclePlanRepository) - - [ ] Define `CorrectionResult` dataclass: - - [ ] `success: bool` - Whether correction succeeded - - [ ] `correction_attempt_id: str` - ULID of the correction attempt - - [ ] `new_decision_id: str | None` - ID of new decision (for revert mode) - - [ ] `subplan_id: str | None` - ID of fix subplan (for append mode) - - [ ] `affected_decisions: list[str]` - IDs of invalidated decisions - - [ ] `affected_plans: list[str]` - IDs of invalidated subplans - - [ ] `affected_artifacts: list[str]` - IDs of invalidated artifacts - - [ ] `error: str | None` - Error message if failed - - [ ] Define `ImpactAnalysis` dataclass: - - [ ] `decision_id: str` - Decision being analyzed - - [ ] `downstream_decisions: list[str]` - All affected decisions - - [ ] `downstream_plans: list[str]` - All affected subplans - - [ ] `downstream_artifacts: list[str]` - All affected artifacts - - [ ] `total_tokens_to_recompute: int | None` - Estimated cost - - [ ] Create `CorrectionService` class with DI for repositories - - [ ] Commit: "feat(correction): add CorrectionService scaffold and types" - - [ ] **D4.1b** [Jeff] Implement `correct_decision_revert(decision_id: str, guidance: str) -> CorrectionResult`: - - [ ] **Step 1: Validation** - - [ ] Fetch decision from repository - - [ ] Raise `DecisionNotFoundError` if not exists - - [ ] Fetch parent plan - - [ ] Raise `PlanNotCorrectableError` if plan.phase == APPLIED - - [ ] Raise `PlanNotCorrectableError` if decision.superseded_by is not None (already corrected) - - [ ] **Step 2: Impact Analysis** - - [ ] Call `identify_downstream_impact(decision_id)` to find all affected entities - - [ ] Log: "Correction will affect {n} decisions, {m} subplans, {k} artifacts" - - [ ] **Step 3: Create Correction Attempt Record** - - [ ] Generate new ULID for correction_attempt_id - - [ ] Create `correction_attempts` record with: - - [ ] `attempt_id = correction_attempt_id` - - [ ] `plan_id = decision.plan_id` - - [ ] `original_decision_id = decision_id` - - [ ] `status = 'pending'` - - [ ] `guidance = guidance` - - [ ] `created_at = now()` - - [ ] Persist to database - - [ ] **Step 4: Archive Old Subtree** - - [ ] For each affected decision: - - [ ] Create copy in `archived_decisions` table with original values - - [ ] Store reference to correction_attempt_id - - [ ] For each affected artifact: - - [ ] Move file to archive location - - [ ] Update artifact record with archive path - - [ ] Log: "Archived {n} decisions and {k} artifacts" - - [ ] **Step 5: Invalidate Old Decisions** - - [ ] For each affected decision starting from decision_id: - - [ ] Set `superseded_by = None` (will be filled when new decision created) - - [ ] Mark in execution_log that decision is invalidated - - [ ] For each affected subplan: - - [ ] Set state to CANCELLED - - [ ] Rollback subplan sandboxes - - [ ] **Step 6: Create Correction Decision** - - [ ] Create new Decision with: - - [ ] `decision_id = new ULID` - - [ ] `plan_id = original.plan_id` - - [ ] `parent_decision_id = original.parent_decision_id` (same parent) - - [ ] `sequence_number = original.sequence_number` (replaces in sequence) - - [ ] `decision_type = original.decision_type` - - [ ] `is_correction = True` - - [ ] `corrects_decision_id = decision_id` - - [ ] `question = original.question` - - [ ] `chosen_option = guidance` (user's correction) - - [ ] `rationale = f"User correction: {guidance}"` - - [ ] Update original decision: `superseded_by = new_decision_id` - - [ ] Persist new decision - - [ ] **Step 7: Rollback Sandbox** - - [ ] Get checkpoint_id from original decision's context_snapshot - - [ ] Call `sandbox_manager.rollback_to_checkpoint(plan_id, checkpoint_id)` - - [ ] This restores sandbox state to before the decision was made - - [ ] **Step 8: Re-execute from Decision Point** - - [ ] Build re-execution context: - - [ ] Include all decisions up to (but not including) the corrected one - - [ ] Include the new correction decision - - [ ] Include the guidance as additional context - - [ ] Call appropriate phase handler: - - [ ] If decision was in Strategize: resume strategy actor - - [ ] If decision was in Execute: resume execution actor - - [ ] Let actor generate new downstream decisions - - [ ] Continue normal phase flow - - [ ] **Step 9: Finalize** - - [ ] Update correction_attempt: `status = 'completed'`, `new_decision_id = new_decision.decision_id` - - [ ] Increment plan.attempt counter - - [ ] Return CorrectionResult with all IDs and counts - - [ ] **Error Handling** - - [ ] If any step fails, update correction_attempt: `status = 'failed'`, `error = message` - - [ ] Do NOT rollback the archive (preserve for debugging) - - [ ] Return CorrectionResult with success=False, error message - - [ ] Commit: "feat(correction): implement correct_decision_revert()" - - [ ] **D4.1c** [Jeff] Implement `correct_decision_append(decision_id: str, guidance: str) -> CorrectionResult`: - - [ ] **Step 1: Validation** (same as revert, but less strict) - - [ ] Fetch decision and plan - - [ ] Raise error if plan already Applied (can't modify) - - [ ] **Step 2: Create Fix Subplan** - - [ ] Create new action (or use built-in "fix" action) with: - - [ ] Description based on guidance - - [ ] Target resources from original decision's scope - - [ ] Use PlanLifecycleService.use_action() to create subplan - - [ ] Set subplan.parent_plan_id to original plan - - [ ] Set subplan's prompt to include: - - [ ] Original decision context - - [ ] What went wrong (from guidance) - - [ ] Instructions to fix - - [ ] **Step 3: Link to Decision** - - [ ] Add subplan_id to original decision's downstream_plan_ids - - [ ] Create new decision record of type USER_INTERVENTION - - [ ] Store guidance and fix plan reference - - [ ] **Step 4: Execute Fix Plan** - - [ ] If automation level allows, start subplan execution - - [ ] Otherwise, return subplan_id for manual execution - - [ ] Return CorrectionResult with subplan_id - - [ ] Commit: "feat(correction): implement correct_decision_append()" - - [ ] **D4.1d** [Jeff] Implement `identify_downstream_impact(decision_id: str) -> ImpactAnalysis`: - - [ ] **Recursive Decision Collection** - - [ ] Start with decision_id - - [ ] Query all decisions where parent_decision_id = current - - [ ] Recursively process each child - - [ ] Also check downstream_decision_ids relationship (DAG) - - [ ] Collect all IDs in depth-first order - - [ ] **Subplan Collection** - - [ ] For each decision, check if decision_type == SUBPLAN_SPAWN - - [ ] If so, add downstream_plan_ids to affected plans - - [ ] Recursively get that subplan's decisions too - - [ ] **Artifact Collection** - - [ ] For each affected decision, get artifacts_produced list - - [ ] Deduplicate (same artifact may be referenced multiple times) - - [ ] **Cost Estimation** (optional) - - [ ] Estimate tokens by summing context sizes of affected decisions - - [ ] This helps user decide if correction is worth it - - [ ] Return ImpactAnalysis with all collected data - - [ ] Commit: "feat(correction): implement identify_downstream_impact()" - - [ ] **D4.2** [Jeff] Implement sandbox checkpointing for correction: - - [ ] **D4.2a** [Jeff] Extend SandboxManager to track checkpoints: - - [ ] Add `create_checkpoint(plan_id: str, label: str) -> str`: - - [ ] For git sandboxes: commit current state with checkpoint tag - - [ ] For filesystem sandboxes: snapshot directory (or use git worktree trick) - - [ ] Return checkpoint_id - - [ ] Add `list_checkpoints(plan_id: str) -> list[Checkpoint]`: - - [ ] Return all checkpoints for the plan in chronological order - - [ ] Commit: "feat(sandbox): add checkpoint creation to SandboxManager" - - [ ] **D4.2b** [Jeff] Store checkpoint ID with each decision: - - [ ] Update Decision model: add `checkpoint_id: str | None` field - - [ ] When decision is created, auto-create checkpoint - - [ ] Store checkpoint_id in decision record - - [ ] Commit: "feat(decision): track checkpoint_id per decision" - - [ ] **D4.2c** [Jeff] Implement `rollback_to_checkpoint(plan_id: str, checkpoint_id: str) -> None`: - - [ ] Find all sandboxes for plan_id - - [ ] For each sandbox: - - [ ] If git: `git reset --hard {checkpoint_tag}` - - [ ] If filesystem: restore from snapshot - - [ ] Clear any state beyond checkpoint - - [ ] Update sandbox status - - [ ] Commit: "feat(sandbox): implement rollback_to_checkpoint()" - - [ ] **D4.2d** [Jeff] Handle checkpoint cleanup: - - [ ] After successful apply, old checkpoints can be pruned - - [ ] Keep at least N most recent checkpoints for debugging - - [ ] Implement `prune_checkpoints(plan_id: str, keep_count: int) -> int` - - [ ] Commit: "feat(sandbox): implement checkpoint pruning" - - [ ] **D4.3** [Jeff] Implement re-execution from correction point: - - [ ] **D4.3a** [Jeff] Build context for resumed execution: - - [ ] Create `CorrectionContext` dataclass: - - [ ] `original_decision: Decision` - What was being corrected - - [ ] `correction_decision: Decision` - The new corrected decision - - [ ] `guidance: str` - User's correction text - - [ ] `prior_decisions: list[Decision]` - Decisions that remain valid - - [ ] `invalidated_decisions: list[Decision]` - For reference/diff - - [ ] Commit: "feat(correction): define CorrectionContext" - - [ ] **D4.3b** [Jeff] Inject correction context into actor: - - [ ] Update strategy/execution actor invocation to accept CorrectionContext - - [ ] Actor prompt includes: - - [ ] "You previously decided: {original_decision.chosen_option}" - - [ ] "This decision is being corrected because: {guidance}" - - [ ] "Please reconsider and make a new decision based on this feedback." - - [ ] Actor should acknowledge correction and proceed - - [ ] Commit: "feat(correction): inject correction context into actor" - - [ ] **D4.3c** [Jeff] Resume phase execution from decision point: - - [ ] If correction is in Strategize phase: - - [ ] Resume strategy actor from decision point - - [ ] Actor generates new downstream decisions - - [ ] Continue until Strategize complete - - [ ] If correction is in Execute phase: - - [ ] Resume execution actor - - [ ] Regenerate affected subplans - - [ ] Continue execution - - [ ] Normal Apply phase follows - - [ ] Commit: "feat(correction): implement re-execution from decision point" - - [ ] **D4.3d** [Jeff] Handle correction failures: - - [ ] If actor fails during re-execution: - - [ ] Update correction_attempt status to 'failed' - - [ ] Preserve both old and new state for debugging - - [ ] Allow user to try different guidance - - [ ] Max correction attempts per decision: 3 (configurable) - - [ ] Commit: "feat(correction): handle re-execution failures" - - [ ] **D4.4** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=revert --guidance ""`: - - [ ] **D4.4a** [Hamza] Create correction command in plan CLI: - - [ ] Add `@plan.command("correct")` with Click - - [ ] Required argument: `decision_id: str` - - [ ] Required option: `--mode: str` (choices: revert, append) - - [ ] Required option: `--guidance: str` or `--guidance-file: Path` - - [ ] Optional: `--dry-run` to show impact without executing - - [ ] Commit: "feat(cli): add plan correct command scaffold" - - [ ] **D4.4b** [Hamza] Implement command logic for revert mode: - - [ ] Validate decision_id format (ULID) - - [ ] If --dry-run: - - [ ] Call CorrectionService.identify_downstream_impact() - - [ ] Display impact analysis - - [ ] Exit without making changes - - [ ] Show confirmation prompt: "This will affect {n} decisions. Continue? [y/N]" - - [ ] Support `--yes` to bypass confirmation - - [ ] If confirmed (or --yes), call CorrectionService.correct_decision_revert() - - [ ] Display progress with Rich console: - - [ ] "[1/5] Analyzing impact..." - - [ ] "[2/5] Archiving old decisions..." - - [ ] "[3/5] Rolling back sandbox..." - - [ ] "[4/5] Re-executing from decision point..." - - [ ] "[5/5] Finalizing correction..." - - [ ] On success, display: - - [ ] New decision ID - - [ ] Count of regenerated decisions - - [ ] Diff summary (files changed old vs new) - - [ ] On failure, display error with recovery suggestions - - [ ] Commit: "feat(cli): implement plan correct revert mode" - - [ ] **D4.4c** [Hamza] Handle guidance-file option: - - [ ] If --guidance-file specified: - - [ ] Read file contents as guidance - - [ ] Support `-` for stdin: `cat guidance.txt | agents [--data-dir PATH] [--config-path PATH] plan correct ... --guidance-file=-` - - [ ] Validate guidance is not empty - - [ ] Commit: "feat(cli): add guidance-file support to plan correct" - - [ ] **D4.5** [Hamza] Implement `agents [--data-dir PATH] [--config-path PATH] plan correct --mode=append --guidance ""`: - - [ ] **D4.5a** [Hamza] Implement append mode command: - - [ ] No confirmation needed (additive, not destructive) - - [ ] Call CorrectionService.correct_decision_append() - - [ ] Display created subplan ID - - [ ] If automation allows, show execution progress - - [ ] Otherwise show: "Fix subplan created: {subplan_id}. Run `agents [--data-dir PATH] [--config-path PATH] plan execute {subplan_id}` to apply fix." - - [ ] Commit: "feat(cli): implement plan correct append mode" - - [ ] Tests: Correction mechanism tests (CRITICAL for 30-day goal) - - [ ] **D4.6** [Rui] Write Behave scenarios in `features/decision_correction.feature`: - - [ ] **D4.6a** [Rui] Revert mode basic scenarios: - - [ ] Scenario: Correct early decision re-executes downstream work - - [ ] Given a plan with 3 decisions (D1 → D2 → D3) in sequence - - [ ] And D1 chose "use PostgreSQL" with downstream D2 choosing "use psycopg2" - - [ ] When I correct D1 with guidance "use SQLite instead" - - [ ] Then D1 is marked superseded - - [ ] And a new D1' is created with chosen_option "use SQLite" - - [ ] And D2, D3 are invalidated and regenerated - - [ ] And new D2' reflects SQLite (e.g., "use sqlite3") - - [ ] And the correction_attempt record shows status='completed' - - [ ] Commit: "test(behave): add basic revert correction scenario" - - [ ] **D4.6b** [Rui] Subplan invalidation scenarios: - - [ ] Scenario: Correct decision with subplans invalidates subplans - - [ ] Given a plan where D2 is a SUBPLAN_SPAWN decision - - [ ] And subplan SP1 was created from D2 - - [ ] And SP1 has completed some work - - [ ] When I correct D2 with new guidance - - [ ] Then SP1 is marked as CANCELLED - - [ ] And SP1's sandbox is rolled back - - [ ] And a new subplan SP2 is created based on new guidance - - [ ] And SP2 uses the corrected context - - [ ] Commit: "test(behave): add subplan invalidation correction scenario" - - [ ] **D4.6c** [Rui] Append mode scenarios: - - [ ] Scenario: Append mode creates fix subplan without modifying history - - [ ] Given a plan with D1, D2, D3 all completed - - [ ] And the outcome has a bug due to D2's decision - - [ ] When I correct D2 with mode=append and guidance "add error handling" - - [ ] Then D2 is NOT marked as superseded - - [ ] And a new fix subplan is created - - [ ] And the fix subplan's prompt includes the guidance - - [ ] And the fix subplan targets the same resources as D2 - - [ ] Commit: "test(behave): add append mode correction scenario" - - [ ] **D4.6d** [Rui] Safety and error scenarios: - - [ ] Scenario: Cannot correct decision in Applied plan - - [ ] Given a plan that has been Applied successfully - - [ ] When I try to correct any decision - - [ ] Then I receive error "Cannot correct decisions in an applied plan" - - [ ] And no changes are made - - [ ] Scenario: Cannot correct already-corrected decision - - [ ] Given decision D1 that was already corrected to D1' - - [ ] When I try to correct D1 again - - [ ] Then I receive error "Decision already superseded" - - [ ] And hint "Correct the replacement decision D1' instead" - - [ ] Commit: "test(behave): add correction safety scenarios" - - [ ] **D4.6e** [Rui] History preservation scenarios: - - [ ] Scenario: Correction preserves history for comparison - - [ ] Given I correct decision D1 with new guidance - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan tree {plan_id} --show-superseded` - - [ ] Then I see the original D1 decision details - - [ ] And I see the correction D1' decision details - - [ ] And I can compare the outcomes via `agents [--data-dir PATH] [--config-path PATH] plan diff --correction {plan_id}` - - [ ] Scenario: Archived artifacts are accessible - - [ ] Given correction archived some generated files - - [ ] Then I can retrieve archived files for diff comparison - - [ ] Commit: "test(behave): add history preservation scenarios" - - [ ] **D4.6f** [Rui] Dry-run and impact analysis scenarios: - - [ ] Scenario: Dry-run shows impact without making changes - - [ ] Given a plan with 5 decisions and 2 subplans - - [ ] When I run `agents [--data-dir PATH] [--config-path PATH] plan correct D2 --mode=revert --guidance "..." --dry-run` - - [ ] Then I see "This will affect:" - - [ ] And I see "- 3 decisions" - - [ ] And I see "- 1 subplan" - - [ ] And I see "- Estimated recomputation: ~5000 tokens" - - [ ] And no changes are made to the plan - - [ ] Commit: "test(behave): add dry-run and impact analysis scenarios" +**Parallel Group D4: Decision Correction [Jeff + Luis]** (depends on D2/D3) +- [ ] **COMMIT (Owner: Jeff | Group: D4.revert) - Commit message: "feat(service): add decision correction revert flow"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement correction impact analysis and dry-run reporting. + - [ ] Code [Jeff]: Revert flow with checkpoint rollback, supersede downstream decisions, and subtree re-exec. + - [ ] Code [Jeff]: Persist correction attempt IDs and link them to superseded decisions. + - [ ] Docs [Jeff]: Add `docs/reference/decision_correction.md` for revert behavior. + - [ ] Tests (Behave) [Rui]: Add revert + dry-run scenarios. + - [ ] Tests (Robot) [Rui]: Add revert integration tests with checkpoint rollback. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_correction_revert_bench.py` for correction overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(service): add decision correction revert flow"`. +- [ ] **COMMIT (Owner: Jeff | Group: D4.append) - Commit message: "feat(service): add decision correction append flow"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Append flow creating fix subplan without rewriting history; link correction attempt + decision tree updates. + - [ ] Code [Jeff]: Record append corrections as separate subtree with explicit lineage. + - [ ] Docs [Jeff]: Extend correction docs for append mode and guidance-file usage. + - [ ] Tests (Behave) [Rui]: Add append correction scenarios. + - [ ] Tests (Robot) [Rui]: Add append correction smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_correction_append_bench.py` for append overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(service): add decision correction append flow"`. -- [ ] **Stage D5: Decision Persistence** (Day 18-19) **[Hamza]** - - **SEQUENTIAL ORDER**: D5.1 (decisions table) → D5.2 (dependencies table) → D5.3 (correction_attempts) → D5.4 (context_snapshots) → D5.5 (DecisionModel) → D5.6 (DecisionRepository) → D5.7 (Tests) - - - [ ] Code: Decision database schema - - [ ] **D5.1** [Hamza] Create Alembic migration for `decisions` table: - - [ ] **D5.1a** [Hamza] Generate migration file: - - [ ] Run `alembic revision --autogenerate -m "create_decisions_table"` - - [ ] Commit: "chore(db): generate decisions table migration" - - [ ] **D5.1b** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'decisions', - # Identity - sa.Column('decision_id', sa.Text(), nullable=False), - sa.Column('plan_id', sa.Text(), nullable=False), - - # Tree structure - sa.Column('parent_decision_id', sa.Text(), nullable=True), - sa.Column('sequence_number', sa.Integer(), nullable=False), - - # Decision content - sa.Column('decision_type', sa.Text(), nullable=False), - sa.Column('question', sa.Text(), nullable=False), - sa.Column('chosen_option', sa.Text(), nullable=False), - sa.Column('alternatives_considered', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('confidence_score', sa.Float(), nullable=True), - sa.Column('rationale', sa.Text(), nullable=False, server_default=''), - sa.Column('actor_reasoning', sa.Text(), nullable=True), - - # Context - sa.Column('context_snapshot_id', sa.Text(), nullable=False), - sa.Column('checkpoint_id', sa.Text(), nullable=True), - - # Downstream relationships (denormalized for query performance) - sa.Column('downstream_decision_ids', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('downstream_plan_ids', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('artifacts_produced', sa.JSON(), nullable=False, server_default='[]'), - - # Correction tracking - sa.Column('is_correction', sa.Boolean(), nullable=False, server_default='false'), - sa.Column('corrects_decision_id', sa.Text(), nullable=True), - sa.Column('superseded_by', sa.Text(), nullable=True), - - # Timestamp - sa.Column('created_at', sa.Text(), nullable=False), - - # Constraints - sa.PrimaryKeyConstraint('decision_id'), - sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['parent_decision_id'], ['decisions.decision_id'], ondelete='SET NULL'), - sa.ForeignKeyConstraint(['context_snapshot_id'], ['context_snapshots.snapshot_id']), - ) - ``` - - [ ] Commit: "feat(db): add decisions table schema" - - [ ] **D5.1c** [Hamza] Add indices for query optimization: - ```python - # Indices for common queries - op.create_index('ix_decisions_plan_id', 'decisions', ['plan_id']) - op.create_index('ix_decisions_parent_id', 'decisions', ['parent_decision_id']) - op.create_index('ix_decisions_plan_sequence', 'decisions', ['plan_id', 'sequence_number']) - op.create_index('ix_decisions_type', 'decisions', ['decision_type']) - op.create_index('ix_decisions_superseded', 'decisions', ['superseded_by'], - postgresql_where=sa.text('superseded_by IS NOT NULL')) - ``` - - [ ] Commit: "feat(db): add decisions table indices" - - [ ] **D5.1d** [Hamza] Add downgrade: - ```python - def downgrade(): - op.drop_index('ix_decisions_superseded') - op.drop_index('ix_decisions_type') - op.drop_index('ix_decisions_plan_sequence') - op.drop_index('ix_decisions_parent_id') - op.drop_index('ix_decisions_plan_id') - op.drop_table('decisions') - ``` - - [ ] Commit: "feat(db): add decisions table downgrade" - - [ ] **D5.2** [Hamza] Create Alembic migration for `decision_dependencies` table: - - [ ] **D5.2a** [Hamza] Define schema for DAG relationships: - ```python - def upgrade(): - op.create_table( - 'decision_dependencies', - sa.Column('id', sa.Integer(), autoincrement=True, nullable=False), - sa.Column('upstream_decision_id', sa.Text(), nullable=False), - sa.Column('downstream_decision_id', sa.Text(), nullable=False), - sa.Column('dependency_type', sa.Text(), nullable=False), # 'data', 'ordering', 'spawned' - sa.Column('created_at', sa.Text(), nullable=False), - - sa.PrimaryKeyConstraint('id'), - sa.ForeignKeyConstraint(['upstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['downstream_decision_id'], ['decisions.decision_id'], ondelete='CASCADE'), - sa.UniqueConstraint('upstream_decision_id', 'downstream_decision_id', name='uq_decision_dependency') - ) - op.create_index('ix_dep_upstream', 'decision_dependencies', ['upstream_decision_id']) - op.create_index('ix_dep_downstream', 'decision_dependencies', ['downstream_decision_id']) - ``` - - [ ] Commit: "feat(db): add decision_dependencies table" - - [ ] **D5.2b** [Hamza] Add downgrade: - - [ ] Drop indices and table - - [ ] Commit: "feat(db): add decision_dependencies downgrade" - - [ ] **D5.3** [Hamza] Create Alembic migration for `correction_attempts` table: - - [ ] **D5.3a** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'correction_attempts', - sa.Column('attempt_id', sa.Text(), nullable=False), - sa.Column('plan_id', sa.Text(), nullable=False), - sa.Column('original_decision_id', sa.Text(), nullable=False), - sa.Column('new_decision_id', sa.Text(), nullable=True), # Set when complete - sa.Column('mode', sa.Text(), nullable=False), # 'revert' or 'append' - sa.Column('guidance', sa.Text(), nullable=False), - sa.Column('status', sa.Text(), nullable=False), # pending, completed, failed - sa.Column('error_message', sa.Text(), nullable=True), - sa.Column('affected_decisions', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('affected_plans', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('created_at', sa.Text(), nullable=False), - sa.Column('completed_at', sa.Text(), nullable=True), - - sa.PrimaryKeyConstraint('attempt_id'), - sa.ForeignKeyConstraint(['plan_id'], ['lifecycle_plans.plan_id'], ondelete='CASCADE'), - sa.ForeignKeyConstraint(['original_decision_id'], ['decisions.decision_id']), - sa.ForeignKeyConstraint(['new_decision_id'], ['decisions.decision_id']) - ) - op.create_index('ix_correction_plan', 'correction_attempts', ['plan_id']) - op.create_index('ix_correction_status', 'correction_attempts', ['status']) - ``` - - [ ] Commit: "feat(db): add correction_attempts table" - - [ ] **D5.3b** [Hamza] Add downgrade: - - [ ] Commit: "feat(db): add correction_attempts downgrade" - - [ ] **D5.4** [Hamza] Create Alembic migration for `context_snapshots` table: - - [ ] **D5.4a** [Hamza] Define schema: - ```python - def upgrade(): - op.create_table( - 'context_snapshots', - sa.Column('snapshot_id', sa.Text(), nullable=False), - sa.Column('hot_context_hash', sa.Text(), nullable=False), - sa.Column('hot_context_ref', sa.Text(), nullable=False), # File path or blob ID - sa.Column('relevant_resources', sa.JSON(), nullable=False, server_default='[]'), - sa.Column('actor_state_ref', sa.Text(), nullable=True), - sa.Column('file_versions', sa.JSON(), nullable=False, server_default='{}'), - sa.Column('content_size_bytes', sa.Integer(), nullable=False, server_default='0'), - sa.Column('created_at', sa.Text(), nullable=False), - - sa.PrimaryKeyConstraint('snapshot_id') - ) - # Index for content deduplication - op.create_index('ix_snapshot_hash', 'context_snapshots', ['hot_context_hash']) - ``` - - [ ] Commit: "feat(db): add context_snapshots table" - - [ ] **D5.4b** [Hamza] Add downgrade: - - [ ] Commit: "feat(db): add context_snapshots downgrade" - - [ ] **D5.5** [Hamza] Create `DecisionModel` in `src/cleveragents/infrastructure/database/models.py`: - - [ ] **D5.5a** [Hamza] Define SQLAlchemy model: - ```python - class DecisionModel(Base): - __tablename__ = 'decisions' - - decision_id = Column(Text, primary_key=True) - plan_id = Column(Text, ForeignKey('lifecycle_plans.plan_id', ondelete='CASCADE'), nullable=False) - parent_decision_id = Column(Text, ForeignKey('decisions.decision_id', ondelete='SET NULL'), nullable=True) - sequence_number = Column(Integer, nullable=False) - - decision_type = Column(Text, nullable=False) - question = Column(Text, nullable=False) - chosen_option = Column(Text, nullable=False) - alternatives_considered = Column(JSON, nullable=False, default=list) - confidence_score = Column(Float, nullable=True) - rationale = Column(Text, nullable=False, default='') - actor_reasoning = Column(Text, nullable=True) - - context_snapshot_id = Column(Text, ForeignKey('context_snapshots.snapshot_id'), nullable=False) - checkpoint_id = Column(Text, nullable=True) - - downstream_decision_ids = Column(JSON, nullable=False, default=list) - downstream_plan_ids = Column(JSON, nullable=False, default=list) - artifacts_produced = Column(JSON, nullable=False, default=list) - - is_correction = Column(Boolean, nullable=False, default=False) - corrects_decision_id = Column(Text, nullable=True) - superseded_by = Column(Text, nullable=True) - - created_at = Column(Text, nullable=False) - - # Relationships - plan = relationship("LifecyclePlanModel", back_populates="decisions") - parent = relationship("DecisionModel", remote_side=[decision_id], backref="children") - context_snapshot = relationship("ContextSnapshotModel") - ``` - - [ ] Commit: "feat(db): add DecisionModel SQLAlchemy class" - - [ ] **D5.5b** [Hamza] Add domain conversion methods: - ```python - def to_domain(self) -> Decision: - """Convert to domain model.""" - return Decision( - decision_id=self.decision_id, - plan_id=self.plan_id, - parent_decision_id=self.parent_decision_id, - sequence_number=self.sequence_number, - decision_type=DecisionType(self.decision_type), - question=self.question, - chosen_option=self.chosen_option, - alternatives_considered=self.alternatives_considered or [], - confidence_score=self.confidence_score, - rationale=self.rationale, - actor_reasoning=self.actor_reasoning, - context_snapshot=self.context_snapshot.to_domain(), - checkpoint_id=self.checkpoint_id, - downstream_decision_ids=self.downstream_decision_ids or [], - downstream_plan_ids=self.downstream_plan_ids or [], - artifacts_produced=self.artifacts_produced or [], - is_correction=self.is_correction, - corrects_decision_id=self.corrects_decision_id, - superseded_by=self.superseded_by, - created_at=datetime.fromisoformat(self.created_at) - ) - - @classmethod - def from_domain(cls, decision: Decision) -> "DecisionModel": - """Create from domain model.""" - return cls( - decision_id=decision.decision_id, - plan_id=decision.plan_id, - parent_decision_id=decision.parent_decision_id, - sequence_number=decision.sequence_number, - decision_type=decision.decision_type.value, - question=decision.question, - chosen_option=decision.chosen_option, - alternatives_considered=decision.alternatives_considered, - confidence_score=decision.confidence_score, - rationale=decision.rationale, - actor_reasoning=decision.actor_reasoning, - context_snapshot_id=decision.context_snapshot.snapshot_id, - checkpoint_id=decision.checkpoint_id, - downstream_decision_ids=decision.downstream_decision_ids, - downstream_plan_ids=decision.downstream_plan_ids, - artifacts_produced=decision.artifacts_produced, - is_correction=decision.is_correction, - corrects_decision_id=decision.corrects_decision_id, - superseded_by=decision.superseded_by, - created_at=decision.created_at.isoformat() - ) - ``` - - [ ] Commit: "feat(db): add DecisionModel conversion methods" - - [ ] **D5.6** [Hamza] Implement `DecisionRepository` in `src/cleveragents/infrastructure/database/repositories.py`: - - [ ] **D5.6a** [Hamza] Define repository class: - ```python - class DecisionRepository: - """Repository for Decision persistence.""" - - def __init__(self, session_factory: Callable[[], Session]): - self._session_factory = session_factory - ``` - - [ ] Commit: "feat(repo): add DecisionRepository scaffold" - - [ ] **D5.6b** [Hamza] Implement `create()`: - ```python - def create(self, decision: Decision) -> Decision: - """Persist a new decision.""" - with self._session_factory() as session: - model = DecisionModel.from_domain(decision) - session.add(model) - try: - session.commit() - except IntegrityError as e: - session.rollback() - if "FOREIGN KEY" in str(e): - raise PlanNotFoundError(decision.plan_id) - raise - return decision - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.create()" - - [ ] **D5.6c** [Hamza] Implement `get_by_id()`: - ```python - def get_by_id(self, decision_id: str) -> Decision | None: - """Get decision by ID.""" - with self._session_factory() as session: - model = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(decision_id=decision_id).first() - return model.to_domain() if model else None - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_id()" - - [ ] **D5.6d** [Hamza] Implement `get_by_plan()`: - ```python - def get_by_plan(self, plan_id: str) -> list[Decision]: - """Get all decisions for a plan, ordered by sequence.""" - with self._session_factory() as session: - models = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(plan_id=plan_id).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_by_plan()" - - [ ] **D5.6e** [Hamza] Implement `get_children()`: - ```python - def get_children(self, decision_id: str) -> list[Decision]: - """Get direct children of a decision.""" - with self._session_factory() as session: - models = session.query(DecisionModel).options( - joinedload(DecisionModel.context_snapshot) - ).filter_by(parent_decision_id=decision_id).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_children()" - - [ ] **D5.6f** [Hamza] Implement `get_tree()` with recursive CTE: - ```python - def get_tree(self, plan_id: str) -> list[Decision]: - """Get full decision tree for a plan using recursive CTE.""" - with self._session_factory() as session: - # Use recursive CTE for efficient tree retrieval - cte = session.query(DecisionModel).filter( - DecisionModel.plan_id == plan_id, - DecisionModel.parent_decision_id.is_(None) - ).cte(name='decision_tree', recursive=True) - - cte_alias = aliased(DecisionModel, cte) - recursive = session.query(DecisionModel).join( - cte_alias, DecisionModel.parent_decision_id == cte_alias.decision_id - ) - cte = cte.union_all(recursive) - - models = session.query(DecisionModel).select_from(cte).order_by( - DecisionModel.sequence_number - ).all() - return [m.to_domain() for m in models] - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_tree()" - - [ ] **D5.6g** [Hamza] Implement `get_downstream()`: - ```python - def get_downstream(self, decision_id: str) -> list[Decision]: - """Get all downstream decisions (recursive).""" - with self._session_factory() as session: - # Get the starting decision - start = session.query(DecisionModel).filter_by( - decision_id=decision_id - ).first() - if not start: - return [] - - # Recursively collect all downstream - result = [] - to_process = list(start.downstream_decision_ids) - seen = set() - - while to_process: - did = to_process.pop(0) - if did in seen: - continue - seen.add(did) - - d = session.query(DecisionModel).filter_by(decision_id=did).first() - if d: - result.append(d.to_domain()) - to_process.extend(d.downstream_decision_ids) - - return result - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_downstream()" - - [ ] **D5.6h** [Hamza] Implement `update()`: - ```python - def update(self, decision: Decision) -> Decision: - """Update an existing decision.""" - with self._session_factory() as session: - model = session.query(DecisionModel).filter_by( - decision_id=decision.decision_id - ).first() - if not model: - raise DecisionNotFoundError(decision.decision_id) - - # Update fields - model.downstream_decision_ids = decision.downstream_decision_ids - model.downstream_plan_ids = decision.downstream_plan_ids - model.artifacts_produced = decision.artifacts_produced - model.superseded_by = decision.superseded_by - - session.commit() - return decision - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.update()" - - [ ] **D5.6i** [Hamza] Implement `get_max_sequence()`: - ```python - def get_max_sequence(self, plan_id: str) -> int | None: - """Get maximum sequence number for a plan.""" - with self._session_factory() as session: - result = session.query(func.max(DecisionModel.sequence_number)).filter_by( - plan_id=plan_id - ).scalar() - return result - ``` - - [ ] Commit: "feat(repo): implement DecisionRepository.get_max_sequence()" - - [ ] **D5.6j** [Hamza] Add retry decorator to all methods: - - [ ] Same pattern as other repositories - - [ ] Commit: "feat(repo): add retry decorator to DecisionRepository" - - [ ] Tests: Integration tests for decision persistence - - [ ] **D5.7** [Rui] Write Behave scenarios in `features/decision_persistence.feature`: - - [ ] **D5.7a** [Rui] Basic persistence scenarios: - - [ ] Scenario: Decision persists with all fields - - [ ] Given a valid Decision domain object - - [ ] When I call decision_repo.create(decision) - - [ ] Then decision is stored in database - - [ ] And get_by_id returns the decision - - [ ] And all fields match original - - [ ] Scenario: Decision FK to plan enforced - - [ ] Given no plan with ID "nonexistent" - - [ ] When I try to create decision with that plan_id - - [ ] Then PlanNotFoundError is raised - - [ ] Commit: "test(behave): add basic decision persistence scenarios" - - [ ] **D5.7b** [Rui] Tree query scenarios: - - [ ] Scenario: get_by_plan returns decisions in sequence order - - [ ] Given plan with decisions at sequences 0, 1, 2 - - [ ] When I call get_by_plan(plan_id) - - [ ] Then decisions are returned in sequence order - - [ ] Scenario: get_tree returns full hierarchy - - [ ] Given plan with 3-level decision tree - - [ ] When I call get_tree(plan_id) - - [ ] Then all decisions are returned - - [ ] And tree structure is preserved - - [ ] Scenario: get_children returns only direct children - - [ ] Given D1 -> D2 -> D3 hierarchy - - [ ] When I call get_children(D1.id) - - [ ] Then only D2 is returned - - [ ] Commit: "test(behave): add decision tree query scenarios" - - [ ] **D5.7c** [Rui] Context snapshot scenarios: - - [ ] Scenario: Context snapshot stored and retrievable - - [ ] Given decision with context_snapshot - - [ ] When decision is persisted - - [ ] Then context_snapshot_id is stored - - [ ] And snapshot can be retrieved by ID - - [ ] Scenario: Snapshot deduplication by hash - - [ ] Given two decisions with identical context content - - [ ] Then only one snapshot is stored - - [ ] And both decisions reference same snapshot - - [ ] Commit: "test(behave): add context snapshot persistence scenarios" - - [ ] **D5.7d** [Rui] Correction tracking scenarios: - - [ ] Scenario: Correction attempt persists - - [ ] Given correction attempt with all fields - - [ ] When persisted via CorrectionAttemptRepository - - [ ] Then can be retrieved by attempt_id - - [ ] And status can be updated - - [ ] Scenario: superseded_by updates correctly - - [ ] Given decision D1 - - [ ] When mark_superseded(D1.id, D2.id) called - - [ ] Then D1.superseded_by equals D2.id - - [ ] Commit: "test(behave): add correction persistence scenarios" +**Parallel Group D5: Decision Persistence [Hamza + Luis]** (depends on D1) +- [ ] **COMMIT (Owner: Hamza | Group: D5.db) - Commit message: "feat(db): add decision tables"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Add Alembic migrations for `decisions` and `context_snapshots` with indexes. + - [ ] Code [Hamza]: Add indexes for plan_id, decision_type, and superseded flags for fast tree queries. + - [ ] Docs [Hamza]: Update `docs/reference/database_schema.md` with decision tables. + - [ ] Tests (Behave) [Rui]: Add migration verification scenarios. + - [ ] Tests (Robot) [Rui]: Add DB migration smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_migration_bench.py` for migration baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(db): add decision tables"`. +- [ ] **COMMIT (Owner: Hamza | Group: D5.repo) - Commit message: "feat(repo): add decision repositories"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement DecisionRepository + ContextSnapshotRepository with tree queries and max-sequence helpers. + - [ ] Code [Hamza]: Add repository methods for superseded decision lookup and subtree retrieval. + - [ ] Docs [Hamza]: Document repository interfaces in `docs/reference/repositories.md`. + - [ ] Tests (Behave) [Rui]: Add decision persistence scenarios (create/query/superseded). + - [ ] Tests (Robot) [Rui]: Add repository integration smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_repository_bench.py` for tree query performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(repo): add decision repositories"`. +- [ ] **COMMIT (Owner: Luis | Group: D5.di) - Commit message: "feat(di): wire decision services"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Wire decision repositories + services into DI and CLI. + - [ ] Docs [Luis]: Update DI docs for decision wiring. + - [ ] Tests (Behave) [Rui]: Add DI wiring scenarios for decision commands. + - [ ] Tests (Robot) [Rui]: Add CLI smoke test using persisted decisions. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_di_bench.py` for DI resolution overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(di): wire decision services"`. +- [ ] **COMMIT (Owner: Rui | Group: D5.tests) - Commit message: "test(persistence): add decision persistence suites"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Tests (Behave) [Rui]: Add `features/decision_persistence.feature` scenarios. + - [ ] Tests (Robot) [Rui]: Add `robot/decision_persistence.robot` E2E coverage. + - [ ] Docs [Rui]: Update `docs/development/testing.md` with decision suites. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/decision_persistence_bench.py` for DB persistence throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Rui]: `git commit -m "test(persistence): add decision persistence suites"`. -**M4 SUCCESS CRITERIA** (Day 21): -- [ ] Decisions are recorded during Strategize phase with full context -- [ ] Decision tree can be viewed via `agents [--data-dir PATH] [--config-path PATH] plan tree` command -- [ ] `agents [--data-dir PATH] [--config-path PATH] plan explain ` shows full decision details -- [ ] Correction with `--mode=revert` rolls back and re-executes from decision point -- [ ] Correction with `--mode=append` creates fix subplan without modifying history -- [ ] Decisions persist to database and survive restart -- [ ] Context snapshots stored and retrievable for replay +**Parallel Group DOD: Definition of Done + Invariants [Luis + Jeff]** (depends on D2/D4) +- [ ] **COMMIT (Owner: Luis | Group: DOD.dod) - Commit message: "feat(dod): enforce definition-of-done gating"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Evaluate `definition_of_done` before apply; block apply with clear error if unmet. + - [ ] Code [Luis]: Ensure DoD templating uses plan arguments and preserves template in plan metadata. + - [ ] Docs [Luis]: Add `docs/reference/definition_of_done.md` with examples. + - [ ] Tests (Behave) [Rui]: Add DoD pass/fail scenarios. + - [ ] Tests (Robot) [Rui]: Add DoD integration smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/dod_evaluation_bench.py` for evaluation overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(dod): enforce definition-of-done gating"`. +- [ ] **COMMIT (Owner: Jeff | Group: DOD.invariants) - Commit message: "feat(invariant): add invariant models and enforcement"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Add invariant models, merge order (plan > project > action > global), and enforcement before strategize. + - [ ] Code [Jeff]: Add Invariant Reconciliation Actor role and record `invariant_enforced` decisions. + - [ ] Code [Jeff]: Add `agents invariant add/list/remove` CLI with scope flags. + - [ ] Docs [Jeff]: Add `docs/reference/invariants.md` and update CLI reference. + - [ ] Tests (Behave) [Rui]: Add invariant merge + violation scenarios. + - [ ] Tests (Robot) [Rui]: Add invariant CLI integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/invariant_merge_bench.py` for merge overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(invariant): add invariant models and enforcement"`. --- @@ -6596,1224 +3711,74 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation **Target: Milestone M5 (+25 days)** -**CRITICAL FOR 30-DAY GOAL**: Subplans enable large project handling (e.g., converting Firefox to Rust uses hierarchical decomposition into thousands of subplans) +**Parallel Group E1: Subplan Domain [Luis + Rui]** +- [ ] **COMMIT (Owner: Luis | Group: E1.domain) - Commit message: "feat(domain): add subplan config and status models"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add `ExecutionMode`, `MergeStrategy`, `SubplanConfig`, `SubplanStatus`, and `SubplanAttempt` models. + - [ ] Code [Luis]: Extend `Plan` with parent/root IDs, subplan statuses, and helpers (`is_subplan`, `has_subplans`). + - [ ] Docs [Luis]: Add `docs/reference/subplan_model.md`. + - [ ] Tests (Behave) [Rui]: Add `features/subplan_model.feature` scenarios. + - [ ] Tests (Robot) [Rui]: Add `robot/subplan_model.robot` smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/subplan_model_bench.py` for model validation. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(domain): add subplan config and status models"`. -- [ ] **Stage E1: Subplan Model** (Day 12) **[Luis]** - - **SEQUENTIAL ORDER**: E1.1 (Enums) → E1.2 (SubplanConfig) → E1.3 (Plan extension) → E1.4 (SubplanStatus) → E1.5 (Failure rules) → E1.6 (Tests) - - - [ ] **E1.1** [Luis] Define execution enums in `src/cleveragents/domain/models/core/plan.py`: - - [ ] **E1.1a** [Luis] Define `ExecutionMode` enum: - ```python - class ExecutionMode(str, Enum): - """How subplans should be executed.""" - SEQUENTIAL = "sequential" # One after another, ordered by sequence - PARALLEL = "parallel" # All at once (up to max_parallel) - DEPENDENCY_ORDERED = "dependency_ordered" # Respect DAG dependencies - ``` - - [ ] Commit: "feat(domain): define ExecutionMode enum" - - [ ] **E1.1b** [Luis] Define `MergeStrategy` enum: - ```python - class MergeStrategy(str, Enum): - """How to merge results from parallel subplans.""" - GIT_THREE_WAY = "git_three_way" # Use git merge-file for code - SEQUENTIAL_APPLY = "sequential_apply" # Apply in completion order - FAIL_ON_CONFLICT = "fail_on_conflict" # Error if any conflicts - LAST_WINS = "last_wins" # Later changes overwrite earlier - ``` - - [ ] Commit: "feat(domain): define MergeStrategy enum" - - [ ] **E1.2** [Luis] Define `SubplanConfig` model: - - [ ] **E1.2a** [Luis] Create SubplanConfig dataclass: - ```python - class SubplanConfig(BaseModel): - """Configuration for subplan execution.""" - - execution_mode: ExecutionMode = Field( - default=ExecutionMode.SEQUENTIAL, - description="How to execute subplans" - ) - merge_strategy: MergeStrategy = Field( - default=MergeStrategy.GIT_THREE_WAY, - description="How to merge subplan results" - ) - max_parallel: int = Field( - default=5, ge=1, le=50, - description="Max concurrent subplans (for PARALLEL mode)" - ) - fail_fast: bool = Field( - default=False, - description="Stop all subplans on first failure" - ) - timeout_per_subplan_seconds: int | None = Field( - default=None, - description="Timeout for each subplan (None=no timeout)" - ) - retry_failed: bool = Field( - default=True, - description="Automatically retry failed subplans" - ) - max_retries: int = Field( - default=2, ge=0, le=5, - description="Max retry attempts per subplan" - ) - ``` - - [ ] Commit: "feat(domain): define SubplanConfig model" - - [ ] **E1.3** [Luis] Extend Plan model for subplan hierarchy: - - [ ] **E1.3a** [Luis] Add parent/root plan fields (verify exist): - ```python - # In Plan model - parent_plan_id: str | None = Field( - default=None, - description="Parent plan ID if this is a subplan" - ) - root_plan_id: str | None = Field( - default=None, - description="Root plan ID (topmost ancestor)" - ) - ``` - - [ ] Commit: "feat(domain): verify parent/root plan fields on Plan" - - [ ] **E1.3b** [Luis] Add subplan configuration field: - ```python - subplan_config: SubplanConfig | None = Field( - default=None, - description="Config for subplan execution (set on parent plans)" - ) - subplan_statuses: list["SubplanStatus"] = Field( - default_factory=list, - description="Status tracking for spawned subplans" - ) - ``` - - [ ] Commit: "feat(domain): add subplan config and status fields to Plan" - - [ ] **E1.3c** [Luis] Add computed properties: - ```python - @property - def is_subplan(self) -> bool: - """Check if this plan is a subplan (has parent).""" - return self.parent_plan_id is not None - - @property - def is_root_plan(self) -> bool: - """Check if this is the root plan.""" - return self.root_plan_id is None or self.root_plan_id == self.plan_id - - @property - def depth(self) -> int: - """Distance from root plan (0 for root).""" - # Note: This requires parent chain traversal - # For efficiency, may be cached or stored - if self.is_root_plan: - return 0 - # Computed by service layer traversing parent_plan_id chain - return -1 # Placeholder, computed externally - - @property - def has_subplans(self) -> bool: - """Check if this plan has spawned subplans.""" - return len(self.subplan_statuses) > 0 - ``` - - [ ] Commit: "feat(domain): add subplan computed properties to Plan" - - [ ] **E1.4** [Luis] Define `SubplanStatus` tracking model: - - [ ] **E1.4a** [Luis] Create SubplanStatus dataclass: - ```python - @dataclass - class SubplanStatus: - """Track status of a spawned subplan.""" - - subplan_id: str # The subplan's plan_id - action_name: str # Action used to create subplan - target_resources: list[str] # Resources subplan works on - - # Status tracking - status: ProcessingState = ProcessingState.QUEUED - started_at: datetime | None = None - completed_at: datetime | None = None - - # Results - error: str | None = None - changeset_summary: str | None = None # Brief summary of changes - files_changed: int = 0 - - # Retries - attempt_number: int = 1 - previous_attempts: list["SubplanAttempt"] = field(default_factory=list) - ``` - - [ ] Commit: "feat(domain): define SubplanStatus dataclass" - - [ ] **E1.4b** [Luis] Define SubplanAttempt for retry tracking: - ```python - @dataclass - class SubplanAttempt: - """Record of a subplan execution attempt.""" - attempt_number: int - started_at: datetime - completed_at: datetime | None - error: str | None - was_retried: bool - ``` - - [ ] Commit: "feat(domain): define SubplanAttempt dataclass" - - [ ] **E1.5** [Luis] Define subplan failure handling rules: - - [ ] **E1.5a** [Luis] Create `SubplanFailureHandler` class: - ```python - class SubplanFailureHandler: - """Handle subplan failures based on configuration.""" - - def should_stop_others( - self, - config: SubplanConfig, - failed_status: SubplanStatus - ) -> bool: - """Determine if other subplans should stop.""" - if config.fail_fast: - return True - if config.execution_mode == ExecutionMode.SEQUENTIAL: - return True # Sequential always stops on failure - return False # Parallel continues others - - def should_retry( - self, - config: SubplanConfig, - status: SubplanStatus - ) -> bool: - """Determine if failed subplan should be retried.""" - if not config.retry_failed: - return False - if status.attempt_number > config.max_retries: - return False - # Don't retry on certain errors - if status.error and "ValidationError" in status.error: - return True # Validation failures can be retried - if status.error and "TimeoutError" in status.error: - return True # Timeouts can be retried - return False - ``` - - [ ] Commit: "feat(domain): define SubplanFailureHandler" - - [ ] **E1.5b** [Luis] Add failure state constants: - ```python - # Error = application/system bug, likely not recoverable - # Failure = task couldn't complete (tests fail, validation fail), may be retryable - - RETRIABLE_FAILURES = { - "ValidationError", - "TimeoutError", - "TemporaryResourceError", - "MergeConflictError" # May succeed with different merge strategy - } - - NON_RETRIABLE_ERRORS = { - "ConfigurationError", - "AuthenticationError", - "MissingResourceError", - "CircularDependencyError" - } - ``` - - [ ] Commit: "feat(domain): define retriable vs non-retriable failures" - - [ ] **E1.6** [Rui] Write Behave tests for subplan model: - - [ ] **E1.6a** [Rui] Plan hierarchy scenarios: - - [ ] Scenario: Plan with parent_plan_id has is_subplan=True - - [ ] Given Plan with parent_plan_id set - - [ ] Then is_subplan returns True - - [ ] And is_root_plan returns False - - [ ] Scenario: Root plan has is_subplan=False and is_root_plan=True - - [ ] Scenario: SubplanConfig validates max_parallel bounds - - [ ] Commit: "test(behave): add plan hierarchy scenarios" - - [ ] **E1.6b** [Rui] Execution mode scenarios: - - [ ] Scenario: ExecutionMode enum has all required values - - [ ] Scenario: MergeStrategy enum has all required values - - [ ] Scenario: SubplanConfig defaults are applied - - [ ] Commit: "test(behave): add execution mode scenarios" - - [ ] **E1.6c** [Rui] SubplanStatus scenarios: - - [ ] Scenario: SubplanStatus tracks state correctly - - [ ] Scenario: SubplanAttempt records retry history - - [ ] Scenario: Failure handler respects fail_fast setting - - [ ] Commit: "test(behave): add SubplanStatus scenarios" +**Parallel Group E2: Subplan Spawning [Jeff + Aditya]** (depends on D2 + E1) +- [ ] **COMMIT (Owner: Jeff | Group: E2.service) - Commit message: "feat(service): add subplan service and spawn workflow"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Implement `SubplanService` with `spawn_subplan`, `spawn_batch`, tree queries, bounded context builder. + - [ ] Code [Jeff]: Link SUBPLAN_SPAWN decisions to created subplans and status tracking. + - [ ] Docs [Jeff]: Add `docs/reference/subplan_service.md`. + - [ ] Tests (Behave) [Rui]: Add subplan spawn scenarios. + - [ ] Tests (Robot) [Rui]: Add subplan spawn integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/subplan_spawn_bench.py` for spawn throughput. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(service): add subplan service and spawn workflow"`. +- [ ] **COMMIT (Owner: Aditya | Group: E2.actor) - Commit message: "feat(actor): add plan_subplan tool and decision emission"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Aditya]: Add `plan_subplan` tool to strategy actors and emit SUBPLAN_SPAWN decisions. + - [ ] Docs [Aditya]: Update actor YAML examples for subplan emission. + - [ ] Tests (Behave) [Rui]: Add scenarios for subplan decision emission. + - [ ] Tests (Robot) [Rui]: Add actor tool integration smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/subplan_actor_tool_bench.py` for tool invocation overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Aditya]: `git commit -m "feat(actor): add plan_subplan tool and decision emission"`. -- [ ] **Stage E2: Subplan Spawning** (Day 12-13) **[Jeff + Luis]** - - **SEQUENTIAL ORDER**: E2.1 (Service scaffold) → E2.2 (spawn_subplan) → E2.3 (spawn_batch) → E2.4 (queries) → E2.5 (strategy actor) → E2.6 (execute phase) → E2.7 (status tracking) → E2.8 (bounded context) → E2.9 (Tests) - - - [ ] **E2.1** [Jeff] Create `SubplanService` scaffold in `src/cleveragents/application/services/subplan_service.py`: - - [ ] **E2.1a** [Jeff] Define service class: - ```python - class SubplanService: - """Service for spawning and managing subplans.""" - - def __init__( - self, - plan_service: PlanLifecycleService, - decision_service: DecisionService, - plan_repo: LifecyclePlanRepository, - context_builder: ContextBuilder - ): - self._plan_service = plan_service - self._decision_service = decision_service - self._plan_repo = plan_repo - self._context_builder = context_builder - ``` - - [ ] Commit: "feat(service): add SubplanService scaffold" - - [ ] **E2.2** [Jeff] Implement `spawn_subplan()` method: - - [ ] **E2.2a** [Jeff] Core implementation: - ```python - def spawn_subplan( - self, - parent_plan: Plan, - decision: Decision, - action_name: str, - target_resources: list[str] | None = None, - arguments: dict | None = None - ) -> Plan: - """Spawn a single subplan from a parent plan.""" - # Validate parent is not already a deep subplan - if parent_plan.depth >= 10: # Max nesting depth - raise MaxSubplanDepthError(f"Cannot spawn subplan at depth {parent_plan.depth + 1}") - - # Create subplan via plan service - subplan = self._plan_service.use_action( - action_name=action_name, - project_ids=target_resources or parent_plan.project_ids, - arguments=arguments or {}, - parent_plan_id=parent_plan.plan_id, - root_plan_id=parent_plan.root_plan_id or parent_plan.plan_id, - automation_level=parent_plan.automation_level - ) - - # Link to decision - self._decision_service.add_downstream_plan(decision.decision_id, subplan.plan_id) - - # Create status tracking - status = SubplanStatus( - subplan_id=subplan.plan_id, - action_name=action_name, - target_resources=target_resources or [] - ) - - # Update parent with new subplan status - self._update_parent_subplan_status(parent_plan.plan_id, status) - - logger.info(f"Spawned subplan {subplan.plan_id} from parent {parent_plan.plan_id}") - return subplan - ``` - - [ ] Commit: "feat(service): implement spawn_subplan()" - - [ ] **E2.2b** [Jeff] Add bounded context calculation: - ```python - def _build_bounded_context( - self, - parent_plan: Plan, - decision: Decision, - target_resources: list[str] - ) -> BoundedContext: - """Build bounded context for subplan from decision scope.""" - return self._context_builder.build_from_decision( - parent_context=parent_plan.context, - decision=decision, - resource_filter=target_resources - ) - ``` - - [ ] Commit: "feat(service): add bounded context for subplans" - - [ ] **E2.3** [Jeff] Implement `spawn_batch()` method: - - [ ] **E2.3a** [Jeff] Batch spawning implementation: - ```python - def spawn_batch( - self, - parent_plan: Plan, - decisions: list[Decision], - execution_mode: ExecutionMode = ExecutionMode.PARALLEL - ) -> list[Plan]: - """Spawn multiple subplans at once.""" - subplans = [] - - for decision in decisions: - if decision.decision_type != DecisionType.SUBPLAN_SPAWN: - continue - - # Extract spawn parameters from decision - spawn_params = self._extract_spawn_params(decision) - - subplan = self.spawn_subplan( - parent_plan=parent_plan, - decision=decision, - action_name=spawn_params["action_name"], - target_resources=spawn_params.get("target_resources"), - arguments=spawn_params.get("arguments") - ) - subplans.append(subplan) - - # Update parent's execution mode - self._update_parent_execution_mode(parent_plan.plan_id, execution_mode) - - logger.info(f"Spawned {len(subplans)} subplans from parent {parent_plan.plan_id}") - return subplans - - def _extract_spawn_params(self, decision: Decision) -> dict: - """Extract spawn parameters from SUBPLAN_SPAWN decision.""" - # Parse chosen_option which contains action name and params - # Format: "action_name:arg1=val1,arg2=val2" - return { - "action_name": decision.chosen_option.split(":")[0], - "target_resources": decision.context_snapshot.relevant_resources, - "arguments": {} # Parsed from decision metadata - } - ``` - - [ ] Commit: "feat(service): implement spawn_batch()" - - [ ] **E2.4** [Jeff] Implement query methods: - - [ ] **E2.4a** [Jeff] Implement `get_subplans()`: - ```python - def get_subplans(self, parent_plan_id: str) -> list[Plan]: - """Get all direct child subplans.""" - return self._plan_repo.get_children(parent_plan_id) - - def get_subplan_statuses(self, parent_plan_id: str) -> list[SubplanStatus]: - """Get status tracking for all subplans.""" - parent = self._plan_repo.get_by_id(parent_plan_id) - return parent.subplan_statuses if parent else [] - ``` - - [ ] Commit: "feat(service): implement get_subplans()" - - [ ] **E2.4b** [Jeff] Implement `get_full_tree()`: - ```python - def get_full_tree(self, root_plan_id: str) -> PlanTree: - """Get full subplan tree from root.""" - plans = self._plan_repo.get_tree(root_plan_id) - return self._build_plan_tree(plans, root_plan_id) - - def _build_plan_tree(self, plans: list[Plan], root_id: str) -> PlanTree: - """Build tree structure from flat list.""" - by_parent: dict[str, list[Plan]] = {} - root = None - - for plan in plans: - if plan.plan_id == root_id: - root = plan - elif plan.parent_plan_id: - by_parent.setdefault(plan.parent_plan_id, []).append(plan) - - def build_node(plan: Plan) -> PlanTreeNode: - children = [build_node(c) for c in by_parent.get(plan.plan_id, [])] - return PlanTreeNode(plan=plan, children=children) - - return PlanTree(root=build_node(root), total_count=len(plans)) - ``` - - [ ] Commit: "feat(service): implement get_full_tree()" - - [ ] **E2.5** [Aditya] Configure strategy actor to emit subplan_spawn decisions: - - [ ] **E2.5a** [Aditya] Add plan_subplan tool to strategy actor: - ```yaml - # In strategy_actor.yaml - tools: - - name: plan_subplan - description: | - Decompose work into a subplan for parallel or sequential execution. - Use when work can be broken into independent pieces. - parameters: - - name: action_name - type: string - description: Action to use for subplan (e.g., "local/code-fix") - - name: description - type: string - description: What the subplan should accomplish - - name: target_files - type: array - description: Files this subplan should work on - - name: execution_mode - type: string - enum: [sequential, parallel, dependency_ordered] - description: How this relates to other subplans - - name: depends_on - type: array - description: IDs of subplans this depends on (for dependency_ordered) - code: | - result = context.create_subplan_decision( - action_name=input_data["action_name"], - description=input_data["description"], - target_files=input_data.get("target_files", []), - execution_mode=input_data.get("execution_mode", "parallel"), - depends_on=input_data.get("depends_on", []) - ) - ``` - - [ ] Commit: "feat(actor): add plan_subplan tool to strategy actor" - - [ ] **E2.5b** [Aditya] Implement `context.create_subplan_decision()`: - ```python - def create_subplan_decision( - self, - action_name: str, - description: str, - target_files: list[str], - execution_mode: str = "parallel", - depends_on: list[str] | None = None - ) -> str: - """Create a SUBPLAN_SPAWN decision (not actual plan yet).""" - decision = self._decision_service.record_decision( - plan_id=self.plan_id, - decision_type=DecisionType.SUBPLAN_SPAWN, - question=f"Should we create subplan for: {description}", - chosen_option=f"{action_name}:{','.join(target_files)}", - hot_context=self._get_context_for_files(target_files), - resources=target_files, - parent_decision_id=self._current_decision_id, - rationale=description - ) - - # Store metadata for Execute phase to process - self._pending_subplans.append({ - "decision_id": decision.decision_id, - "action_name": action_name, - "target_files": target_files, - "execution_mode": execution_mode, - "depends_on": depends_on or [] - }) - - return decision.decision_id - ``` - - [ ] Commit: "feat(context): implement create_subplan_decision()" - - [ ] **E2.6** [Jeff] Execute phase processes subplan decisions: - - [ ] **E2.6a** [Jeff] Add subplan processing to execute phase: - ```python - # In PlanLifecycleService.execute_execution() - - async def _process_subplan_decisions(self, plan: Plan) -> None: - """Process SUBPLAN_SPAWN decisions after strategy.""" - # Get all pending subplan decisions - decisions = self._decision_service.get_by_plan_and_type( - plan.plan_id, DecisionType.SUBPLAN_SPAWN - ) - - if not decisions: - return - - # Group by execution mode - parallel_decisions = [] - sequential_decisions = [] - dependency_decisions = [] - - for d in decisions: - mode = self._get_execution_mode(d) - if mode == ExecutionMode.PARALLEL: - parallel_decisions.append(d) - elif mode == ExecutionMode.SEQUENTIAL: - sequential_decisions.append(d) - else: - dependency_decisions.append(d) - - # Execute in appropriate order - if parallel_decisions: - await self._execute_parallel_subplans(plan, parallel_decisions) - if sequential_decisions: - await self._execute_sequential_subplans(plan, sequential_decisions) - if dependency_decisions: - await self._execute_dependency_ordered_subplans(plan, dependency_decisions) - ``` - - [ ] Commit: "feat(service): add subplan decision processing to execute phase" - - [ ] **E2.6b** [Jeff] Implement sequential execution: - ```python - async def _execute_sequential_subplans( - self, - parent: Plan, - decisions: list[Decision] - ) -> None: - """Execute subplans one at a time in order.""" - for decision in decisions: - subplan = self._subplan_service.spawn_subplan( - parent_plan=parent, - decision=decision, - action_name=self._extract_action_name(decision) - ) - - # Execute and wait - await self._execute_subplan(subplan) - - # Check result - status = self._get_subplan_status(parent, subplan.plan_id) - if status.status == ProcessingState.ERRORED: - if parent.subplan_config.fail_fast: - raise SubplanFailedError(subplan.plan_id, status.error) - # Otherwise continue to next - ``` - - [ ] Commit: "feat(service): implement sequential subplan execution" - - [ ] **E2.7** [Luis] Implement subplan status tracking: - - [ ] **E2.7a** [Luis] Create status update mechanism: - ```python - class SubplanStatusTracker: - """Track and update subplan statuses.""" - - def __init__(self, plan_repo: LifecyclePlanRepository): - self._plan_repo = plan_repo - self._listeners: dict[str, list[Callable]] = {} - - def update_status( - self, - parent_plan_id: str, - subplan_id: str, - new_status: ProcessingState, - error: str | None = None - ) -> None: - """Update status of a subplan.""" - parent = self._plan_repo.get_by_id(parent_plan_id) - if not parent: - return - - # Find and update status - for status in parent.subplan_statuses: - if status.subplan_id == subplan_id: - status.status = new_status - if new_status == ProcessingState.PROCESSING: - status.started_at = datetime.utcnow() - elif new_status in (ProcessingState.COMPLETE, ProcessingState.ERRORED): - status.completed_at = datetime.utcnow() - if error: - status.error = error - break - - # Persist - self._plan_repo.update(parent) - - # Notify listeners - self._notify_listeners(parent_plan_id, subplan_id, new_status) - - def subscribe(self, parent_plan_id: str, callback: Callable) -> None: - """Subscribe to status updates for a parent plan.""" - self._listeners.setdefault(parent_plan_id, []).append(callback) - ``` - - [ ] Commit: "feat(service): implement SubplanStatusTracker" - - [ ] **E2.7b** [Luis] Determine parent state from subplan states: - ```python - def compute_parent_state(self, parent: Plan) -> ProcessingState: - """Compute parent state based on subplan states.""" - statuses = parent.subplan_statuses - - if not statuses: - return parent.state - - # Count states - errored = sum(1 for s in statuses if s.status == ProcessingState.ERRORED) - complete = sum(1 for s in statuses if s.status == ProcessingState.COMPLETE) - processing = sum(1 for s in statuses if s.status == ProcessingState.PROCESSING) - - config = parent.subplan_config or SubplanConfig() - - # Determine parent state - if processing > 0: - return ProcessingState.PROCESSING - - if errored > 0: - if config.execution_mode == ExecutionMode.PARALLEL: - # Parallel: error only if ALL failed - if errored == len(statuses): - return ProcessingState.ERRORED - else: - # Sequential: error on first failure - return ProcessingState.ERRORED - - if complete == len(statuses): - return ProcessingState.COMPLETE - - return ProcessingState.QUEUED # Some still pending - ``` - - [ ] Commit: "feat(service): implement parent state computation" - - [ ] **E2.8** [Luis] Implement bounded context for subplans: - - [ ] **E2.8a** [Luis] Create `ContextBuilder` for bounded contexts: - ```python - class ContextBuilder: - """Build bounded contexts for subplans.""" - - def build_from_decision( - self, - parent_context: PlanContext, - decision: Decision, - resource_filter: list[str] - ) -> BoundedContext: - """Build context bounded to decision scope.""" - # Filter files to only those relevant - relevant_files = self._filter_files( - parent_context.files, - resource_filter - ) - - # Include decision chain for reference - decision_chain = self._get_decision_chain(decision) - - return BoundedContext( - files=relevant_files, - decision_chain=decision_chain, - parent_context_ref=parent_context.context_id, - boundary=resource_filter - ) - - def _filter_files( - self, - files: dict[str, FileContent], - patterns: list[str] - ) -> dict[str, FileContent]: - """Filter files to match patterns.""" - import fnmatch - result = {} - for path, content in files.items(): - if any(fnmatch.fnmatch(path, p) for p in patterns): - result[path] = content - return result - ``` - - [ ] Commit: "feat(context): implement ContextBuilder for bounded contexts" - - [ ] **E2.9** [Rui] Write integration tests for subplan spawning: - - [ ] **E2.9a** [Rui] Spawn scenarios: - - [ ] Scenario: SUBPLAN_SPAWN decision creates child plan - - [ ] Given strategy produces SUBPLAN_SPAWN decision - - [ ] When execute phase processes decisions - - [ ] Then child plan is created with correct parent_plan_id - - [ ] And decision.downstream_plan_ids contains subplan ID - - [ ] Scenario: spawn_batch creates multiple subplans - - [ ] Given 3 SUBPLAN_SPAWN decisions - - [ ] When spawn_batch is called - - [ ] Then 3 subplans are created - - [ ] Commit: "test(behave): add subplan spawn scenarios" - - [ ] **E2.9b** [Rui] Execution order scenarios: - - [ ] Scenario: Sequential subplans execute in order - - [ ] Given 3 sequential subplans S1, S2, S3 - - [ ] When executed - - [ ] Then S1 completes before S2 starts - - [ ] And S2 completes before S3 starts - - [ ] Scenario: Parallel subplans execute concurrently - - [ ] Given 3 parallel subplans - - [ ] When executed with max_parallel=3 - - [ ] Then all 3 start at approximately same time - - [ ] Commit: "test(behave): add execution order scenarios" - - [ ] **E2.9c** [Rui] Failure scenarios: - - [ ] Scenario: Failed sequential subplan stops processing - - [ ] Given sequential subplans S1, S2, S3 - - [ ] When S2 fails - - [ ] Then S3 is not started - - [ ] And parent enters ERRORED state - - [ ] Scenario: Failed parallel subplan allows others to finish - - [ ] Given parallel subplans S1, S2, S3 - - [ ] And fail_fast=False - - [ ] When S2 fails - - [ ] Then S1 and S3 continue to completion - - [ ] Commit: "test(behave): add subplan failure scenarios" +**Parallel Group E3: Parallel Execution [Luis + Jeff]** (depends on E1/E2) +- [ ] **COMMIT (Owner: Luis | Group: E3.exec) - Commit message: "feat(service): add subplan scheduler and execution"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add subplan scheduler with `max_parallel`, dependency ordering, and fail-fast handling. + - [ ] Code [Luis]: Track status updates for subplans and propagate to parent plan. + - [ ] Docs [Luis]: Add `docs/reference/subplan_execution.md`. + - [ ] Tests (Behave) [Rui]: Add parallel + dependency execution scenarios. + - [ ] Tests (Robot) [Rui]: Add parallel execution integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/subplan_scheduler_bench.py` for scheduler overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(service): add subplan scheduler and execution"`. -- [ ] **Stage E3: Parallel Execution** (Day 19) **[Jeff + Luis]** - - **SEQUENTIAL ORDER**: E3.1 (AsyncExecutor) → E3.2 (Semaphore) → E3.3 (Timeouts) → E3.4 (DAG) → E3.5 (Isolation) → E3.6 (Tests) - - - [ ] **E3.1** [Jeff] Implement async subplan executor: - - [ ] **E3.1a** [Jeff] Create `AsyncSubplanExecutor` class: - ```python - class AsyncSubplanExecutor: - """Execute subplans asynchronously with concurrency control.""" - - def __init__( - self, - plan_service: PlanLifecycleService, - status_tracker: SubplanStatusTracker - ): - self._plan_service = plan_service - self._status_tracker = status_tracker - - async def execute_parallel( - self, - parent: Plan, - subplans: list[Plan], - config: SubplanConfig - ) -> list[SubplanResult]: - """Execute subplans in parallel with concurrency limit.""" - semaphore = asyncio.Semaphore(config.max_parallel) - - async def execute_with_limit(subplan: Plan) -> SubplanResult: - async with semaphore: - return await self._execute_single(subplan, config) - - tasks = [execute_with_limit(sp) for sp in subplans] - results = await asyncio.gather(*tasks, return_exceptions=True) - - return self._process_results(results, subplans) - ``` - - [ ] Commit: "feat(executor): add AsyncSubplanExecutor" - - [ ] **E3.1b** [Jeff] Implement single subplan execution: - ```python - async def _execute_single( - self, - subplan: Plan, - config: SubplanConfig - ) -> SubplanResult: - """Execute a single subplan with timeout.""" - try: - # Apply timeout if configured - if config.timeout_per_subplan_seconds: - result = await asyncio.wait_for( - self._run_subplan(subplan), - timeout=config.timeout_per_subplan_seconds - ) - else: - result = await self._run_subplan(subplan) - - return SubplanResult( - subplan_id=subplan.plan_id, - success=True, - changeset=result.changeset - ) - - except asyncio.TimeoutError: - self._status_tracker.update_status( - subplan.parent_plan_id, - subplan.plan_id, - ProcessingState.ERRORED, - error="Timeout exceeded" - ) - return SubplanResult( - subplan_id=subplan.plan_id, - success=False, - error="TimeoutError" - ) - - except Exception as e: - self._status_tracker.update_status( - subplan.parent_plan_id, - subplan.plan_id, - ProcessingState.ERRORED, - error=str(e) - ) - return SubplanResult( - subplan_id=subplan.plan_id, - success=False, - error=str(e) - ) - ``` - - [ ] Commit: "feat(executor): implement single subplan execution with timeout" - - [ ] **E3.2** [Jeff] Implement dependency-ordered execution: - - [ ] **E3.2a** [Jeff] Build and validate dependency DAG: - ```python - def build_dependency_dag( - self, - decisions: list[Decision] - ) -> DependencyGraph: - """Build DAG from subplan decisions with depends_on.""" - graph = DependencyGraph() - - for decision in decisions: - graph.add_node(decision.decision_id) - depends_on = self._get_depends_on(decision) - for dep_id in depends_on: - graph.add_edge(dep_id, decision.decision_id) - - # Validate no cycles - if graph.has_cycle(): - cycle = graph.find_cycle() - raise CircularDependencyError(f"Cycle detected: {' -> '.join(cycle)}") - - return graph - ``` - - [ ] Commit: "feat(executor): implement dependency DAG building" - - [ ] **E3.2b** [Jeff] Execute in topological order: - ```python - async def execute_dependency_ordered( - self, - parent: Plan, - decisions: list[Decision], - config: SubplanConfig - ) -> list[SubplanResult]: - """Execute subplans respecting dependency order.""" - dag = self.build_dependency_dag(decisions) - execution_order = dag.topological_sort() - - results = [] - completed: set[str] = set() - - # Process in waves - each wave contains independent nodes - while execution_order: - # Find all nodes whose dependencies are satisfied - ready = [ - node for node in execution_order - if all(dep in completed for dep in dag.get_dependencies(node)) - ] - - if not ready: - break # Stuck - shouldn't happen with valid DAG - - # Execute ready nodes in parallel - ready_decisions = [d for d in decisions if d.decision_id in ready] - subplans = [self._spawn_subplan(parent, d) for d in ready_decisions] - - wave_results = await self.execute_parallel(parent, subplans, config) - results.extend(wave_results) - - # Mark completed - for r in wave_results: - if r.success: - completed.add(r.decision_id) - - # Remove from order - execution_order = [n for n in execution_order if n not in ready] - - return results - ``` - - [ ] Commit: "feat(executor): implement dependency-ordered execution" - - [ ] **E3.3** [Luis] Implement subplan isolation: - - [ ] **E3.3a** [Luis] Ensure separate sandboxes: - ```python - def ensure_isolated_sandbox( - self, - subplan: Plan, - resource_service: ResourceService - ) -> None: - """Ensure subplan has its own isolated sandbox.""" - for resource_id in subplan.project_ids: - resource = self._get_resource(resource_id) - # Each subplan gets unique sandbox for same resource - sandbox = resource_service.access_resource( - plan_id=subplan.plan_id, # Use subplan ID, not parent - resource=resource, - mode=AccessMode.WRITE - ) - # Sandbox is isolated by plan_id - ``` - - [ ] Commit: "feat(executor): ensure isolated sandboxes for subplans" - - [ ] **E3.3b** [Luis] Prevent cross-subplan visibility: - ```python - def validate_isolation( - self, - subplan: Plan, - other_subplans: list[Plan] - ) -> None: - """Verify subplan cannot access other subplans' state.""" - subplan_sandbox = self._get_sandbox(subplan.plan_id) - - for other in other_subplans: - if other.plan_id == subplan.plan_id: - continue - other_sandbox = self._get_sandbox(other.plan_id) - - # Verify different paths - if subplan_sandbox.sandbox_path == other_sandbox.sandbox_path: - raise IsolationViolationError( - f"Subplans {subplan.plan_id} and {other.plan_id} share sandbox" - ) - ``` - - [ ] Commit: "feat(executor): add isolation validation" - - [ ] **E3.4** [Rui] Write tests for parallel execution: - - [ ] **E3.4a** [Rui] Concurrency scenarios: - - [ ] Scenario: 10 independent subplans run with max_parallel=5 - - [ ] Given 10 subplans with no dependencies - - [ ] And max_parallel=5 - - [ ] When executed in parallel - - [ ] Then at most 5 run concurrently at any time - - [ ] And all 10 complete successfully - - [ ] Commit: "test(behave): add concurrency limit scenarios" - - [ ] **E3.4b** [Rui] Dependency scenarios: - - [ ] Scenario: Dependency chain executes in correct order - - [ ] Given subplans A -> B -> C (B depends on A, C depends on B) - - [ ] When executed with dependency ordering - - [ ] Then A completes before B starts - - [ ] And B completes before C starts - - [ ] Scenario: Diamond dependency executes correctly - - [ ] Given A -> B, A -> C, B -> D, C -> D - - [ ] When executed - - [ ] Then A runs first - - [ ] Then B and C run in parallel - - [ ] Then D runs last - - [ ] Commit: "test(behave): add dependency ordering scenarios" - - [ ] **E3.4c** [Rui] Timeout scenarios: - - [ ] Scenario: Subplan timeout triggers failure - - [ ] Given subplan with timeout_per_subplan_seconds=10 - - [ ] When subplan takes 15 seconds - - [ ] Then subplan is marked ERRORED with TimeoutError - - [ ] Commit: "test(behave): add timeout scenarios" +**Parallel Group E4: Result Merging [Jeff + Luis]** (depends on E3) +- [ ] **COMMIT (Owner: Jeff | Group: E4.merge) - Commit message: "feat(merge): add subplan merge strategies"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Add three-way merge strategy for file changes and conflict markers. + - [ ] Code [Luis]: Add sequential merge and JSON merge strategies; expose merge result artifacts. + - [ ] Docs [Jeff]: Add `docs/reference/subplan_merge.md`. + - [ ] Tests (Behave) [Rui]: Add merge + conflict scenarios. + - [ ] Tests (Robot) [Rui]: Add merge integration tests for multi-subplan plans. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/subplan_merge_bench.py` for merge performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(merge): add subplan merge strategies"`. -- [ ] **Stage E4: Result Merging** (Day 20) **[Jeff + Luis]** - - **SEQUENTIAL ORDER**: E4.1 (MergeService) → E4.2 (MergeResult) → E4.3 (ThreeWayMerge) → E4.4 (SequentialMerge) → E4.5 (Validation) → E4.6 (Tests) - - - [ ] **E4.1** [Jeff] Create `MergeService` in `src/cleveragents/application/services/merge_service.py`: - - [ ] **E4.1a** [Jeff] Define service class: - ```python - class MergeService: - """Service for merging subplan results.""" - - def __init__( - self, - sandbox_manager: SandboxManager, - validation_service: ValidationService - ): - self._sandbox_manager = sandbox_manager - self._validation_service = validation_service - ``` - - [ ] Commit: "feat(service): add MergeService scaffold" - - [ ] **E4.1b** [Jeff] Implement `merge_subplan_results()`: - ```python - def merge_subplan_results( - self, - parent: Plan, - subplans: list[Plan], - strategy: MergeStrategy = MergeStrategy.GIT_THREE_WAY - ) -> MergeResult: - """Merge changesets from all completed subplans.""" - # Collect changesets - changesets = [sp.changeset for sp in subplans if sp.changeset] - - # Group changes by file - changes_by_file: dict[str, list[Change]] = {} - for cs in changesets: - for change in cs.changes: - changes_by_file.setdefault(change.path, []).append(change) - - # Detect and handle conflicts - merged_changes = [] - conflicts = [] - - for path, changes in changes_by_file.items(): - if len(changes) == 1: - merged_changes.append(changes[0]) - else: - # Multiple subplans modified same file - result = self._merge_file_changes(path, changes, strategy) - if result.has_conflict: - conflicts.append(result) - merged_changes.append(result.merged_change) - - return MergeResult( - merged_changeset=ChangeSet(changes=merged_changes), - conflicts=conflicts, - source_subplan_ids=[sp.plan_id for sp in subplans] - ) - ``` - - [ ] Commit: "feat(service): implement merge_subplan_results()" - - [ ] **E4.2** [Jeff] Define merge result types: - - [ ] **E4.2a** [Jeff] Define MergeResult dataclass: - ```python - @dataclass - class MergeResult: - """Result of merging subplan changesets.""" - merged_changeset: ChangeSet - conflicts: list["FileConflict"] - source_subplan_ids: list[str] - - @property - def has_conflicts(self) -> bool: - return len(self.conflicts) > 0 - - @property - def conflict_count(self) -> int: - return len(self.conflicts) - - @dataclass - class FileConflict: - """Conflict in a single file.""" - path: str - conflict_regions: list["ConflictRegion"] - subplan_ids: list[str] # Which subplans caused conflict - merged_content_with_markers: str - - @dataclass - class ConflictRegion: - """Region of conflict within a file.""" - start_line: int - end_line: int - ours_content: str # From first subplan - theirs_content: str # From second subplan - ``` - - [ ] Commit: "feat(domain): define merge result types" - - [ ] **E4.3** [Jeff] Implement git-style three-way merge: - - [ ] **E4.3a** [Jeff] Implement `_merge_file_changes()`: - ```python - def _merge_file_changes( - self, - path: str, - changes: list[Change], - strategy: MergeStrategy - ) -> FileMergeResult: - """Merge multiple changes to same file.""" - if strategy == MergeStrategy.GIT_THREE_WAY: - return self._git_three_way_merge(path, changes) - elif strategy == MergeStrategy.SEQUENTIAL_APPLY: - return self._sequential_merge(path, changes) - elif strategy == MergeStrategy.LAST_WINS: - return FileMergeResult( - merged_change=changes[-1], - has_conflict=False - ) - else: - raise ValueError(f"Unknown strategy: {strategy}") - ``` - - [ ] Commit: "feat(service): implement merge strategy dispatch" - - [ ] **E4.3b** [Jeff] Implement `_git_three_way_merge()`: - ```python - def _git_three_way_merge( - self, - path: str, - changes: list[Change] - ) -> FileMergeResult: - """Perform git-style three-way merge.""" - import subprocess - import tempfile - - # Get base (original) content - base_content = self._get_base_content(path) - - # For now, handle 2 changes; extend for more - if len(changes) != 2: - # Fall back to sequential for >2 changes - return self._sequential_merge(path, changes) - - ours = changes[0].content or base_content - theirs = changes[1].content or base_content - - # Write to temp files - with tempfile.NamedTemporaryFile(mode='w', suffix='.base', delete=False) as f: - f.write(base_content) - base_path = f.name - with tempfile.NamedTemporaryFile(mode='w', suffix='.ours', delete=False) as f: - f.write(ours) - ours_path = f.name - with tempfile.NamedTemporaryFile(mode='w', suffix='.theirs', delete=False) as f: - f.write(theirs) - theirs_path = f.name - - try: - # Run git merge-file - result = subprocess.run( - ['git', 'merge-file', '-p', ours_path, base_path, theirs_path], - capture_output=True, - text=True - ) - - merged_content = result.stdout - has_conflict = result.returncode != 0 - - # Parse conflict markers if present - conflicts = [] - if has_conflict: - conflicts = self._parse_conflict_markers(merged_content) - - merged_change = Change( - path=path, - operation=OperationType.MODIFY, - content=merged_content - ) - - return FileMergeResult( - merged_change=merged_change, - has_conflict=has_conflict, - conflict_regions=conflicts - ) - finally: - # Cleanup temp files - for p in [base_path, ours_path, theirs_path]: - os.unlink(p) - ``` - - [ ] Commit: "feat(service): implement git three-way merge" - - [ ] **E4.4** [Luis] Implement sequential merge: - - [ ] **E4.4a** [Luis] Apply changes in order: - ```python - def _sequential_merge( - self, - path: str, - changes: list[Change] - ) -> FileMergeResult: - """Apply changes sequentially in completion order.""" - current_content = self._get_base_content(path) - - for change in changes: - if change.edits: - # Apply edits - current_content = self._apply_edits(current_content, change.edits) - elif change.content: - # Full replacement - current_content = change.content - - return FileMergeResult( - merged_change=Change( - path=path, - operation=OperationType.MODIFY, - content=current_content - ), - has_conflict=False - ) - ``` - - [ ] Commit: "feat(service): implement sequential merge" - - [ ] **E4.5** [Luis] Implement post-merge validation: - - [ ] **E4.5a** [Luis] Validate merged state: - ```python - async def validate_merged_result( - self, - parent: Plan, - merge_result: MergeResult - ) -> ValidationResult: - """Run validation on merged changes.""" - # Apply merged changes to temporary sandbox - temp_sandbox = self._sandbox_manager.create_temp_sandbox( - parent.plan_id, suffix="_merge_validation" - ) - - try: - # Apply merged changes - for change in merge_result.merged_changeset.changes: - self._apply_change_to_sandbox(temp_sandbox, change) - - # Run validation - result = await self._validation_service.validate_changeset( - merge_result.merged_changeset, - parent.project - ) - - if not result.passed: - logger.warning( - f"Post-merge validation failed: {result.errors}" - ) - - return result - finally: - temp_sandbox.cleanup() - ``` - - [ ] Commit: "feat(service): implement post-merge validation" - - [ ] **E4.5b** [Luis] Handle validation failures: - ```python - async def handle_validation_failure( - self, - parent: Plan, - merge_result: MergeResult, - validation_result: ValidationResult - ) -> MergeRecoveryAction: - """Determine recovery action for failed validation.""" - # Options: - # 1. Retry with different merge strategy - # 2. Escalate to user - # 3. Fall back to sequential execution - - if parent.subplan_config.merge_strategy == MergeStrategy.GIT_THREE_WAY: - # Try sequential as fallback - return MergeRecoveryAction.RETRY_SEQUENTIAL - - # Escalate to user - return MergeRecoveryAction.ESCALATE_TO_USER - ``` - - [ ] Commit: "feat(service): implement validation failure handling" - - [ ] **E4.6** [Rui] Write tests for result merging: - - [ ] **E4.6a** [Rui] Clean merge scenarios: - - [ ] Scenario: Two subplans modifying different files merge cleanly - - [ ] Given subplan A modifies file1.py - - [ ] And subplan B modifies file2.py - - [ ] When merged - - [ ] Then both changes are in merged_changeset - - [ ] And has_conflicts is False - - [ ] Scenario: Same file different lines merges cleanly - - [ ] Given subplan A changes line 10 of file.py - - [ ] And subplan B changes line 50 of file.py - - [ ] When merged with GIT_THREE_WAY - - [ ] Then both changes are preserved - - [ ] And has_conflicts is False - - [ ] Commit: "test(behave): add clean merge scenarios" - - [ ] **E4.6b** [Rui] Conflict scenarios: - - [ ] Scenario: Same lines creates conflict markers - - [ ] Given subplan A changes line 10 to "version A" - - [ ] And subplan B changes line 10 to "version B" - - [ ] When merged with GIT_THREE_WAY - - [ ] Then has_conflicts is True - - [ ] And merged content contains conflict markers - - [ ] Scenario: LAST_WINS strategy has no conflicts - - [ ] Given conflicting changes - - [ ] When merged with LAST_WINS - - [ ] Then has_conflicts is False - - [ ] And later change overwrites earlier - - [ ] Commit: "test(behave): add conflict merge scenarios" - - [ ] **E4.6c** [Rui] Validation scenarios: - - [ ] Scenario: Post-merge validation catches broken code - - [ ] Given merged code with syntax error - - [ ] When post-merge validation runs - - [ ] Then validation fails - - [ ] And recovery action is suggested - - [ ] Commit: "test(behave): add post-merge validation scenarios" +**Parallel Group E5: Multi-Project Plans [Hamza + Luis]** (depends on E2/E4) +- [ ] **COMMIT (Owner: Hamza | Group: E5.multi) - Commit message: "feat(plan): add multi-project subplan support"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Allow plans to target multiple projects with separate resource link contexts. + - [ ] Code [Luis]: Ensure sandbox isolation and cross-project dependency resolution. + - [ ] Docs [Hamza]: Add `docs/reference/multi_project_plans.md`. + - [ ] Tests (Behave) [Rui]: Add multi-project subplan scenarios. + - [ ] Tests (Robot) [Rui]: Add multi-project integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/multi_project_bench.py` for multi-project overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(plan): add multi-project subplan support"`. - [ ] **Stage E5: Multi-Project Plans** (Day 25) **[Hamza]** @@ -8006,81 +3971,73 @@ TESTING [Rui - Continuous]: Write tests BEFORE implementation --- -### Section 8: Server Connectivity [DEFERRED - Beyond Day 30] +**Parallel Group G1: Large-Project Decomposition [Jeff + Luis]** +- [ ] **COMMIT (Owner: Jeff | Group: G1.decompose) - Commit message: "feat(plan): add large-project decomposition and dependency closure"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Jeff]: Add hierarchical decomposition with 4+ levels and bounded context per subplan. + - [ ] Code [Luis]: Add dependency closure computation for large graphs and DAG execution ordering. + - [ ] Docs [Jeff]: Add `docs/reference/large_project_decomposition.md`. + - [ ] Tests (Behave) [Rui]: Add deep hierarchy + dependency closure scenarios. + - [ ] Tests (Robot) [Rui]: Add large-project decomposition integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/large_project_decompose_bench.py` for decomposition runtime. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "feat(plan): add large-project decomposition and dependency closure"`. -**Target: Post-30-day work (NOT part of initial 30-day timeline)** +**Parallel Group G2: Checkpointing & Rollback [Luis]** +- [ ] **COMMIT (Owner: Luis | Group: G2.checkpoint) - Commit message: "feat(checkpoint): add checkpointing and rollback"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add checkpoint declarations for tools and plan-level rollback policy. + - [ ] Code [Luis]: Implement `plan rollback ` command. + - [ ] Docs [Luis]: Add `docs/reference/checkpointing.md`. + - [ ] Tests (Behave) [Rui]: Add checkpoint/rollback scenarios. + - [ ] Tests (Robot) [Rui]: Add rollback integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/checkpoint_rollback_bench.py` for rollback latency. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(checkpoint): add checkpointing and rollback"`. -> **IMPORTANT**: This section covers **client-side interfaces for connecting to an external server**. The server itself is a **separate project** that will be developed independently. This client will NOT include server functionality—it operates purely as a client that can either run in stand-alone local-only mode or connect to a separately deployed CleverAgents server. +**Parallel Group G3: Semantic Validation [Luis]** +- [ ] **COMMIT (Owner: Luis | Group: G3.semantic) - Commit message: "feat(validation): add semantic validation service"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add semantic validation hooks during strategize/execute and error-pattern checks. + - [ ] Docs [Luis]: Add `docs/reference/semantic_validation.md`. + - [ ] Tests (Behave) [Rui]: Add semantic validation scenarios. + - [ ] Tests (Robot) [Rui]: Add semantic validation integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/semantic_validation_bench.py` for validation cost. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(validation): add semantic validation service"`. -#### Stage F0: Server Client Interface Stubs [Day 28-29 - REQUIRED DURING MVP] +**Parallel Group G4: Context Tiers & Views [Hamza + Rui]** +- [ ] **COMMIT (Owner: Hamza | Group: G4.context) - Commit message: "feat(context): add hot/warm/cold tiers and actor views"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Implement hot/warm/cold tiers with indexing, LRU eviction, and promotion/demotion. + - [ ] Code [Hamza]: Add per-actor context views (strategist/executor/reviewer) and filtered presentation. + - [ ] Docs [Hamza]: Add `docs/reference/context_tiers.md`. + - [ ] Tests (Behave) [Rui]: Add context tier scenarios. + - [ ] Tests (Robot) [Rui]: Add context tier integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/context_tiers_bench.py` for tier lookup performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(context): add hot/warm/cold tiers and actor views"`. -These stubs ensure the client architecture supports future server connectivity without implementing it: +**Parallel Group G5: Cost & Risk Estimation [Hamza]** +- [ ] **COMMIT (Owner: Hamza | Group: G5.estimate) - Commit message: "feat(estimation): add cost and risk estimation actor"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Hamza]: Add optional `estimation_actor` role and cost/risk estimation outputs. + - [ ] Docs [Hamza]: Add `docs/reference/estimation.md` with output format. + - [ ] Tests (Behave) [Rui]: Add estimation scenarios. + - [ ] Tests (Robot) [Rui]: Add estimation integration smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/estimation_actor_bench.py` for estimation runtime. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Hamza]: `git commit -m "feat(estimation): add cost and risk estimation actor"`. -- [ ] **Stage F0: Server Client Interface Stubs** (Day 28-29) **[Luis - Required]** - - [ ] **F0.1** [Luis] Create `src/cleveragents/interfaces/server_client.py` with protocol stubs: - - [ ] `class ServerClient(Protocol):` with all method signatures for client-to-server communication - - [ ] `async def connect(server_url: str) -> None: raise NotImplementedError("Server connectivity not yet implemented")` - - [ ] `async def disconnect() -> None: raise NotImplementedError(...)` - - [ ] `async def sync_action(action: Action) -> Action: raise NotImplementedError(...)` - - [ ] `async def request_remote_execution(plan_id: str) -> str: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add ServerClient protocol stub" - - [ ] **F0.2** [Luis] Create `src/cleveragents/interfaces/remote_execution_client.py`: - - [ ] `class RemoteExecutionClient(Protocol):` - protocol for requesting remote plan execution from server - - [ ] `async def submit_plan(plan_id: str, server_url: str) -> str: raise NotImplementedError(...)` - - [ ] `async def poll_status(execution_id: str) -> ExecutionStatus: raise NotImplementedError(...)` - - [ ] `async def fetch_results(execution_id: str) -> ExecutionResult: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add RemoteExecutionClient protocol stub" - - [ ] **F0.3** [Luis] Create `src/cleveragents/interfaces/auth_client.py`: - - [ ] `class AuthClient(Protocol):` - protocol for client authentication with server - - [ ] `async def authenticate(credentials: Credentials) -> AuthToken: raise NotImplementedError(...)` - - [ ] `async def validate_token(token: str) -> TokenValidation: raise NotImplementedError(...)` - - [ ] `async def refresh_token(token: str) -> AuthToken: raise NotImplementedError(...)` - - [ ] Commit: "feat(interfaces): add AuthClient protocol stub" - - [ ] **F0.4** [Luis] Add `agents [--data-dir PATH] [--config-path PATH] connect` CLI command as stub: - - [ ] Add to `src/cleveragents/cli/commands/server_client.py` - - [ ] Command signature: `@click.command("connect") @click.argument("server_url")` - - [ ] Implementation: `click.echo("Server connectivity not yet implemented. Coming soon!")`; `raise SystemExit(1)` - - [ ] Commit: "feat(cli): add connect command stub" - - [ ] **F0.5** [Rui] Write minimal tests verifying stubs raise NotImplementedError: - - [ ] Test: Calling ServerClient methods raises NotImplementedError - - [ ] Test: `agents [--data-dir PATH] [--config-path PATH] connect` displays "not implemented" message and exits - - [ ] Commit: "test(behave): add server client stub tests" - ---- - -#### Stages F1-F4: Server Client Implementation [DEFERRED - Beyond Day 30] - -> **These stages are OUT OF SCOPE for the 30-day timeline.** Do not begin work on them until after Day 30 milestone is achieved. Note: These stages implement **client-side** connectivity; the server is a separate project. - -- [ ] **Stage F1: Server Client Infrastructure** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F1.1** [Luis] Create HTTP client in `src/cleveragents/infrastructure/server_client.py` - - [ ] **F1.2** [Luis] Connection health check and version negotiation - - [ ] **F1.3** [Luis] API client code generation from server OpenAPI spec - - [ ] **F1.4** [Rui] Client connection tests (with mock server) - -- [ ] **Stage F2: Plan Sync Client** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F2.1** [Luis] Sync local actions to server - - [ ] **F2.2** [Luis] Request plan creation on server - - [ ] **F2.3** [Luis] Request plan execution on server - - [ ] **F2.4** [Luis] Request `agents [--data-dir PATH] [--config-path PATH] plan apply` on server - - [ ] **F2.5** [Luis] Fetch plan status from server - - [ ] **F2.6** [Rui] Client-side API integration tests (with mock server) - -- [ ] **Stage F3: WebSocket Client** (Post-Day 30) **[Luis]** **[DEFERRED]** - - [ ] **F3.1** [Luis] WebSocket client for receiving plan updates from server - - [ ] **F3.2** [Luis] Handle phase transitions, node completions from server - - [ ] **F3.3** [Rui] WebSocket client tests (with mock server) - -- [ ] **Stage F4: Remote Project Support** (Post-Day 30) **[Hamza]** **[DEFERRED]** - - [ ] **F4.1** [Hamza] Client can specify remote resources for server execution - - [ ] **F4.2** [Hamza] Client sends execution requests to server for remote resources - - [ ] **F4.3** [Rui] End-to-end tests for remote execution (with mock server) - -**M7 SUCCESS CRITERIA** (Post-Day 30): -- [ ] `agents [--data-dir PATH] [--config-path PATH] connect ` establishes connection to an external server -- [ ] Plans can be synced and executed on a remote server -- [ ] Real-time updates received via WebSocket from server -- [ ] Remote projects can be specified and executed on server +**Parallel Group G6: CLI Polish [All]** +- [ ] **COMMIT (Owner: Jeff | Group: G6.cli) - Commit message: "chore(cli): polish help and output"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [All]: Standardize help text, progress indicators, and error messages with recovery hints. + - [ ] Docs [All]: Update CLI output examples where needed. + - [ ] Tests (Robot) [Rui]: Add CLI UX smoke tests for critical commands. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/cli_render_bench.py` for output rendering overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Jeff]: `git commit -m "chore(cli): polish help and output"`. **--- MERGE POINT 2: Day 30 - Large Project Autonomy Target (LOCAL MODE ONLY) ---** @@ -8099,47 +4056,17 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Target: Milestone M7 (+35 days)** -- [ ] **Stage G1: Automation Levels Enhancement** (Day 31-32) **[Luis]** - - [ ] **G1.1** [Luis] Full manual mode with decision prompts: - - [ ] Every decision point pauses for human input - - [ ] Display context, alternatives considered, recommendation - - [ ] Accept explicit choice or custom guidance - - [ ] Record user decisions in decision tree - - [ ] **G1.2** [Luis] Review-before-apply with diff display: - - [ ] AI makes all decisions autonomously during Strategize - - [ ] Execution completes in sandbox - - [ ] Human reviews complete diff before apply: - - [ ] Show changed files summary - - [ ] Show full unified diff with syntax highlighting - - [ ] Show risk warnings (auth code, migrations, etc.) - - [ ] User can approve, reject, or correct specific decisions - - [ ] **G1.3** [Luis] Full automation with confidence escalation: - - [ ] AI makes all decisions autonomously - - [ ] Execution proceeds through apply without pause - - [ ] Human notified of completion - - [ ] Rollback available if issues detected post-apply - - [ ] EVEN in full automation, critical decisions escalate: - - [ ] If confidence below threshold, request human guidance - - [ ] If touching critical files (defined by project), escalate - - [ ] If cost exceeds budget, escalate - - [ ] **G1.4** [Luis] Progressive trust building (track success rates): - - [ ] Track decision success rates per decision type - - [ ] Track codebase familiarity scores per project - - [ ] Confidence increases with successful history - - [ ] Allow automatic upgrade: manual -> review -> full - - [ ] After N successful plans in manual, suggest review mode - - [ ] After N successful plans in review, suggest full mode - - [ ] **G1.5** [Luis] Implement `AutonomyController` class: - - [ ] Method `assess_decision_confidence(decision, context) -> float` - - [ ] Method `should_escalate(decision, confidence, automation_level) -> bool` - - [ ] Method `get_historical_success(decision_type) -> float` - - [ ] Method `get_familiarity_score(project) -> float` - - [ ] **G1.6** [Rui] Tests for each automation level: - - [ ] Scenario: Manual mode pauses at each decision - - [ ] Scenario: Review-before-apply shows diff before apply - - [ ] Scenario: Full automation completes without pause - - [ ] Scenario: Low confidence in full automation escalates - - [ ] Scenario: Progressive trust upgrade suggestion shown +**Parallel Group F0: Server Client Stubs [Luis + Rui]** (required for M6; no server implementation) +- [ ] **COMMIT (Owner: Luis | Group: F0.stubs) - Commit message: "feat(interfaces): add server client stubs"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add protocol stubs for `ServerClient`, `RemoteExecutionClient`, and `AuthClient` with NotImplementedError. + - [ ] Code [Luis]: Add `agents connect ` CLI stub in `cli/commands/server_client.py`. + - [ ] Docs [Luis]: Add `docs/reference/server_client_stubs.md` noting client-only behavior. + - [ ] Tests (Behave) [Rui]: Add stub behavior scenarios. + - [ ] Tests (Robot) [Rui]: Add CLI stub smoke test. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/server_stub_bench.py` (baseline no-op). + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(interfaces): add server client stubs"`. - [ ] **Stage G2: Checkpointing & Rollback** (Day 32-33) **[Luis]** - [ ] **G2.1** [Luis] Skill-level checkpoint declarations @@ -8262,55 +4189,66 @@ By Day 30, the system must be able to (all in LOCAL MODE, server connectivity is **Note**: Quality automation setup is in Section 0; Section 10 focuses on async infrastructure and later-stage validation support. -- [ ] **Stage 10A: Async Infrastructure** (Days 10-12) **[Luis]** - - [ ] Code: Implement async patterns per ADR-002 - - [ ] **10A.1** [Luis] Implement async command execution - - [X] **10A.2** Implement the 33 retry patterns with tenacity (COMPLETED 2025-11-17) - - [X] **10A.3** Add circuit breaker for failures (COMPLETED 2025-11-17) - - [ ] **10A.4** [Luis] Add background workers (convert 7 concurrency patterns to asyncio tasks) - - [ ] **10A.5** [Luis] Integrate retry patterns into new services +**Parallel Group 10A: Async Infrastructure [Luis]** +- [ ] **COMMIT (Owner: Luis | Group: 10A.async) - Commit message: "feat(async): add async command execution and workers"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Implement async command execution per ADR-002 with cancellation and timeout handling. + - [ ] Code [Luis]: Add background worker orchestration for plan lifecycle events. + - [ ] Docs [Luis]: Update `docs/reference/async_architecture.md` with execution flow and shutdown rules. + - [ ] Tests (Behave) [Rui]: Add `features/async_execution.feature` for async command handling. + - [ ] Tests (Robot) [Rui]: Add `robot/async_execution.robot` smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/async_execution_bench.py` for worker scheduling overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(async): add async command execution and workers"`. +- [ ] **COMMIT (Owner: Luis | Group: 10A.retry) - Commit message: "feat(async): wire retry policies into services"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Integrate retry/circuit breaker policies into service layer operations. + - [ ] Docs [Luis]: Document retry policy defaults and override points. + - [ ] Tests (Behave) [Rui]: Add retry/circuit breaker behavior scenarios. + - [ ] Tests (Robot) [Rui]: Add resilience smoke tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/retry_policy_bench.py` for retry overhead. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "feat(async): wire retry policies into services"`. -- [ ] **Stage 10B: Selective Quality Review** (Days 4-8) **[Brent]** - - [ ] Focus Areas: Only review critical items - - [ ] **10B.1** [Brent] Review architectural decisions in PRs: - - [ ] Service layer design choices - - [ ] Database schema decisions - - [ ] API contract definitions - - [ ] Skip: formatting, simple CRUD, test files - - [ ] **10B.2** [Brent] Review complex algorithms: - - [ ] Decision tree traversal logic - - [ ] Merge conflict resolution - - [ ] Dependency closure computation - - [ ] Skip: straightforward implementations - - [ ] **10B.3** [Brent] Review security-sensitive code: - - [ ] Authentication/authorization - - [ ] Input validation - - [ ] Sandbox boundaries - - [ ] Skip: code already scanned by bandit - - [ ] **10B.4** [Brent] Monitor automated quality metrics: - - [ ] Daily check of CI/CD dashboard - - [ ] Weekly quality report generation - - [ ] Escalate only if metrics drop +**Parallel Group 10B: Selective Quality Review [Brent]** +- [ ] **COMMIT (Owner: Brent | Group: 10B.review) - Commit message: "docs(qa): add review playbook and priority matrix"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Docs [Brent]: Create `docs/development/review_playbook.md` with focus areas and skip rules. + - [ ] Docs [Brent]: Add priority matrix and review SLA guidance. + - [ ] Tests (Behave) [Rui]: Add scenarios validating review playbook references exist. + - [ ] Tests (Robot) [Rui]: Add docs build smoke test covering the new guide. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/docs_build_bench.py` for docs build baseline. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Brent]: `git commit -m "docs(qa): add review playbook and priority matrix"`. -- [ ] **Stage 10C: Validation Testing Support** (Days 9-30) **[Brent + Luis]** - - [ ] High-impact testing work - - [ ] **10C.1** [Brent] Create edge case test scenarios: - - [ ] Concurrent plan execution edge cases - - [ ] Resource conflict scenarios - - [ ] Validation failure chains - - [ ] Rollback edge cases - - [ ] **10C.2** [Brent + Luis] Implement semantic validation tests: - - [ ] API compatibility validation - - [ ] Business invariant preservation - - [ ] Cross-resource consistency - - [ ] **10C.3** [Brent] Performance testing for scale: - - [ ] 10K+ file repository handling - - [ ] Memory usage profiling - - [ ] Context tier performance - - [ ] **10C.4** [Brent] Create validation test fixtures: - - [ ] Invalid code samples - - [ ] Edge case project structures - - [ ] Malformed input data +**Parallel Group 10C: Validation Testing Support [Brent + Luis]** +- [ ] **COMMIT (Owner: Brent | Group: 10C.edge) - Commit message: "test(validation): add edge case suites"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Brent]: Add shared edge-case fixtures under `features/fixtures/validation/`. + - [ ] Docs [Brent]: Update `docs/development/testing.md` with validation test catalog. + - [ ] Tests (Behave) [Rui]: Add edge-case scenarios for concurrency, conflicts, and rollbacks. + - [ ] Tests (Robot) [Rui]: Add integration coverage for edge-case suites. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/validation_edge_bench.py` for edge-case runtime. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Brent]: `git commit -m "test(validation): add edge case suites"`. +- [ ] **COMMIT (Owner: Luis | Group: 10C.semantic) - Commit message: "test(validation): add semantic validation suites"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Luis]: Add semantic validation fixtures and error-pattern samples. + - [ ] Docs [Luis]: Document semantic validation coverage expectations. + - [ ] Tests (Behave) [Rui]: Add semantic validation scenarios. + - [ ] Tests (Robot) [Rui]: Add semantic validation integration tests. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/semantic_validation_suite_bench.py` for suite runtime. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Luis]: `git commit -m "test(validation): add semantic validation suites"`. +- [ ] **COMMIT (Owner: Brent | Group: 10C.performance) - Commit message: "test(perf): add scale test fixtures"** (Only check after all subitems + `nox` pass + coverage >=97%, then commit) + - [ ] Code [Brent]: Add scale fixtures for 1K/5K/10K file repos in `features/fixtures/scale/`. + - [ ] Docs [Brent]: Add scale test runbook and environment notes. + - [ ] Tests (Behave) [Rui]: Add scale test scenarios validating thresholds. + - [ ] Tests (Robot) [Rui]: Add large-project Robot tests for performance runs. + - [ ] Tests (ASV) [Rui]: Add `asv/benchmarks/scale_fixture_bench.py` for baseline performance. + - [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark). + - [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. + - [ ] Commit [Brent]: `git commit -m "test(perf): add scale test fixtures"`. ---