diff --git a/docs/implementation-notes.md b/docs/implementation-notes.md new file mode 100644 index 000000000..edec63b2e --- /dev/null +++ b/docs/implementation-notes.md @@ -0,0 +1,519 @@ +# Implementation Notes + +This document is the chronological record of decisions, discoveries, bugfixes, and +design notes made during the development of CleverAgents. Each entry is timestamped +and attributed to the developer who performed the work. These notes were captured +in real time as the implementation progressed and are preserved here as the +authoritative development history. + +--- + +## Phase 0: Discovery and Requirements Elaboration + +- **2025-11-01**: Implemented CLI inventory extraction tools, server endpoint extraction, data contract extraction, shell asset extraction. +- **2025-11-02**: Implemented environment variable extraction and implicit behavior extraction tools. +- **2025-11-04**: Completed workflow parity matrix generator and cloud features identification. All discovery artifacts stored in `docs/reference/`. + +--- + +## Phase 1: Target Architecture Definition + +- **2025-11-04**: Completed all 10 Architecture Decision Records. +- **2025-11-05**: Package structure created based on ADR-001. All ADRs documented in `docs/architecture/decisions/`. +- **2026-02-11**: CLI syntax updated to spec-aligned forms (`action create` via `--config`, `plan use` positional args, `project create` positional). Legacy `--name`/`--project` examples retained in historical notes only. +- **2026-02-11**: Action config YAML baseline (spec-aligned): + ```yaml + name: local/example-action + description: Example action for CLI flows + strategy_actor: openai/gpt-4 + execution_actor: openai/gpt-4 + definition_of_done: "All steps complete" + ``` + +--- + +## Phase 2: Runtime Foundation and Ongoing Development + +### Legacy Phase 2 Work (Pre-Rebaseline) + +**2025-11-22**: Week 12 Complete, Phase 2 Core Functionality DONE + +- CLI Streaming Integration fully implemented. +- AutoDebugGraph Implementation complete. +- Mock provider enhancements with configurable failure modes. +- Infrastructure improvements (DebugAttempt model, repositories). +- 2026-02-11: CLI syntax refreshed to spec-aligned forms in current execution sections; legacy `--name`/`--project` examples retained only in historical notes. + +**2025-12-05**: LangSmith observability integration complete. + +**2025-12-08**: Actor-first provider wiring complete. + +**2025-12-17**: Stage 7 performance optimization complete. + +**2026-02-02**: Stage 7.5 Actor Configuration System complete. + +### Plan and Action Domain Models (A1-A4) + +**2026-02-05**: Stage A1 & A2 Complete - Plan and Action Domain Models + +- Created `Plan` model with `PlanPhase` enum (ACTION, STRATEGIZE, EXECUTE, APPLY, APPLIED), `ActionState` enum (AVAILABLE, DRAFT, ARCHIVED), `ProcessingState` enum (QUEUED, PROCESSING, ERRORED, COMPLETE, CANCELLED), `NamespacedName` model with `parse()` and `str()` methods, `PlanIdentity` model with ULID validation, `can_transition()` function for phase transition validation. +- Created `Action` model with `ActionArgument` model with `parse()` method for CLI argument parsing, argument validation including type checking. +- Added 52 Behave test scenarios across 2 feature files: `plan_model.feature` (30 scenarios), `action_model.feature` (22 scenarios). + +**2026-02-05**: Stage A3 Complete - PlanLifecycleService + +- Created `PlanLifecycleService` with full plan lifecycle management (Action -> Strategize -> Execute -> Apply -> Applied), Action CRUD operations, phase transition methods, state management, custom exceptions (`InvalidPhaseTransitionError`, `ActionNotAvailableError`, `PlanNotReadyError`). +- Added python-ulid dependency for ULID generation. +- Total new test scenarios: 81 (30 + 22 + 29). + +**2026-02-05**: Stage A4 In Progress - Plan CLI Commands + +- Created `action.py` CLI with `action create`, `action list`, `action show`, `action available`, `action archive` commands. +- Extended `plan.py` with v3 lifecycle commands: `plan use`, `plan execute`, `plan apply`, `plan status`, `plan list`, `plan cancel`. +- Total test scenarios: 96 (81 + 15). + +### Critical Architectural Decision + +**2026-02-06**: CRITICAL ARCHITECTURAL DECISION - Tool-Based Resource Modification + +- **REPLACED**: OutputParser/code fence parsing approach. +- **WITH**: Tool-based change tracking (modern approach used by Claude Code, Cursor, Aider). +- **Key changes**: + - LLMs call skills/tools directly (edit_file, write_file, delete_file, etc.). + - Skills operate on sandbox state directly. + - ChangeSet is built from skill invocation history, NOT by parsing LLM text output. +- Added built-in resource skills (now C4.file/C4.search/C4.git): file ops, dir ops, search, git ops. +- Added MCP skill adapter (now C7.mcp): connect to external MCP servers. + - Replaced C4 "Multi-File ChangeSet Generation" with "Tool-Based Change Tracking". + - Added SkillInvocationTracker and ToolCallRouter components. +- **Rationale**: + - No parsing ambiguity (is this code or explanation?). + - Each operation is explicit, typed, and trackable. + - Supports rollback (replay inverse of recorded changes). + - Resource-agnostic (works for files, databases, APIs, any resource type). + - Compatible with MCP standard for external tools. + +### Quality Automation Setup (Q0-Q2) [Brent] + +**2026-02-09**: Task Q0.6b Complete - README.md Setup Instructions + +- Updated README.md Quick Start: added `dev` extras to `pip install`, added `scripts/setup-dev.sh` step. +- Updated README.md Developing section: fixed `oxt` typo -> `nox`, added quality/security nox sessions, linked to `docs/development/quality-automation.md`. + +**2026-02-09**: Ruff Cleanup in src/cleveragents/ - StrEnum Migration + +- Fixed all 26 ruff findings (all UP042: `str, Enum` -> `StrEnum`) + 12 consequent F401 unused `Enum` imports. +- Migrated 26 enum classes across 15 files from `class Foo(str, Enum)` to `class Foo(StrEnum)`. +- `StrEnum` (Python 3.11+) is the modern replacement; project targets Python 3.13. +- Semantic difference: `str(StrEnum.MEMBER)` returns the value (e.g., `"foo"`) rather than `"ClassName.MEMBER"` — this is the correct/intended behavior for config/JSON string enums. +- Verified: no code uses `str()` on enum members in the old format; all tests pass (304 scenarios, 0 failures). + +**2026-02-09**: Task Q0.9 Complete - Ruff Lint Findings in features/ + +- Fixed all 200 ruff lint findings in `features/` directory -> **0 findings**. +- **Config-level suppressions** (168 findings): Added `per-file-ignores` in `pyproject.toml` for Behave-specific patterns (F811 for redefined `step_impl`, E501 for long step decorator strings). +- **Manual fixes** (31 findings across 18 files): SIM115, UP028, SIM117, RUF005, B904, RUF012, SIM105, B007, B018, F821, SIM102, I001. + +**2026-02-09**: Task Q0.8 Complete - Bandit Security Findings Remediation + +- Fixed all 16 pre-existing bandit findings (2 HIGH, 3 MEDIUM, 11 LOW) -> **0 findings**. +- **Security hardening** (HIGH+MEDIUM): Replaced `jinja2.Environment` with `jinja2.sandbox.SandboxedEnvironment`, added `_validate_code_ast()` helper for AST-based pre-validation, added `_validate_lambda_ast()` helper restricting `eval()` to lambda-only expressions. +- **Code quality** (LOW): Replaced 6 `assert` statements with proper `if`/`raise`, replaced `try/except/pass` with `contextlib.suppress(Exception)`, added logging to silent exception handlers. + +**2026-02-09**: Stages Q0, Q1, Q2 Complete - Full Quality Automation Setup + +- **Stage Q0 - Pre-commit Hooks**: Created `.pre-commit-config.yaml` with 12 hooks across 5 categories (branch protection, general checks, Ruff, Pyright, Bandit, Vulture, Semgrep, Commitizen). +- **Discovery**: CI platform is Forgejo (`.forgejo/`), NOT GitHub. Stage Q1 must use Forgejo Actions. +- **Discovery**: 37 source files have formatting issues, ~27 files have trailing whitespace - pre-existing debt. +- **Discovery**: `features/steps/actor_cli_steps.py` has many F811 (redefined step_impl) violations - Behave pattern. +- **Discovery**: Average code complexity is A (3.56) across 979 blocks - good baseline. +- **Discovery**: High complexity methods identified: `LegacyDataMigrator.migrate_project_data` E(37), `ProviderRegistry._create_provider_llm` C(20), `ProviderRegistry.create_ai_provider` C(18). +- **Stage Q1 - CI/CD Pipeline**: Extended `.forgejo/workflows/ci.yml` with security, quality, coverage jobs. Created `scripts/check-quality-gates.py`. +- **Stage Q2 - Advanced Automation**: Created nightly quality monitoring workflow, ADR compliance checker, PR template. New nox sessions: `pre_commit`, `security_scan`, `dead_code`, `complexity`, `adr_compliance`. + +**2026-02-10**: Task 10B.4 - Quality Metrics Baseline Established [Brent] + +- Ran full quality suite via nox to establish current baseline: + - **Unit Tests**: 105 features, 1613 scenarios, 7555 steps - ALL PASS + - **Lint (ruff)**: 0 findings + - **Typecheck (pyright)**: 0 errors, 0 warnings + - **Security (bandit)**: 0 findings + - **Dead Code (vulture)**: 0 findings + - **Complexity (radon)**: Average A (3.56), 981 blocks analyzed, no grade-F methods + - **Coverage**: 96% (9860 statements, 269 missing, 2852 branches, 213 branch-miss) +- Fixed pre-existing test failure: Rich table column wrapping at narrow terminal widths. Fixed by patching console width to 200 in test setup. +- Fixed missing dependency: added `langchain-anthropic>=0.2.0` to `pyproject.toml`. + +**2026-02-10**: Task Q1.5 Complete - Branch Protection Rules Documentation [Brent] + +- Created `docs/development/ci-cd.md` (224 lines) documenting branch protection rules, review priority matrix, CI job dependency graph, quality gates summary. + +### Edge Case and Validation Tests [Brent] + +**2026-02-10**: Task 10C.1 Complete - Edge Case Test Scenarios + +- Created `features/edge_case_plan_scenarios.feature` (26 scenarios, 141 steps) covering concurrent plan execution, resource conflict scenarios, validation failure chains, rollback edge cases. + +**2026-02-10**: Task 10C.4 Complete - Validation Test Fixtures + +- Created `features/validation_test_fixtures.feature` (34 scenarios, 81 steps) covering AST security validation, lambda AST validation, Python content sanitization, project model validation, change list coercion, ActionArgument parsing. +- Fixed step name collisions with existing step files. +- Fixed irrecoverable syntax scenario: `"def broken("` is actually recoverable via docstring wrapping; replaced with null byte input which is truly irrecoverable. + +### CI/CD Pipeline Stabilization [Brent] + +**2026-02-12**: Task Q0-min-ci In Progress - Nox-Based PR Validation Workflow + +- Rewrote `.forgejo/workflows/ci.yml` to route ALL jobs through nox sessions instead of direct tool invocations. +- **Coverage boost 92%->97%**: Wrote 108 new Behave scenarios covering 6 largest coverage gaps. +- **Key discovery**: Behave loads ALL step definition files globally; any `@given/@when/@then` pattern matching an existing one causes `AmbiguousStep` errors. + +**2026-02-12**: Bugfix - Integration Test Failure in Robot.Actor Configuration + +- **Root cause**: `robot/ci_nox_validation.robot` ran `nox --list` during integration tests. When `pabot` executed all Robot suites in parallel, the `nox --list` stdout leaked into the sibling `robot/actor_configuration.robot` process, causing `JSONDecodeError`. +- **Fix**: Added `slow` tag to interfering test cases; replaced hardcoded venv paths with `python`. + +**2026-02-12**: Bugfix - CI `security` Job Failure (session name mismatch) + +- **Root cause**: CI referenced `nox -s security_scan` and `nox -s dead_code`, but the noxfile only had a session named `security`. +- **Fix**: Renamed `security` to `security_scan`, added standalone `dead_code` session. + +**2026-02-12**: Bugfix - CI `unit_tests` Failure: Rich Console Line-Wrapping + +- **Root cause**: Rich's `Console` wraps output at 80 columns. On CI, temp directory paths exceed 80 chars, splitting filenames across lines. +- **Fix**: Collapse Rich line-wraps via `clean_output.replace("\n", "")` before assertions. + +**2026-02-12**: Bugfix - CI `integration_tests` Failures (3 categories) + +- **Issue 1**: `ModuleNotFoundError` — hardcoded `/app` path doesn't exist on CI. Fixed with `${CURDIR}/..` relative paths. +- **Issue 2**: `--load-context` help text assertion failures — Rich output goes to stderr when no TTY. Fixed with `stderr=STDOUT` merge. +- **Issue 3**: `FileNotFoundError: robot` — pabot subprocess spawning exhausts container resources. Restored `--exclude discovery` flag. + +**2026-02-13**: Bugfix - CI `security` Job Failure (missing `build/` directory) + +- **Root cause**: `build/` directory does not exist in fresh CI checkout. Bandit cannot create intermediate directories. +- **Fix**: Added `os.makedirs("build", exist_ok=True)` before bandit invocation. + +**2026-02-13**: Bugfix - CI `integration_tests` Failures (3 categories) + +- **Issue 1**: `FileNotFoundError: robot` after ~17 suites — container resource exhaustion. Capped pabot parallelism to `min(default, 2)`, explicitly prepend venv `bin/` to PATH. +- **Issue 2**: Rich ANSI escape codes split `--load-context` string. Added `env:NO_COLOR=1`. +- **Issue 3**: `initial_next_command_test.robot` exit code 3 — missing `OPENAI_API_KEY`. Tagged as `slow`. + +**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 2 (pabot resource exhaustion + ANSI codes) + +- `robot` is findable at session start yet pabot still fails after ~24 suites even with `--processes 2`. +- **Root cause confirmed**: `FileNotFoundError` from `subprocess.Popen` on Linux occurs when the system cannot `execve()` due to exhausted PIDs or file descriptors, not because the binary is missing. +- **Fix**: Replaced `pabot` with `robot` (sequential execution). CI values reliability over speed. +- `NO_COLOR=1` insufficient — Rich still embeds non-color ANSI sequences (bold, reset). Added `env:TERM=dumb`. + +**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 3 (venv PATH, Resource paths, hanging tests) + +- **Issue 1**: `python -m cleveragents` uses system Python — venv PATH line was accidentally removed. Restored. +- **Issue 2**: Robot `Resource` file resolution fails on CI — bare relative references. Converted all to `${CURDIR}/` absolute paths across 30 robot files. +- **Issue 3**: `rxpy_route_validation.robot` tests hang — missing `timeout` parameters on `Run Process` calls. Added `timeout=30s` to all 9 calls. + +**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 4 (duplicate Settings blocks, venv Python injection, CI debug) + +- **Issue 1**: Duplicate `*** Settings ***` blocks in 14 robot files created by sed-based conversion. Merged into single blocks. +- **Issue 2**: Bare `python` in `Run Process` calls resolves to system Python on CI. Noxfile now passes `--variable PYTHON:` to `robot`. All 14 files updated to use `${PYTHON}`. +- **Issue 3**: Hardcoded `/app/src` path in `system_prompt_template_rendering.robot`. Removed (venv Python already has package installed). + +**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 5 (guard cleanup when OpenAI key missing) + +- **Root cause**: `robot/scientific_paper_basic.robot` suite setup skips on missing `OPENAI_API_KEY`, leaving `${CONTEXT_DIR}` empty. Teardown's `Remove Directory ${CONTEXT_DIR} recursive=True` resolves to CWD, deleting the repo root (including `.nox` venv and `robot/*.resource` files), causing all downstream suites to fail. +- **Fix**: Initialize `${CONTEXT_DIR}` to `${TEMPDIR}/paper_basic_contexts` and guard cleanup with empty check. + +**2026-02-13**: Enhancement - Restore parallel Robot runs + silence discovery resource warning + +- Reintroduced `pabot` with conservative parallelism (max 2 processes) now that teardown bug is fixed. +- Added `robot/discovery_common.resource` stub to silence non-fatal import warning. + +### Coverage and Security Enforcement [Brent] + +**2026-02-13**: Task Q0-min-coverage In Progress - Coverage Threshold Enforcement + +- Enhanced `coverage_report` nox session: `COVERAGE_THRESHOLD = 97` constant, generates HTML/XML/JSON reports before checking threshold, emits CI-parseable summary line. +- Created `docs/development/testing.md` (180+ lines): full testing guide covering unit (Behave), integration (Robot), benchmarks (ASV), and coverage. +- Created `features/coverage_threshold_enforcement.feature` (11 scenarios). +- Verification: 2235 unit scenarios passed, 211 integration tests passed, 97.5% coverage. + +**2026-02-13**: Task Q0-adv-security In Progress - Align Security Scans with Nox + +- Added Semgrep as Step 3 in `security_scan` session (between Bandit and Vulture). +- Changed semgrep hook from wrapper script to direct invocation. +- Updated quality-automation docs with revised thresholds (85->97%). + +### Plan Model Alignment [Luis] + +**2026-02-13**: Stage A2b.beta Complete - Plan Model Alignment + +- Rewrote `plan.py` to align with spec: `processing_state` replaces `action_state`/`state`, `project_links` replaces `project_ids`, added `action_name`, `automation_profile`, `invariants`, actor fields, subplan hierarchy. +- Rebased `feature/m1-plan-model` onto A2b.alpha commit. Resolved 30 merge conflicts. Key design decisions during merge: + 1. Plan field name: `processing_state` (per spec), NOT `state`. + 2. Invariant enum: `InvariantSource` (our naming), NOT `InvariantScope`. + 3. Action model: HEAD taken as-is (authoritative for action domain). + 4. No `action_id` on Action domain model — uses `namespaced_name` only. + 5. `project_links` used everywhere (NOT `project_ids`). +- Fixed 6 post-rebase test failures. +- Final results: 130 Behave features / 2246 scenarios / 9868 steps all pass, 206 Robot tests all pass, `plan.py` coverage 100%. + +### Persistence Layer (A5) + +**2026-02-09**: Stage A5.3 + A5.4 Complete - v3 SQLAlchemy Models + +- Created `LifecycleActionModel` (table: `actions_v3`) and `LifecyclePlanModel` (table: `lifecycle_plans`) with full spec alignment. +- Used `__allow_unmapped__ = True` on both models for `from __future__ import annotations` compatibility. +- **Legacy note**: These v3 ORM models use `action_id`/`project_ids` and draft-style fields; they are superseded by the A5 rebaseline. + +**2026-02-09**: Stage A5.6 Complete - ActionRepository (Luis tasks A5.6a-A5.6j) + +- Created `ActionRepository` with session-factory pattern, 10 methods, `@database_retry` on all public methods. +- All mutating methods flush (do NOT commit) — caller/UnitOfWork handles commit. +- Uses `Any` type hints for domain Action objects to avoid circular import issues. + +### Sandbox Infrastructure [Luis] + +**2026-02-09**: Stage B3.1 + B3.2 + B3.5 + B3.6 + B3.7 + B3.8 Complete - Sandbox Infrastructure + +- Created `src/cleveragents/infrastructure/sandbox/` package with 6 modules: `protocol.py`, `no_sandbox.py`, `factory.py`, `manager.py`, `merge.py`, `__init__.py`. +- Design decisions: + - Factory accepts raw strings rather than Resource objects (Resource model not yet implemented). This decouples the sandbox layer from the domain model. + - `SandboxStatus.assert_transition()` provides a convenient guard that raises `SandboxStateError` on invalid transitions. + - `NoSandbox.rollback()` always raises `SandboxRollbackError` rather than being a no-op, because silently swallowing rollback requests for unsandboxed resources would hide bugs. + +**2026-02-09**: Stage A2.1 Already Complete + +- `estimation_actor` field already present in action model. `review_actor` field already present. Also has `apply_actor` (bonus field from earlier work). +- `safety_profile` remains DEFERRED to post-30 per plan. + +### Security Hardening (SEC1) [Luis] + +**2026-02-09**: Stage SEC1 Complete - Remove eval() Vulnerability + +- **Audit results** (7 matches in `src/cleveragents/`): 2 real vulnerabilities, 4 false positives. +- **Fixes applied**: `SimpleToolAgent` — added named operation registry (`_SAFE_OPERATIONS`), legacy `code` blocks fully rejected. `ReactiveStreamRouter` — added named transform registry, legacy `fn` string expressions that are not in the registry fully rejected. No `eval()` or `exec()` calls remain. + +**2026-02-10**: Stage SEC1.5 Complete - Security BDD tests + +- Created `features/security_eval.feature` with 8 scenarios covering code injection rejection, malicious transform rejection, valid config acceptance, custom registered operations. + +### Subplan Model [Luis] + +**2026-02-09**: Stage E1 Complete - Subplan Model (Luis tasks E1.1-E1.5) + +- Added `ExecutionMode`, `SubplanMergeStrategy` enums, `SubplanConfig`, `SubplanStatus`, `SubplanAttempt` models, `SubplanFailureHandler` class. +- Renamed from `MergeStrategy` to `SubplanMergeStrategy` to avoid collision with infrastructure-level `MergeStrategy` protocol. +- Used Pydantic BaseModel instead of dataclass (per ADR-004) for `SubplanAttempt` and `SubplanStatus`. + +### Automation Levels [Luis] + +**2026-02-09**: Stage A6 Complete - Automation Levels Foundation + +- Added `AutomationLevel` enum (MANUAL, REVIEW_BEFORE_APPLY, FULL_AUTOMATION). +- Updated `PlanLifecycleService` with `should_auto_progress()`, `auto_progress()`, `pause_plan()`, `resume_plan()`, `set_plan_automation_level()`. +- `complete_strategize()` and `complete_execute()` now call `auto_progress()` automatically. + +**2026-02-10**: Stage A6.5 Complete - Automation Levels BDD Tests + +- Created `features/automation_levels.feature` with 24 Behave scenarios. +- **Discovery**: pydantic-settings `BaseSettings` does not accept constructor keyword overrides for fields with `validation_alias`; must set attribute after construction. + +### Action YAML Schema [Aditya] + +**2026-02-13**: Stage A2b.gamma Complete - Action YAML Schema + Examples + Loader + +- Created `docs/schema/action.schema.yaml`, 5 example action YAMLs, `ActionConfigSchema` Pydantic model with `from_yaml()`/`from_yaml_file()` factory methods, camelCase->snake_case key normalization, `${ENV_VAR}` interpolation, invariant normalization. +- 31 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmark suites. Coverage: 98%. + +### Tool and Skill Domain Models [Jeff] + +**2026-02-13**: Stage C0.domain Complete - Tool and Validation Domain Models + +- Created `tool.py` (624 lines) with 6 StrEnums, 5 Pydantic models, full validators, factory methods. +- Key decision: YAML loader implemented as `from_config()` class methods (matches action.py pattern) rather than separate module. +- Key decision: `Validation` extends `Tool` via Pydantic model inheritance. +- Key decision: `wraps` field sets source to `wrapped` implicitly; `transform` required when `wraps` is set. + +**2026-02-13**: Stage C0.skill.domain Complete - Skill Domain Model and Resolver + +- Created `skill.py` (475 lines) with 9 Pydantic models. `SkillResolver.resolve()` flattens includes, de-duplicates tools, rejects cycles with path traces. +- Key decision: `SkillResolver` is a standalone class (not a method on Skill) to support registry-backed resolution. +- Key decision: `ResolvedToolEntry` includes source tracking for compiler diagnostics. + +### Resource Registry [Jeff] + +**2026-02-13**: Stage B1.core.db Complete - Resource Registry Database Tables + +- Created Alembic migration with 3 tables: `resource_types`, `resources`, `resource_edges`. +- Key design decision: `resource_types` PK is the namespaced name string (not ULID) since types are schema-level definitions with stable identifiers. +- Key design decision: `link_type` enum ('contains', 'references', 'derived_from') covers hierarchical containment, cross-layer references, and derived virtual resources. +- Key design decision: `auto_discovered` flag on both resources and edges tracks provenance for cleanup/refresh operations. + +### Session Domain [Jeff] + +**2026-02-13**: Stage A7.domain Complete - Session Domain Models and Contracts + +- Created `session.py` with `MessageRole` enum, `SessionMessage`, `SessionTokenUsage`, `Session` models, error classes, `SessionService` ABC. +- Key decision: SessionService is an ABC (not Protocol) to enforce explicit inheritance and make contract violations visible at class definition time. +- Key decision: Messages are ordered by `sequence_number` (not timestamp) to guarantee stable ordering across serialization boundaries. + +### CLI Spec Alignment [Jeff] + +**2026-02-14**: Stage A4b.action Complete - Action CLI Spec Alignment + +- Rewrote `action.py`: config-only `action create --config `, removed `action available` command, added filters to `action list`. + +**2026-02-14**: Stage A4b.plan Complete - Plan CLI Flag Alignment + +- Updated `plan.py`: `plan use` now accepts multiple PROJECT args, `--automation-profile`, `--invariant` (repeatable), actor overrides, `--arg name=value`. Aligned `plan list` filters. Added `--reason` to `plan cancel`. + +**2026-02-14**: Stage A4b.outputs Complete - CLI Output Format Stabilization + +- Created `formatting.py`: shared `format_output()` helper with `OutputFormat` enum, JSON/YAML/plain/table/rich serializers. Handles datetime, Enum, nested structures. + +### Tool Runtime and Built-ins [Jeff] + +**2026-02-14**: Stage C0.runtime Complete - Tool Runtime Core + +- Created `runtime.py` (ToolSpec, ToolResult, ToolError), `registry.py` (thread-safe ToolRegistry with RLock), `runner.py` (ToolRunner with 4-stage lifecycle: discover/activate/execute/deactivate). +- New tool modules have 100% coverage. + +**2026-02-14**: Stage C0.files Complete - Built-in File Tools + +- Created `src/cleveragents/tool/builtins/` package with 6 file tools (file-read/write/edit/delete/list/search), path traversal prevention, ChangeSetEntry/ChangeSet/ChangeSetCapture models. +- 2026-02-14: Commit traceability sweep found no matching git log entries for skill schema/examples, skill CLI, provider actors, MCP config/adapter, actor compiler, or skill registry persistence; Section 2B traceability tasks remain open. + +### Tool and Session Persistence [Luis] + +**2026-02-14**: Stage C1.tool.registry Complete - Tool Registry Persistence + +- Created Alembic migration with 3 tables: `tools`, `tool_resource_bindings`, `validation_attachments`. +- Created `ToolRegistryRepository` (5 methods), `ValidationAttachmentRepository` (4 methods), `ToolRegistryService` (8 methods). +- Key decision: `ToolModel.to_domain()` returns a dict (not a domain object) to decouple persistence from domain model until full integration is complete. +- Key decision: `ToolRegistryRepository.delete()` pre-checks for active `ValidationAttachmentModel` rows before allowing deletion. +- Key decision: Validation attachments are resource-scoped with optional project_name and plan_id narrowing. + +**2026-02-15**: Stage A7.persistence Complete - Session Persistence and Repositories + +- Created Alembic migration with 2 tables: `sessions`, `session_messages`. +- Created `SessionRepository` (5 methods), `SessionMessageRepository` (3 methods), `PersistentSessionService` (8 methods). +- Key decision: `PersistentSessionService` extends the domain `SessionService` ABC. +- Key decision: `append_message()` auto-calculates sequence number and updates parent session's `updated_at` timestamp. +- Key decision: `import_session()` validates schema version and SHA-256 checksum before persisting, then generates fresh ULIDs to avoid ID collisions. +- Key decision: `update_token_usage()` uses cumulative addition (+=) rather than replacement. + +**2026-02-15**: Bugfix - Benchmark Unique Name Constraint [Luis] + +- ASV benchmark suites that create tools/resources now use unique names per iteration to avoid UNIQUE constraint failures when ASV runs multiple iterations. + +### Skill and Actor YAML [Aditya] + +**2026-02-16**: C0.skill.schema Complete - Skill YAML Schema + Examples + Loader + +- Created `docs/schema/skill.schema.yaml`, 5 example skill YAMLs, `SkillConfigSchema` Pydantic model with `from_yaml()`/`from_yaml_file()` factory methods, camelCase->snake_case normalization, `${ENV_VAR}` interpolation. +- 36 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmarks. + +**2026-02-17**: Task C1.schema Complete - Actor YAML Schema Models + +- Created `src/cleveragents/actor/schema.py` (695 lines) with three actor types (LLM, TOOL, GRAPH), graph topology validation with cycle detection using DFS, type-specific field requirements, environment variable interpolation. +- 5 example actor YAMLs, 50+ Behave scenarios, 11 Robot test cases, 8 ASV benchmark classes. + +**2026-02-17**: C0.skill.cli Complete - Skill CLI Commands + Output Formatting + +- Created `SkillService` with dual-mode storage pattern (in-memory fallback, ready for `UnitOfWork` persistence). +- Created `skill.py` Typer command group with 5 subcommands (`skill add`, `remove`, `list`, `show`, `tools`), all supporting 6 output formats. +- Key decision: Module-level `_service` singleton (not DI container) matches the pattern in `action.py` CLI; `_reset_skill_service()` enables test isolation. + +### Bugfixes and Merges + +**2026-02-17**: Bugfix - Unit Test and Benchmark Failures After Master Merge [Brent] + +- **Unit test failure** — Session mismatch in project repository steps. The `NamespacedProjectRepository` creates a new SQLAlchemy session per method call. Fix: Changed session factory to always return the same session instance. +- **Benchmark failures** — 5 distinct issues across 7 files: missing parameterized setup params, renamed `PlanPhase.APPLIED` -> `APPLY`, per-iteration plan creation, batch counters for unique IDs across ASV iterations, simplified action reuse. + +**2026-02-18**: Task C1.examples Complete - Actor YAML Examples and Documentation + +- Created `docs/reference/actors_examples.md` (1098 lines) with pattern-based organization (Strategist, Executor, Reviewer, Tool-Only, Validation-Node, Graph, Hierarchical). +- 25 Behave scenarios, 16 Robot test cases, 7 ASV benchmarks. +- Bug fixes: Missing `description` fields in test YAML strings, `context_view` step definition conflict, enhanced parallel execution detection. + +### Actor Loader [Aditya] + +**2026-02-19**: Task C2.loader Complete - Actor Registry and Loader + +- Created `src/cleveragents/actor/loader.py` (277 lines) — `ActorLoader` class with thread-safe discovery, SHA-256 content-hash caching, and namespace management. +- 23 Behave scenarios, 4 Robot test cases, 4 ASV benchmarks. Coverage: 98%. +- Key design decisions: SHA-256 content hashing (not mtime) for deterministic cache invalidation, single consolidated `ValidationError` for duplicates/errors, tool reference resolution at load time via optional `ToolRegistry`, `local/` default namespace applied during YAML parsing. + +### Plan Checklist Rebaseline + +**2026-02-13**: Plan Update - Rebaseline A5 Persistence Checklist + +- Rewrote A5 plan persistence checklist to replace legacy tasks with spec-aligned rebaseline migrations, ORM mappings, repositories, and persistence tests. + +**2026-02-13**: Plan Update - A5 Ownership + Validation Attachments + +- Reassigned A5.alpha migrations to Hamza to align DB/migration expertise. +- Rebased tool registry tasks to make validation attachments explicitly resource-scoped with required/informational mode. + +**2026-02-13**: Plan Update - Consolidate Skills/Actors/MCP + B0 DB Ownership + +- Retired duplicate C2/C3/C4 blocks; canonical tasks consolidated under C0.skill, C1/C2, C7.mcp, C8.providers. +- Reassigned B0.db resource/project migrations to Hamza. + +**2026-02-13**: Plan Update - Tool Runtime Naming Consolidation + +- Renamed M1 tool runtime block to C0 runtime core and M3 lifecycle block to C0.lifecycle to avoid duplicate naming. + +**2026-02-13**: Plan Update - Spec Alignment Pass + +- Updated A4b CLI to use `processing_state` terminology. +- Corrected skill registry dependency references. + +**2026-02-13**: Plan Update - C1 Numbering Disambiguation + +- Renamed tool registry group to C1.tool to avoid collision with actor schema C1 group. + +### Change Tracking and Tool Router + +**2026-02-19**: Stage C5.model COMMIT Complete - ChangeSet models and invocation tracker [Luis] + +- Added `ToolInvocation` Pydantic model, `InvocationTracker` Protocol and `InMemoryInvocationTracker`, per-change field validators, `sorted_entries()` and `grouped_by_resource()` methods, `normalize_change_path()` utility. +- 21 BDD scenarios, 5 integration smoke tests, 4 ASV benchmark suites. + +**2026-02-19**: Stage C4.git COMMIT Complete - Add git operation skills [Luis] + +- Created `git_tools.py`: 4 read-only git tools (status, diff, log, blame) with safe environment variables, path traversal guards, and user-friendly error mapping for 5 common git failure patterns. +- 16 BDD scenarios, 5 integration smoke tests, 3 ASV benchmark suites. + +**2026-02-19**: Combined merge branch fix - Alembic merge migration [Luis] + +- Created merge migration to resolve "Multiple head revisions" error between `a7_001_session_tables` and `c0_001_skill_registry` heads. + +**2026-02-19**: Stage SEC1.eval COMMIT Complete - Remove eval-based config parsing [Luis] + +- Created config security scanner detecting 15 disallowed patterns across YAML/TOML/generic configs. Reports file path + line number + severity for each violation. +- Extended security eval feature from 8 to 16 scenarios. + +### Execution Pipeline and Apply + +**2026-02-20**: D0b.execute — Plan Execute via Actors (M1 Critical Blocker) + +- Created `PlanExecutor` orchestrator with `StrategizeStubActor` (read-only, produces decisions) and `ExecuteStubActor` (sandbox resources, routes tool calls). +- Strategize phase is read-only. Execute phase uses sandbox resources and routes tool calls through ToolRunner. +- Phase guards prevent Execute when Plan is not in Strategize COMPLETE state. +- Streaming hooks emit interim status updates. + +**2026-02-20**: D0b.apply Complete - Diff Review + Apply Integration [Jeff] + +- Created `PlanApplyService` with `diff()`, `artifacts()`, `persist_apply_summary()`, `handle_merge_failure()`, `guard_empty_changeset()`. +- Key decision: Used `error_details` dict on Plan as general metadata store for apply summary fields until a dedicated metadata field lands. +- Key decision: Service accepts optional `changeset_store` — when None, builds stub SpecChangeSet from plan metadata only. + +**2026-02-20**: C5.router Complete - Tool Call Router for LLM Providers [Jeff] + +- Created `router.py` (~890 lines) — `ToolCallRouter` translating between LLM provider tool call formats and internal `ToolRunner`: provider format detection, normalization, deterministic ID generation, error classification, schema adaptation, batch routing, streaming support. +- Key discovery: `ToolRunner.execute()` catches handler exceptions internally and returns `ToolResult(success=False)` rather than raising, so streaming produces COMPLETE + failed result (not ERROR update) for tool failures. +- Key discovery: Pyright strict mode requires all Pydantic model fields (even those with defaults via `Field(default=...)`) to be explicitly passed in constructor calls. +- Key discovery: Behave step name collision with existing `coverage_boost_extra_steps.py`; renamed to avoid `AmbiguousStep` error. diff --git a/mkdocs.yml b/mkdocs.yml index c1125c008..a2b3aadbb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -17,6 +17,7 @@ nav: - Review Playbook: development/review_playbook.md - Scale Testing: development/scale_testing.md - Implementation Timeline: timeline.md + - Implementation Notes: implementation-notes.md - Work Remaining: work-remaining.md - FAQ: faq.md - Reference: reference/