Docs: Removed implementation-notes and migrated that information as comments on tickets
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 13s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 19s
CI / typecheck (pull_request) Successful in 32s
CI / security (pull_request) Successful in 53s
CI / integration_tests (pull_request) Successful in 3m29s
CI / unit_tests (pull_request) Successful in 6m46s
CI / docker (pull_request) Successful in 39s
CI / benchmark-regression (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 13s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 19s
CI / typecheck (pull_request) Successful in 32s
CI / security (pull_request) Successful in 53s
CI / integration_tests (pull_request) Successful in 3m29s
CI / unit_tests (pull_request) Successful in 6m46s
CI / docker (pull_request) Successful in 39s
CI / benchmark-regression (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
This commit is contained in:
@@ -1,553 +0,0 @@
|
||||
# Implementation Notes
|
||||
|
||||
This document is the chronological record of decisions, discoveries, bugfixes, and
|
||||
design notes made during the development of CleverAgents. Each entry is timestamped
|
||||
and attributed to the developer who performed the work. These notes were captured
|
||||
in real time as the implementation progressed and are preserved here as the
|
||||
authoritative development history.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Discovery and Requirements Elaboration
|
||||
|
||||
- **2025-11-01**: Implemented CLI inventory extraction tools, server endpoint extraction, data contract extraction, shell asset extraction.
|
||||
- **2025-11-02**: Implemented environment variable extraction and implicit behavior extraction tools.
|
||||
- **2025-11-04**: Completed workflow parity matrix generator and cloud features identification. All discovery artifacts stored in `docs/reference/`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Target Architecture Definition
|
||||
|
||||
- **2025-11-04**: Completed all 10 Architecture Decision Records.
|
||||
- **2025-11-05**: Package structure created based on ADR-001. All ADRs documented in `docs/architecture/decisions/`.
|
||||
- **2026-02-11**: CLI syntax updated to spec-aligned forms (`action create` via `--config`, `plan use` positional args, `project create` positional). Legacy `--name`/`--project` examples retained in historical notes only.
|
||||
- **2026-02-11**: Action config YAML baseline (spec-aligned):
|
||||
```yaml
|
||||
name: local/example-action
|
||||
description: Example action for CLI flows
|
||||
strategy_actor: openai/gpt-4
|
||||
execution_actor: openai/gpt-4
|
||||
definition_of_done: "All steps complete"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Runtime Foundation and Ongoing Development
|
||||
|
||||
### Legacy Phase 2 Work (Pre-Rebaseline)
|
||||
|
||||
**2025-11-22**: Week 12 Complete, Phase 2 Core Functionality DONE
|
||||
|
||||
- CLI Streaming Integration fully implemented.
|
||||
- AutoDebugGraph Implementation complete.
|
||||
- Mock provider enhancements with configurable failure modes.
|
||||
- Infrastructure improvements (DebugAttempt model, repositories).
|
||||
- 2026-02-11: CLI syntax refreshed to spec-aligned forms in current execution sections; legacy `--name`/`--project` examples retained only in historical notes.
|
||||
|
||||
**2025-12-05**: LangSmith observability integration complete.
|
||||
|
||||
**2025-12-08**: Actor-first provider wiring complete.
|
||||
|
||||
**2025-12-17**: Stage 7 performance optimization complete.
|
||||
|
||||
**2026-02-02**: Stage 7.5 Actor Configuration System complete.
|
||||
|
||||
### Plan and Action Domain Models (A1-A4)
|
||||
|
||||
**2026-02-05**: Stage A1 & A2 Complete - Plan and Action Domain Models
|
||||
|
||||
- Created `Plan` model with `PlanPhase` enum (ACTION, STRATEGIZE, EXECUTE, APPLY, APPLIED), `ActionState` enum (AVAILABLE, DRAFT, ARCHIVED), `ProcessingState` enum (QUEUED, PROCESSING, ERRORED, COMPLETE, CANCELLED), `NamespacedName` model with `parse()` and `str()` methods, `PlanIdentity` model with ULID validation, `can_transition()` function for phase transition validation.
|
||||
- Created `Action` model with `ActionArgument` model with `parse()` method for CLI argument parsing, argument validation including type checking.
|
||||
- Added 52 Behave test scenarios across 2 feature files: `plan_model.feature` (30 scenarios), `action_model.feature` (22 scenarios).
|
||||
|
||||
**2026-02-05**: Stage A3 Complete - PlanLifecycleService
|
||||
|
||||
- Created `PlanLifecycleService` with full plan lifecycle management (Action -> Strategize -> Execute -> Apply -> Applied), Action CRUD operations, phase transition methods, state management, custom exceptions (`InvalidPhaseTransitionError`, `ActionNotAvailableError`, `PlanNotReadyError`).
|
||||
- Added python-ulid dependency for ULID generation.
|
||||
- Total new test scenarios: 81 (30 + 22 + 29).
|
||||
|
||||
**2026-02-05**: Stage A4 In Progress - Plan CLI Commands
|
||||
|
||||
- Created `action.py` CLI with `action create`, `action list`, `action show`, `action available`, `action archive` commands.
|
||||
- Extended `plan.py` with v3 lifecycle commands: `plan use`, `plan execute`, `plan apply`, `plan status`, `plan list`, `plan cancel`.
|
||||
- Total test scenarios: 96 (81 + 15).
|
||||
|
||||
### Critical Architectural Decision
|
||||
|
||||
**2026-02-06**: CRITICAL ARCHITECTURAL DECISION - Tool-Based Resource Modification
|
||||
|
||||
- **REPLACED**: OutputParser/code fence parsing approach.
|
||||
- **WITH**: Tool-based change tracking (modern approach used by Claude Code, Cursor, Aider).
|
||||
- **Key changes**:
|
||||
- LLMs call skills/tools directly (edit_file, write_file, delete_file, etc.).
|
||||
- Skills operate on sandbox state directly.
|
||||
- ChangeSet is built from skill invocation history, NOT by parsing LLM text output.
|
||||
- Added built-in resource skills (now C4.file/C4.search/C4.git): file ops, dir ops, search, git ops.
|
||||
- Added MCP skill adapter (now C7.mcp): connect to external MCP servers.
|
||||
- Replaced C4 "Multi-File ChangeSet Generation" with "Tool-Based Change Tracking".
|
||||
- Added SkillInvocationTracker and ToolCallRouter components.
|
||||
- **Rationale**:
|
||||
- No parsing ambiguity (is this code or explanation?).
|
||||
- Each operation is explicit, typed, and trackable.
|
||||
- Supports rollback (replay inverse of recorded changes).
|
||||
- Resource-agnostic (works for files, databases, APIs, any resource type).
|
||||
- Compatible with MCP standard for external tools.
|
||||
|
||||
### Quality Automation Setup (Q0-Q2) [Brent]
|
||||
|
||||
**2026-02-09**: Task Q0.6b Complete - README.md Setup Instructions
|
||||
|
||||
- Updated README.md Quick Start: added `dev` extras to `pip install`, added `scripts/setup-dev.sh` step.
|
||||
- Updated README.md Developing section: fixed `oxt` typo -> `nox`, added quality/security nox sessions, linked to `docs/development/quality-automation.md`.
|
||||
|
||||
**2026-02-09**: Ruff Cleanup in src/cleveragents/ - StrEnum Migration
|
||||
|
||||
- Fixed all 26 ruff findings (all UP042: `str, Enum` -> `StrEnum`) + 12 consequent F401 unused `Enum` imports.
|
||||
- Migrated 26 enum classes across 15 files from `class Foo(str, Enum)` to `class Foo(StrEnum)`.
|
||||
- `StrEnum` (Python 3.11+) is the modern replacement; project targets Python 3.13.
|
||||
- Semantic difference: `str(StrEnum.MEMBER)` returns the value (e.g., `"foo"`) rather than `"ClassName.MEMBER"` — this is the correct/intended behavior for config/JSON string enums.
|
||||
- Verified: no code uses `str()` on enum members in the old format; all tests pass (304 scenarios, 0 failures).
|
||||
|
||||
**2026-02-09**: Task Q0.9 Complete - Ruff Lint Findings in features/
|
||||
|
||||
- Fixed all 200 ruff lint findings in `features/` directory -> **0 findings**.
|
||||
- **Config-level suppressions** (168 findings): Added `per-file-ignores` in `pyproject.toml` for Behave-specific patterns (F811 for redefined `step_impl`, E501 for long step decorator strings).
|
||||
- **Manual fixes** (31 findings across 18 files): SIM115, UP028, SIM117, RUF005, B904, RUF012, SIM105, B007, B018, F821, SIM102, I001.
|
||||
|
||||
**2026-02-09**: Task Q0.8 Complete - Bandit Security Findings Remediation
|
||||
|
||||
- Fixed all 16 pre-existing bandit findings (2 HIGH, 3 MEDIUM, 11 LOW) -> **0 findings**.
|
||||
- **Security hardening** (HIGH+MEDIUM): Replaced `jinja2.Environment` with `jinja2.sandbox.SandboxedEnvironment`, added `_validate_code_ast()` helper for AST-based pre-validation, added `_validate_lambda_ast()` helper restricting `eval()` to lambda-only expressions.
|
||||
- **Code quality** (LOW): Replaced 6 `assert` statements with proper `if`/`raise`, replaced `try/except/pass` with `contextlib.suppress(Exception)`, added logging to silent exception handlers.
|
||||
|
||||
**2026-02-09**: Stages Q0, Q1, Q2 Complete - Full Quality Automation Setup
|
||||
|
||||
- **Stage Q0 - Pre-commit Hooks**: Created `.pre-commit-config.yaml` with 12 hooks across 5 categories (branch protection, general checks, Ruff, Pyright, Bandit, Vulture, Semgrep, Commitizen).
|
||||
- **Discovery**: CI platform is Forgejo (`.forgejo/`), NOT GitHub. Stage Q1 must use Forgejo Actions.
|
||||
- **Discovery**: 37 source files have formatting issues, ~27 files have trailing whitespace - pre-existing debt.
|
||||
- **Discovery**: `features/steps/actor_cli_steps.py` has many F811 (redefined step_impl) violations - Behave pattern.
|
||||
- **Discovery**: Average code complexity is A (3.56) across 979 blocks - good baseline.
|
||||
- **Discovery**: High complexity methods identified: `LegacyDataMigrator.migrate_project_data` E(37), `ProviderRegistry._create_provider_llm` C(20), `ProviderRegistry.create_ai_provider` C(18).
|
||||
- **Stage Q1 - CI/CD Pipeline**: Extended `.forgejo/workflows/ci.yml` with security, quality, coverage jobs. Created `scripts/check-quality-gates.py`.
|
||||
- **Stage Q2 - Advanced Automation**: Created nightly quality monitoring workflow, ADR compliance checker, PR template. New nox sessions: `pre_commit`, `security_scan`, `dead_code`, `complexity`, `adr_compliance`.
|
||||
|
||||
**2026-02-10**: Task 10B.4 - Quality Metrics Baseline Established [Brent]
|
||||
|
||||
- Ran full quality suite via nox to establish current baseline:
|
||||
- **Unit Tests**: 105 features, 1613 scenarios, 7555 steps - ALL PASS
|
||||
- **Lint (ruff)**: 0 findings
|
||||
- **Typecheck (pyright)**: 0 errors, 0 warnings
|
||||
- **Security (bandit)**: 0 findings
|
||||
- **Dead Code (vulture)**: 0 findings
|
||||
- **Complexity (radon)**: Average A (3.56), 981 blocks analyzed, no grade-F methods
|
||||
- **Coverage**: 96% (9860 statements, 269 missing, 2852 branches, 213 branch-miss)
|
||||
- Fixed pre-existing test failure: Rich table column wrapping at narrow terminal widths. Fixed by patching console width to 200 in test setup.
|
||||
- Fixed missing dependency: added `langchain-anthropic>=0.2.0` to `pyproject.toml`.
|
||||
|
||||
**2026-02-10**: Task Q1.5 Complete - Branch Protection Rules Documentation [Brent]
|
||||
|
||||
- Created `docs/development/ci-cd.md` (224 lines) documenting branch protection rules, review priority matrix, CI job dependency graph, quality gates summary.
|
||||
|
||||
### Edge Case and Validation Tests [Brent]
|
||||
|
||||
**2026-02-10**: Task 10C.1 Complete - Edge Case Test Scenarios
|
||||
|
||||
- Created `features/edge_case_plan_scenarios.feature` (26 scenarios, 141 steps) covering concurrent plan execution, resource conflict scenarios, validation failure chains, rollback edge cases.
|
||||
|
||||
**2026-02-10**: Task 10C.4 Complete - Validation Test Fixtures
|
||||
|
||||
- Created `features/validation_test_fixtures.feature` (34 scenarios, 81 steps) covering AST security validation, lambda AST validation, Python content sanitization, project model validation, change list coercion, ActionArgument parsing.
|
||||
- Fixed step name collisions with existing step files.
|
||||
- Fixed irrecoverable syntax scenario: `"def broken("` is actually recoverable via docstring wrapping; replaced with null byte input which is truly irrecoverable.
|
||||
|
||||
### CI/CD Pipeline Stabilization [Brent]
|
||||
|
||||
**2026-02-12**: Task Q0-min-ci In Progress - Nox-Based PR Validation Workflow
|
||||
|
||||
- Rewrote `.forgejo/workflows/ci.yml` to route ALL jobs through nox sessions instead of direct tool invocations.
|
||||
- **Coverage boost 92%->97%**: Wrote 108 new Behave scenarios covering 6 largest coverage gaps.
|
||||
- **Key discovery**: Behave loads ALL step definition files globally; any `@given/@when/@then` pattern matching an existing one causes `AmbiguousStep` errors.
|
||||
|
||||
**2026-02-12**: Bugfix - Integration Test Failure in Robot.Actor Configuration
|
||||
|
||||
- **Root cause**: `robot/ci_nox_validation.robot` ran `nox --list` during integration tests. When `pabot` executed all Robot suites in parallel, the `nox --list` stdout leaked into the sibling `robot/actor_configuration.robot` process, causing `JSONDecodeError`.
|
||||
- **Fix**: Added `slow` tag to interfering test cases; replaced hardcoded venv paths with `python`.
|
||||
|
||||
**2026-02-12**: Bugfix - CI `security` Job Failure (session name mismatch)
|
||||
|
||||
- **Root cause**: CI referenced `nox -s security_scan` and `nox -s dead_code`, but the noxfile only had a session named `security`.
|
||||
- **Fix**: Renamed `security` to `security_scan`, added standalone `dead_code` session.
|
||||
|
||||
**2026-02-12**: Bugfix - CI `unit_tests` Failure: Rich Console Line-Wrapping
|
||||
|
||||
- **Root cause**: Rich's `Console` wraps output at 80 columns. On CI, temp directory paths exceed 80 chars, splitting filenames across lines.
|
||||
- **Fix**: Collapse Rich line-wraps via `clean_output.replace("\n", "")` before assertions.
|
||||
|
||||
**2026-02-12**: Bugfix - CI `integration_tests` Failures (3 categories)
|
||||
|
||||
- **Issue 1**: `ModuleNotFoundError` — hardcoded `/app` path doesn't exist on CI. Fixed with `${CURDIR}/..` relative paths.
|
||||
- **Issue 2**: `--load-context` help text assertion failures — Rich output goes to stderr when no TTY. Fixed with `stderr=STDOUT` merge.
|
||||
- **Issue 3**: `FileNotFoundError: robot` — pabot subprocess spawning exhausts container resources. Restored `--exclude discovery` flag.
|
||||
|
||||
**2026-02-13**: Bugfix - CI `security` Job Failure (missing `build/` directory)
|
||||
|
||||
- **Root cause**: `build/` directory does not exist in fresh CI checkout. Bandit cannot create intermediate directories.
|
||||
- **Fix**: Added `os.makedirs("build", exist_ok=True)` before bandit invocation.
|
||||
|
||||
**2026-02-13**: Bugfix - CI `integration_tests` Failures (3 categories)
|
||||
|
||||
- **Issue 1**: `FileNotFoundError: robot` after ~17 suites — container resource exhaustion. Capped pabot parallelism to `min(default, 2)`, explicitly prepend venv `bin/` to PATH.
|
||||
- **Issue 2**: Rich ANSI escape codes split `--load-context` string. Added `env:NO_COLOR=1`.
|
||||
- **Issue 3**: `initial_next_command_test.robot` exit code 3 — missing `OPENAI_API_KEY`. Tagged as `slow`.
|
||||
|
||||
**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 2 (pabot resource exhaustion + ANSI codes)
|
||||
|
||||
- `robot` is findable at session start yet pabot still fails after ~24 suites even with `--processes 2`.
|
||||
- **Root cause confirmed**: `FileNotFoundError` from `subprocess.Popen` on Linux occurs when the system cannot `execve()` due to exhausted PIDs or file descriptors, not because the binary is missing.
|
||||
- **Fix**: Replaced `pabot` with `robot` (sequential execution). CI values reliability over speed.
|
||||
- `NO_COLOR=1` insufficient — Rich still embeds non-color ANSI sequences (bold, reset). Added `env:TERM=dumb`.
|
||||
|
||||
**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 3 (venv PATH, Resource paths, hanging tests)
|
||||
|
||||
- **Issue 1**: `python -m cleveragents` uses system Python — venv PATH line was accidentally removed. Restored.
|
||||
- **Issue 2**: Robot `Resource` file resolution fails on CI — bare relative references. Converted all to `${CURDIR}/` absolute paths across 30 robot files.
|
||||
- **Issue 3**: `rxpy_route_validation.robot` tests hang — missing `timeout` parameters on `Run Process` calls. Added `timeout=30s` to all 9 calls.
|
||||
|
||||
**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 4 (duplicate Settings blocks, venv Python injection, CI debug)
|
||||
|
||||
- **Issue 1**: Duplicate `*** Settings ***` blocks in 14 robot files created by sed-based conversion. Merged into single blocks.
|
||||
- **Issue 2**: Bare `python` in `Run Process` calls resolves to system Python on CI. Noxfile now passes `--variable PYTHON:<venv-python-path>` to `robot`. All 14 files updated to use `${PYTHON}`.
|
||||
- **Issue 3**: Hardcoded `/app/src` path in `system_prompt_template_rendering.robot`. Removed (venv Python already has package installed).
|
||||
|
||||
**2026-02-13**: Bugfix - CI `integration_tests` Failures Round 5 (guard cleanup when OpenAI key missing)
|
||||
|
||||
- **Root cause**: `robot/scientific_paper_basic.robot` suite setup skips on missing `OPENAI_API_KEY`, leaving `${CONTEXT_DIR}` empty. Teardown's `Remove Directory ${CONTEXT_DIR} recursive=True` resolves to CWD, deleting the repo root (including `.nox` venv and `robot/*.resource` files), causing all downstream suites to fail.
|
||||
- **Fix**: Initialize `${CONTEXT_DIR}` to `${TEMPDIR}/paper_basic_contexts` and guard cleanup with empty check.
|
||||
|
||||
**2026-02-13**: Enhancement - Restore parallel Robot runs + silence discovery resource warning
|
||||
|
||||
- Reintroduced `pabot` with conservative parallelism (max 2 processes) now that teardown bug is fixed.
|
||||
- Added `robot/discovery_common.resource` stub to silence non-fatal import warning.
|
||||
|
||||
### Coverage and Security Enforcement [Brent]
|
||||
|
||||
**2026-02-13**: Task Q0-min-coverage In Progress - Coverage Threshold Enforcement
|
||||
|
||||
- Enhanced `coverage_report` nox session: `COVERAGE_THRESHOLD = 97` constant, generates HTML/XML/JSON reports before checking threshold, emits CI-parseable summary line.
|
||||
- Created `docs/development/testing.md` (180+ lines): full testing guide covering unit (Behave), integration (Robot), benchmarks (ASV), and coverage.
|
||||
- Created `features/coverage_threshold_enforcement.feature` (11 scenarios).
|
||||
- Verification: 2235 unit scenarios passed, 211 integration tests passed, 97.5% coverage.
|
||||
|
||||
**2026-02-13**: Task Q0-adv-security In Progress - Align Security Scans with Nox
|
||||
|
||||
- Added Semgrep as Step 3 in `security_scan` session (between Bandit and Vulture).
|
||||
- Changed semgrep hook from wrapper script to direct invocation.
|
||||
- Updated quality-automation docs with revised thresholds (85->97%).
|
||||
|
||||
### Plan Model Alignment [Luis]
|
||||
|
||||
**2026-02-13**: Stage A2b.beta Complete - Plan Model Alignment
|
||||
|
||||
- Rewrote `plan.py` to align with spec: `processing_state` replaces `action_state`/`state`, `project_links` replaces `project_ids`, added `action_name`, `automation_profile`, `invariants`, actor fields, subplan hierarchy.
|
||||
- Rebased `feature/m1-plan-model` onto A2b.alpha commit. Resolved 30 merge conflicts. Key design decisions during merge:
|
||||
1. Plan field name: `processing_state` (per spec), NOT `state`.
|
||||
2. Invariant enum: `InvariantSource` (our naming), NOT `InvariantScope`.
|
||||
3. Action model: HEAD taken as-is (authoritative for action domain).
|
||||
4. No `action_id` on Action domain model — uses `namespaced_name` only.
|
||||
5. `project_links` used everywhere (NOT `project_ids`).
|
||||
- Fixed 6 post-rebase test failures.
|
||||
- Final results: 130 Behave features / 2246 scenarios / 9868 steps all pass, 206 Robot tests all pass, `plan.py` coverage 100%.
|
||||
|
||||
### Persistence Layer (A5)
|
||||
|
||||
**2026-02-09**: Stage A5.3 + A5.4 Complete - v3 SQLAlchemy Models
|
||||
|
||||
- Created `LifecycleActionModel` (table: `actions_v3`) and `LifecyclePlanModel` (table: `lifecycle_plans`) with full spec alignment.
|
||||
- Used `__allow_unmapped__ = True` on both models for `from __future__ import annotations` compatibility.
|
||||
- **Legacy note**: These v3 ORM models use `action_id`/`project_ids` and draft-style fields; they are superseded by the A5 rebaseline.
|
||||
|
||||
**2026-02-09**: Stage A5.6 Complete - ActionRepository (Luis tasks A5.6a-A5.6j)
|
||||
|
||||
- Created `ActionRepository` with session-factory pattern, 10 methods, `@database_retry` on all public methods.
|
||||
- All mutating methods flush (do NOT commit) — caller/UnitOfWork handles commit.
|
||||
- Uses `Any` type hints for domain Action objects to avoid circular import issues.
|
||||
|
||||
### Sandbox Infrastructure [Luis]
|
||||
|
||||
**2026-02-09**: Stage B3.1 + B3.2 + B3.5 + B3.6 + B3.7 + B3.8 Complete - Sandbox Infrastructure
|
||||
|
||||
- Created `src/cleveragents/infrastructure/sandbox/` package with 6 modules: `protocol.py`, `no_sandbox.py`, `factory.py`, `manager.py`, `merge.py`, `__init__.py`.
|
||||
- Design decisions:
|
||||
- Factory accepts raw strings rather than Resource objects (Resource model not yet implemented). This decouples the sandbox layer from the domain model.
|
||||
- `SandboxStatus.assert_transition()` provides a convenient guard that raises `SandboxStateError` on invalid transitions.
|
||||
- `NoSandbox.rollback()` always raises `SandboxRollbackError` rather than being a no-op, because silently swallowing rollback requests for unsandboxed resources would hide bugs.
|
||||
|
||||
**2026-02-09**: Stage A2.1 Already Complete
|
||||
|
||||
- `estimation_actor` field already present in action model. `review_actor` field already present. Also has `apply_actor` (bonus field from earlier work).
|
||||
- `safety_profile` remains DEFERRED to post-30 per plan.
|
||||
|
||||
### Security Hardening (SEC1) [Luis]
|
||||
|
||||
**2026-02-09**: Stage SEC1 Complete - Remove eval() Vulnerability
|
||||
|
||||
- **Audit results** (7 matches in `src/cleveragents/`): 2 real vulnerabilities, 4 false positives.
|
||||
- **Fixes applied**: `SimpleToolAgent` — added named operation registry (`_SAFE_OPERATIONS`), legacy `code` blocks fully rejected. `ReactiveStreamRouter` — added named transform registry, legacy `fn` string expressions that are not in the registry fully rejected. No `eval()` or `exec()` calls remain.
|
||||
|
||||
**2026-02-10**: Stage SEC1.5 Complete - Security BDD tests
|
||||
|
||||
- Created `features/security_eval.feature` with 8 scenarios covering code injection rejection, malicious transform rejection, valid config acceptance, custom registered operations.
|
||||
|
||||
### Subplan Model [Luis]
|
||||
|
||||
**2026-02-09**: Stage E1 Complete - Subplan Model (Luis tasks E1.1-E1.5)
|
||||
|
||||
- Added `ExecutionMode`, `SubplanMergeStrategy` enums, `SubplanConfig`, `SubplanStatus`, `SubplanAttempt` models, `SubplanFailureHandler` class.
|
||||
- Renamed from `MergeStrategy` to `SubplanMergeStrategy` to avoid collision with infrastructure-level `MergeStrategy` protocol.
|
||||
- Used Pydantic BaseModel instead of dataclass (per ADR-004) for `SubplanAttempt` and `SubplanStatus`.
|
||||
|
||||
### Automation Levels [Luis]
|
||||
|
||||
**2026-02-09**: Stage A6 Complete - Automation Levels Foundation
|
||||
|
||||
- Added `AutomationLevel` enum (MANUAL, REVIEW_BEFORE_APPLY, FULL_AUTOMATION).
|
||||
- Updated `PlanLifecycleService` with `should_auto_progress()`, `auto_progress()`, `pause_plan()`, `resume_plan()`, `set_plan_automation_level()`.
|
||||
- `complete_strategize()` and `complete_execute()` now call `auto_progress()` automatically.
|
||||
|
||||
**2026-02-10**: Stage A6.5 Complete - Automation Levels BDD Tests
|
||||
|
||||
- Created `features/automation_levels.feature` with 24 Behave scenarios.
|
||||
- **Discovery**: pydantic-settings `BaseSettings` does not accept constructor keyword overrides for fields with `validation_alias`; must set attribute after construction.
|
||||
|
||||
### Action YAML Schema [Aditya]
|
||||
|
||||
**2026-02-13**: Stage A2b.gamma Complete - Action YAML Schema + Examples + Loader
|
||||
|
||||
- Created `docs/schema/action.schema.yaml`, 5 example action YAMLs, `ActionConfigSchema` Pydantic model with `from_yaml()`/`from_yaml_file()` factory methods, camelCase->snake_case key normalization, `${ENV_VAR}` interpolation, invariant normalization.
|
||||
- 31 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmark suites. Coverage: 98%.
|
||||
|
||||
### Tool and Skill Domain Models [Jeff]
|
||||
|
||||
**2026-02-13**: Stage C0.domain Complete - Tool and Validation Domain Models
|
||||
|
||||
- Created `tool.py` (624 lines) with 6 StrEnums, 5 Pydantic models, full validators, factory methods.
|
||||
- Key decision: YAML loader implemented as `from_config()` class methods (matches action.py pattern) rather than separate module.
|
||||
- Key decision: `Validation` extends `Tool` via Pydantic model inheritance.
|
||||
- Key decision: `wraps` field sets source to `wrapped` implicitly; `transform` required when `wraps` is set.
|
||||
|
||||
**2026-02-13**: Stage C0.skill.domain Complete - Skill Domain Model and Resolver
|
||||
|
||||
- Created `skill.py` (475 lines) with 9 Pydantic models. `SkillResolver.resolve()` flattens includes, de-duplicates tools, rejects cycles with path traces.
|
||||
- Key decision: `SkillResolver` is a standalone class (not a method on Skill) to support registry-backed resolution.
|
||||
- Key decision: `ResolvedToolEntry` includes source tracking for compiler diagnostics.
|
||||
|
||||
### Resource Registry [Jeff]
|
||||
|
||||
**2026-02-13**: Stage B1.core.db Complete - Resource Registry Database Tables
|
||||
|
||||
- Created Alembic migration with 3 tables: `resource_types`, `resources`, `resource_edges`.
|
||||
- Key design decision: `resource_types` PK is the namespaced name string (not ULID) since types are schema-level definitions with stable identifiers.
|
||||
- Key design decision: `link_type` enum ('contains', 'references', 'derived_from') covers hierarchical containment, cross-layer references, and derived virtual resources.
|
||||
- Key design decision: `auto_discovered` flag on both resources and edges tracks provenance for cleanup/refresh operations.
|
||||
|
||||
### Session Domain [Jeff]
|
||||
|
||||
**2026-02-13**: Stage A7.domain Complete - Session Domain Models and Contracts
|
||||
|
||||
- Created `session.py` with `MessageRole` enum, `SessionMessage`, `SessionTokenUsage`, `Session` models, error classes, `SessionService` ABC.
|
||||
- Key decision: SessionService is an ABC (not Protocol) to enforce explicit inheritance and make contract violations visible at class definition time.
|
||||
- Key decision: Messages are ordered by `sequence_number` (not timestamp) to guarantee stable ordering across serialization boundaries.
|
||||
|
||||
### CLI Spec Alignment [Jeff]
|
||||
|
||||
**2026-02-14**: Stage A4b.action Complete - Action CLI Spec Alignment
|
||||
|
||||
- Rewrote `action.py`: config-only `action create --config <file>`, removed `action available` command, added filters to `action list`.
|
||||
|
||||
**2026-02-14**: Stage A4b.plan Complete - Plan CLI Flag Alignment
|
||||
|
||||
- Updated `plan.py`: `plan use` now accepts multiple PROJECT args, `--automation-profile`, `--invariant` (repeatable), actor overrides, `--arg name=value`. Aligned `plan list` filters. Added `--reason` to `plan cancel`.
|
||||
|
||||
**2026-02-14**: Stage A4b.outputs Complete - CLI Output Format Stabilization
|
||||
|
||||
- Created `formatting.py`: shared `format_output()` helper with `OutputFormat` enum, JSON/YAML/plain/table/rich serializers. Handles datetime, Enum, nested structures.
|
||||
|
||||
### Tool Runtime and Built-ins [Jeff]
|
||||
|
||||
**2026-02-14**: Stage C0.runtime Complete - Tool Runtime Core
|
||||
|
||||
- Created `runtime.py` (ToolSpec, ToolResult, ToolError), `registry.py` (thread-safe ToolRegistry with RLock), `runner.py` (ToolRunner with 4-stage lifecycle: discover/activate/execute/deactivate).
|
||||
- New tool modules have 100% coverage.
|
||||
|
||||
**2026-02-14**: Stage C0.files Complete - Built-in File Tools
|
||||
|
||||
- Created `src/cleveragents/tool/builtins/` package with 6 file tools (file-read/write/edit/delete/list/search), path traversal prevention, ChangeSetEntry/ChangeSet/ChangeSetCapture models.
|
||||
- 2026-02-14: Commit traceability sweep found no matching git log entries for skill schema/examples, skill CLI, provider actors, MCP config/adapter, actor compiler, or skill registry persistence; Section 2B traceability tasks remain open.
|
||||
|
||||
### Tool and Session Persistence [Luis]
|
||||
|
||||
**2026-02-14**: Stage C1.tool.registry Complete - Tool Registry Persistence
|
||||
|
||||
- Created Alembic migration with 3 tables: `tools`, `tool_resource_bindings`, `validation_attachments`.
|
||||
- Created `ToolRegistryRepository` (5 methods), `ValidationAttachmentRepository` (4 methods), `ToolRegistryService` (8 methods).
|
||||
- Key decision: `ToolModel.to_domain()` returns a dict (not a domain object) to decouple persistence from domain model until full integration is complete.
|
||||
- Key decision: `ToolRegistryRepository.delete()` pre-checks for active `ValidationAttachmentModel` rows before allowing deletion.
|
||||
- Key decision: Validation attachments are resource-scoped with optional project_name and plan_id narrowing.
|
||||
|
||||
**2026-02-15**: Stage A7.persistence Complete - Session Persistence and Repositories
|
||||
|
||||
- Created Alembic migration with 2 tables: `sessions`, `session_messages`.
|
||||
- Created `SessionRepository` (5 methods), `SessionMessageRepository` (3 methods), `PersistentSessionService` (8 methods).
|
||||
- Key decision: `PersistentSessionService` extends the domain `SessionService` ABC.
|
||||
- Key decision: `append_message()` auto-calculates sequence number and updates parent session's `updated_at` timestamp.
|
||||
- Key decision: `import_session()` validates schema version and SHA-256 checksum before persisting, then generates fresh ULIDs to avoid ID collisions.
|
||||
- Key decision: `update_token_usage()` uses cumulative addition (+=) rather than replacement.
|
||||
|
||||
**2026-02-15**: Bugfix - Benchmark Unique Name Constraint [Luis]
|
||||
|
||||
- ASV benchmark suites that create tools/resources now use unique names per iteration to avoid UNIQUE constraint failures when ASV runs multiple iterations.
|
||||
|
||||
### Skill and Actor YAML [Aditya]
|
||||
|
||||
**2026-02-16**: C0.skill.schema Complete - Skill YAML Schema + Examples + Loader
|
||||
|
||||
- Created `docs/schema/skill.schema.yaml`, 5 example skill YAMLs, `SkillConfigSchema` Pydantic model with `from_yaml()`/`from_yaml_file()` factory methods, camelCase->snake_case normalization, `${ENV_VAR}` interpolation.
|
||||
- 36 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmarks.
|
||||
|
||||
**2026-02-17**: Task C1.schema Complete - Actor YAML Schema Models
|
||||
|
||||
- Created `src/cleveragents/actor/schema.py` (695 lines) with three actor types (LLM, TOOL, GRAPH), graph topology validation with cycle detection using DFS, type-specific field requirements, environment variable interpolation.
|
||||
- 5 example actor YAMLs, 50+ Behave scenarios, 11 Robot test cases, 8 ASV benchmark classes.
|
||||
|
||||
**2026-02-17**: C0.skill.cli Complete - Skill CLI Commands + Output Formatting
|
||||
|
||||
- Created `SkillService` with dual-mode storage pattern (in-memory fallback, ready for `UnitOfWork` persistence).
|
||||
- Created `skill.py` Typer command group with 5 subcommands (`skill add`, `remove`, `list`, `show`, `tools`), all supporting 6 output formats.
|
||||
- Key decision: Module-level `_service` singleton (not DI container) matches the pattern in `action.py` CLI; `_reset_skill_service()` enables test isolation.
|
||||
|
||||
### Bugfixes and Merges
|
||||
|
||||
**2026-02-17**: Bugfix - Unit Test and Benchmark Failures After Master Merge [Brent]
|
||||
|
||||
- **Unit test failure** — Session mismatch in project repository steps. The `NamespacedProjectRepository` creates a new SQLAlchemy session per method call. Fix: Changed session factory to always return the same session instance.
|
||||
- **Benchmark failures** — 5 distinct issues across 7 files: missing parameterized setup params, renamed `PlanPhase.APPLIED` -> `APPLY`, per-iteration plan creation, batch counters for unique IDs across ASV iterations, simplified action reuse.
|
||||
|
||||
**2026-02-18**: Task C1.examples Complete - Actor YAML Examples and Documentation
|
||||
|
||||
- Created `docs/reference/actors_examples.md` (1098 lines) with pattern-based organization (Strategist, Executor, Reviewer, Tool-Only, Validation-Node, Graph, Hierarchical).
|
||||
- 25 Behave scenarios, 16 Robot test cases, 7 ASV benchmarks.
|
||||
- Bug fixes: Missing `description` fields in test YAML strings, `context_view` step definition conflict, enhanced parallel execution detection.
|
||||
|
||||
### Actor Loader [Aditya]
|
||||
|
||||
**2026-02-19**: Task C2.loader Complete - Actor Registry and Loader
|
||||
|
||||
- Created `src/cleveragents/actor/loader.py` (277 lines) — `ActorLoader` class with thread-safe discovery, SHA-256 content-hash caching, and namespace management.
|
||||
- 23 Behave scenarios, 4 Robot test cases, 4 ASV benchmarks. Coverage: 98%.
|
||||
- Key design decisions: SHA-256 content hashing (not mtime) for deterministic cache invalidation, single consolidated `ValidationError` for duplicates/errors, tool reference resolution at load time via optional `ToolRegistry`, `local/` default namespace applied during YAML parsing.
|
||||
|
||||
### Plan Checklist Rebaseline
|
||||
|
||||
**2026-02-13**: Plan Update - Rebaseline A5 Persistence Checklist
|
||||
|
||||
- Rewrote A5 plan persistence checklist to replace legacy tasks with spec-aligned rebaseline migrations, ORM mappings, repositories, and persistence tests.
|
||||
|
||||
**2026-02-13**: Plan Update - A5 Ownership + Validation Attachments
|
||||
|
||||
- Reassigned A5.alpha migrations to Hamza to align DB/migration expertise.
|
||||
- Rebased tool registry tasks to make validation attachments explicitly resource-scoped with required/informational mode.
|
||||
|
||||
**2026-02-13**: Plan Update - Consolidate Skills/Actors/MCP + B0 DB Ownership
|
||||
|
||||
- Retired duplicate C2/C3/C4 blocks; canonical tasks consolidated under C0.skill, C1/C2, C7.mcp, C8.providers.
|
||||
- Reassigned B0.db resource/project migrations to Hamza.
|
||||
|
||||
**2026-02-13**: Plan Update - Tool Runtime Naming Consolidation
|
||||
|
||||
- Renamed M1 tool runtime block to C0 runtime core and M3 lifecycle block to C0.lifecycle to avoid duplicate naming.
|
||||
|
||||
**2026-02-13**: Plan Update - Spec Alignment Pass
|
||||
|
||||
- Updated A4b CLI to use `processing_state` terminology.
|
||||
- Corrected skill registry dependency references.
|
||||
|
||||
**2026-02-13**: Plan Update - C1 Numbering Disambiguation
|
||||
|
||||
- Renamed tool registry group to C1.tool to avoid collision with actor schema C1 group.
|
||||
|
||||
### Change Tracking and Tool Router
|
||||
|
||||
**2026-02-19**: Stage C5.model COMMIT Complete - ChangeSet models and invocation tracker [Luis]
|
||||
|
||||
- Added `ToolInvocation` Pydantic model, `InvocationTracker` Protocol and `InMemoryInvocationTracker`, per-change field validators, `sorted_entries()` and `grouped_by_resource()` methods, `normalize_change_path()` utility.
|
||||
- 21 BDD scenarios, 5 integration smoke tests, 4 ASV benchmark suites.
|
||||
|
||||
**2026-02-19**: Stage C4.git COMMIT Complete - Add git operation skills [Luis]
|
||||
|
||||
- Created `git_tools.py`: 4 read-only git tools (status, diff, log, blame) with safe environment variables, path traversal guards, and user-friendly error mapping for 5 common git failure patterns.
|
||||
- 16 BDD scenarios, 5 integration smoke tests, 3 ASV benchmark suites.
|
||||
|
||||
**2026-02-19**: Combined merge branch fix - Alembic merge migration [Luis]
|
||||
|
||||
- Created merge migration to resolve "Multiple head revisions" error between `a7_001_session_tables` and `c0_001_skill_registry` heads.
|
||||
|
||||
**2026-02-19**: Stage SEC1.eval COMMIT Complete - Remove eval-based config parsing [Luis]
|
||||
|
||||
- Created config security scanner detecting 15 disallowed patterns across YAML/TOML/generic configs. Reports file path + line number + severity for each violation.
|
||||
- Extended security eval feature from 8 to 16 scenarios.
|
||||
|
||||
### Execution Pipeline and Apply
|
||||
|
||||
**2026-02-20**: D0b.execute — Plan Execute via Actors (M1 Critical Blocker)
|
||||
|
||||
- Created `PlanExecutor` orchestrator with `StrategizeStubActor` (read-only, produces decisions) and `ExecuteStubActor` (sandbox resources, routes tool calls).
|
||||
- Strategize phase is read-only. Execute phase uses sandbox resources and routes tool calls through ToolRunner.
|
||||
- Phase guards prevent Execute when Plan is not in Strategize COMPLETE state.
|
||||
- Streaming hooks emit interim status updates.
|
||||
|
||||
**2026-02-20**: D0b.apply Complete - Diff Review + Apply Integration [Jeff]
|
||||
|
||||
- Created `PlanApplyService` with `diff()`, `artifacts()`, `persist_apply_summary()`, `handle_merge_failure()`, `guard_empty_changeset()`.
|
||||
- Key decision: Used `error_details` dict on Plan as general metadata store for apply summary fields until a dedicated metadata field lands.
|
||||
- Key decision: Service accepts optional `changeset_store` — when None, builds stub SpecChangeSet from plan metadata only.
|
||||
|
||||
**2026-02-20**: C5.router Complete - Tool Call Router for LLM Providers [Jeff]
|
||||
|
||||
- Created `router.py` (~890 lines) — `ToolCallRouter` translating between LLM provider tool call formats and internal `ToolRunner`: provider format detection, normalization, deterministic ID generation, error classification, schema adaptation, batch routing, streaming support.
|
||||
- Key discovery: `ToolRunner.execute()` catches handler exceptions internally and returns `ToolResult(success=False)` rather than raising, so streaming produces COMPLETE + failed result (not ERROR update) for tool failures.
|
||||
- Key discovery: Pyright strict mode requires all Pydantic model fields (even those with defaults via `Field(default=...)`) to be explicitly passed in constructor calls.
|
||||
- Key discovery: Behave step name collision with existing `coverage_boost_extra_steps.py`; renamed to avoid `AmbiguousStep` error.
|
||||
|
||||
### A9.service: Config Service with Multi-Level Resolution
|
||||
|
||||
- **2026-02-23** [Jeff]: Implemented `ConfigService` in `src/cleveragents/application/services/config_service.py` with:
|
||||
- Five-level resolution chain: CLI flag > env var > project-scoped > global config > default
|
||||
- Typed config key registry with 24 entries across 6 sections: `core.*` (7), `plan.*` (3), `provider.*` (4), `sandbox.*` (3), `context.*` (3), `index.*` (4)
|
||||
- TOML file management (`~/.cleveragents/config.toml`) with auto-created parent directories via `tomlkit`
|
||||
- `ResolvedValue` return type includes winning value, source level, and optional verbose chain
|
||||
- Env var interpolation following `CLEVERAGENTS_<SECTION>_<KEY>` convention
|
||||
- Validation: unknown keys raise `ValueError` with sample of valid keys; type mismatches raise `TypeError` with actionable messages
|
||||
- Boolean coercion from string: true/false, 1/0, yes/no (case-insensitive)
|
||||
- **2026-02-23** [Jeff]: Added `docs/reference/config_resolution.md` documenting all 24 config keys with types, defaults, env var mappings, and precedence rules
|
||||
- **2026-02-23** [Jeff]: Added `features/config_resolution.feature` with 31 Behave scenarios covering every resolution level, env var overrides, type coercion, unknown key rejection, TOML roundtrips, and verbose chain output
|
||||
- **2026-02-23** [Jeff]: Added `robot/config_resolution.robot` with 8 Robot Framework end-to-end tests exercising default/global/env/CLI resolution, unknown key rejection, type coercion, and env var convention
|
||||
- **2026-02-23** [Jeff]: Added `benchmarks/config_resolution_bench.py` with 6 ASV benchmark suites for resolution performance at each level, bulk operations, and validation throughput
|
||||
- **2026-02-23** [Jeff]: Exported `ConfigService`, `ConfigEntry`, `ConfigLevel`, `ResolvedValue` from `application.services.__init__`; updated vulture whitelist for new public API
|
||||
|
||||
### CLI1.alpha: Plan/Action CLI Extensions
|
||||
|
||||
- **2026-02-23** [Jeff]: Implemented CLI1.alpha. Key changes:
|
||||
- Added `validate_namespaced_actor()` to `plan.py` for strict `namespace/name` validation on actor override flags (`--strategy-actor`, `--execution-actor`, `--estimation-actor`, `--invariant-actor`)
|
||||
- Updated `_plan_spec_dict()` to include `estimation_actor`, `invariant_actor`, and `invariants` fields in all output formats (json, yaml, plain, table)
|
||||
- Updated `_print_lifecycle_plan()` rich panel to display invariants with source provenance tags
|
||||
- Updated `plan_status` summary table to include Profile and Invariants columns
|
||||
- Updated `lifecycle_list_plans` table to include Profile and Invariants columns
|
||||
- Extended `action.py` `_action_spec_dict()` to conditionally include `estimation_actor`, `invariant_actor`, and `inputs_schema` fields
|
||||
- Extended `_print_action()` rich panel to display estimation actor, invariant actor, invariants list, inputs_schema, and automation_profile
|
||||
- `--automation-profile` flag on `plan use` validates against `BUILTIN_PROFILES` and persists to plan metadata
|
||||
- `--invariant` flags on `plan use` are passed through to `PlanLifecycleService.use_action()` as `PlanInvariant` objects with `InvariantSource.PLAN`
|
||||
- Created `docs/reference/plan_cli.md` and `docs/reference/action_cli.md` reference documentation
|
||||
- Created `features/cli_extensions.feature` with 25 Behave scenarios covering all new functionality
|
||||
- Created `features/steps/cli_extensions_steps.py` with all step definitions
|
||||
- Created `robot/cli_extensions.robot` with 5 Robot Framework integration tests
|
||||
- Created `benchmarks/cli_extensions_bench.py` with 6 ASV benchmark suites
|
||||
@@ -17,7 +17,6 @@ nav:
|
||||
- Review Playbook: development/review_playbook.md
|
||||
- Scale Testing: development/scale_testing.md
|
||||
- Implementation Timeline: timeline.md
|
||||
- Implementation Notes: implementation-notes.md
|
||||
- FAQ: faq.md
|
||||
- Reference: reference/
|
||||
- Architecture Decision Records (ADRs):
|
||||
|
||||
Reference in New Issue
Block a user