cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity
Files
b1bf2250fb5171d3a5bcb81b00305d473966778d
cleveragents-core/docs/implementation-notes.md
T

37 KiB
Raw Blame History

Implementation Notes

This document is the chronological record of decisions, discoveries, bugfixes, and design notes made during the development of CleverAgents. Each entry is timestamped and attributed to the developer who performed the work. These notes were captured in real time as the implementation progressed and are preserved here as the authoritative development history.

Phase 0: Discovery and Requirements Elaboration

  • 2025-11-01: Implemented CLI inventory extraction tools, server endpoint extraction, data contract extraction, shell asset extraction.
  • 2025-11-02: Implemented environment variable extraction and implicit behavior extraction tools.
  • 2025-11-04: Completed workflow parity matrix generator and cloud features identification. All discovery artifacts stored in docs/reference/.

Phase 1: Target Architecture Definition

  • 2025-11-04: Completed all 10 Architecture Decision Records.
  • 2025-11-05: Package structure created based on ADR-001. All ADRs documented in docs/architecture/decisions/.
  • 2026-02-11: CLI syntax updated to spec-aligned forms (action create via --config, plan use positional args, project create positional). Legacy --name/--project examples retained in historical notes only.
  • 2026-02-11: Action config YAML baseline (spec-aligned): 
    name: local/example-action
description: Example action for CLI flows
strategy_actor: openai/gpt-4
execution_actor: openai/gpt-4
definition_of_done: "All steps complete"

Phase 2: Runtime Foundation and Ongoing Development

Legacy Phase 2 Work (Pre-Rebaseline)

2025-11-22: Week 12 Complete, Phase 2 Core Functionality DONE

  • CLI Streaming Integration fully implemented.
  • AutoDebugGraph Implementation complete.
  • Mock provider enhancements with configurable failure modes.
  • Infrastructure improvements (DebugAttempt model, repositories).
  • 2026-02-11: CLI syntax refreshed to spec-aligned forms in current execution sections; legacy --name/--project examples retained only in historical notes.

2025-12-05: LangSmith observability integration complete.

2025-12-08: Actor-first provider wiring complete.

2025-12-17: Stage 7 performance optimization complete.

2026-02-02: Stage 7.5 Actor Configuration System complete.

Plan and Action Domain Models (A1-A4)

2026-02-05: Stage A1 & A2 Complete - Plan and Action Domain Models

  • Created Plan model with PlanPhase enum (ACTION, STRATEGIZE, EXECUTE, APPLY, APPLIED), ActionState enum (AVAILABLE, DRAFT, ARCHIVED), ProcessingState enum (QUEUED, PROCESSING, ERRORED, COMPLETE, CANCELLED), NamespacedName model with parse() and str() methods, PlanIdentity model with ULID validation, can_transition() function for phase transition validation.
  • Created Action model with ActionArgument model with parse() method for CLI argument parsing, argument validation including type checking.
  • Added 52 Behave test scenarios across 2 feature files: plan_model.feature (30 scenarios), action_model.feature (22 scenarios).

2026-02-05: Stage A3 Complete - PlanLifecycleService

  • Created PlanLifecycleService with full plan lifecycle management (Action -> Strategize -> Execute -> Apply -> Applied), Action CRUD operations, phase transition methods, state management, custom exceptions (InvalidPhaseTransitionError, ActionNotAvailableError, PlanNotReadyError).
  • Added python-ulid dependency for ULID generation.
  • Total new test scenarios: 81 (30 + 22 + 29).

2026-02-05: Stage A4 In Progress - Plan CLI Commands

  • Created action.py CLI with action create, action list, action show, action available, action archive commands.
  • Extended plan.py with v3 lifecycle commands: plan use, plan execute, plan apply, plan status, plan list, plan cancel.
  • Total test scenarios: 96 (81 + 15).

Critical Architectural Decision

2026-02-06: CRITICAL ARCHITECTURAL DECISION - Tool-Based Resource Modification

  • REPLACED: OutputParser/code fence parsing approach.
  • WITH: Tool-based change tracking (modern approach used by Claude Code, Cursor, Aider).
  • Key changes:
    • LLMs call skills/tools directly (edit_file, write_file, delete_file, etc.).
    • Skills operate on sandbox state directly.
    • ChangeSet is built from skill invocation history, NOT by parsing LLM text output.
  • Added built-in resource skills (now C4.file/C4.search/C4.git): file ops, dir ops, search, git ops.
  • Added MCP skill adapter (now C7.mcp): connect to external MCP servers.
    • Replaced C4 "Multi-File ChangeSet Generation" with "Tool-Based Change Tracking".
    • Added SkillInvocationTracker and ToolCallRouter components.
  • Rationale:
    • No parsing ambiguity (is this code or explanation?).
    • Each operation is explicit, typed, and trackable.
    • Supports rollback (replay inverse of recorded changes).
    • Resource-agnostic (works for files, databases, APIs, any resource type).
    • Compatible with MCP standard for external tools.

Quality Automation Setup (Q0-Q2) [Brent]

2026-02-09: Task Q0.6b Complete - README.md Setup Instructions

  • Updated README.md Quick Start: added dev extras to pip install, added scripts/setup-dev.sh step.
  • Updated README.md Developing section: fixed oxt typo -> nox, added quality/security nox sessions, linked to docs/development/quality-automation.md.

2026-02-09: Ruff Cleanup in src/cleveragents/ - StrEnum Migration

  • Fixed all 26 ruff findings (all UP042: str, Enum -> StrEnum) + 12 consequent F401 unused Enum imports.
  • Migrated 26 enum classes across 15 files from class Foo(str, Enum) to class Foo(StrEnum).
  • StrEnum (Python 3.11+) is the modern replacement; project targets Python 3.13.
  • Semantic difference: str(StrEnum.MEMBER) returns the value (e.g., "foo") rather than "ClassName.MEMBER" — this is the correct/intended behavior for config/JSON string enums.
  • Verified: no code uses str() on enum members in the old format; all tests pass (304 scenarios, 0 failures).

2026-02-09: Task Q0.9 Complete - Ruff Lint Findings in features/

  • Fixed all 200 ruff lint findings in features/ directory -> 0 findings.
  • Config-level suppressions (168 findings): Added per-file-ignores in pyproject.toml for Behave-specific patterns (F811 for redefined step_impl, E501 for long step decorator strings).
  • Manual fixes (31 findings across 18 files): SIM115, UP028, SIM117, RUF005, B904, RUF012, SIM105, B007, B018, F821, SIM102, I001.

2026-02-09: Task Q0.8 Complete - Bandit Security Findings Remediation

  • Fixed all 16 pre-existing bandit findings (2 HIGH, 3 MEDIUM, 11 LOW) -> 0 findings.
  • Security hardening (HIGH+MEDIUM): Replaced jinja2.Environment with jinja2.sandbox.SandboxedEnvironment, added _validate_code_ast() helper for AST-based pre-validation, added _validate_lambda_ast() helper restricting eval() to lambda-only expressions.
  • Code quality (LOW): Replaced 6 assert statements with proper if/raise, replaced try/except/pass with contextlib.suppress(Exception), added logging to silent exception handlers.

2026-02-09: Stages Q0, Q1, Q2 Complete - Full Quality Automation Setup

  • Stage Q0 - Pre-commit Hooks: Created .pre-commit-config.yaml with 12 hooks across 5 categories (branch protection, general checks, Ruff, Pyright, Bandit, Vulture, Semgrep, Commitizen).
  • Discovery: CI platform is Forgejo (.forgejo/), NOT GitHub. Stage Q1 must use Forgejo Actions.
  • Discovery: 37 source files have formatting issues, ~27 files have trailing whitespace - pre-existing debt.
  • Discovery: features/steps/actor_cli_steps.py has many F811 (redefined step_impl) violations - Behave pattern.
  • Discovery: Average code complexity is A (3.56) across 979 blocks - good baseline.
  • Discovery: High complexity methods identified: LegacyDataMigrator.migrate_project_data E(37), ProviderRegistry._create_provider_llm C(20), ProviderRegistry.create_ai_provider C(18).
  • Stage Q1 - CI/CD Pipeline: Extended .forgejo/workflows/ci.yml with security, quality, coverage jobs. Created scripts/check-quality-gates.py.
  • Stage Q2 - Advanced Automation: Created nightly quality monitoring workflow, ADR compliance checker, PR template. New nox sessions: pre_commit, security_scan, dead_code, complexity, adr_compliance.

2026-02-10: Task 10B.4 - Quality Metrics Baseline Established [Brent]

  • Ran full quality suite via nox to establish current baseline:
    • Unit Tests: 105 features, 1613 scenarios, 7555 steps - ALL PASS
    • Lint (ruff): 0 findings
    • Typecheck (pyright): 0 errors, 0 warnings
    • Security (bandit): 0 findings
    • Dead Code (vulture): 0 findings
    • Complexity (radon): Average A (3.56), 981 blocks analyzed, no grade-F methods
    • Coverage: 96% (9860 statements, 269 missing, 2852 branches, 213 branch-miss)
  • Fixed pre-existing test failure: Rich table column wrapping at narrow terminal widths. Fixed by patching console width to 200 in test setup.
  • Fixed missing dependency: added langchain-anthropic>=0.2.0 to pyproject.toml.

2026-02-10: Task Q1.5 Complete - Branch Protection Rules Documentation [Brent]

  • Created docs/development/ci-cd.md (224 lines) documenting branch protection rules, review priority matrix, CI job dependency graph, quality gates summary.

Edge Case and Validation Tests [Brent]

2026-02-10: Task 10C.1 Complete - Edge Case Test Scenarios

  • Created features/edge_case_plan_scenarios.feature (26 scenarios, 141 steps) covering concurrent plan execution, resource conflict scenarios, validation failure chains, rollback edge cases.

2026-02-10: Task 10C.4 Complete - Validation Test Fixtures

  • Created features/validation_test_fixtures.feature (34 scenarios, 81 steps) covering AST security validation, lambda AST validation, Python content sanitization, project model validation, change list coercion, ActionArgument parsing.
  • Fixed step name collisions with existing step files.
  • Fixed irrecoverable syntax scenario: "def broken(" is actually recoverable via docstring wrapping; replaced with null byte input which is truly irrecoverable.

CI/CD Pipeline Stabilization [Brent]

2026-02-12: Task Q0-min-ci In Progress - Nox-Based PR Validation Workflow

  • Rewrote .forgejo/workflows/ci.yml to route ALL jobs through nox sessions instead of direct tool invocations.
  • Coverage boost 92%->97%: Wrote 108 new Behave scenarios covering 6 largest coverage gaps.
  • Key discovery: Behave loads ALL step definition files globally; any @given/@when/@then pattern matching an existing one causes AmbiguousStep errors.

2026-02-12: Bugfix - Integration Test Failure in Robot.Actor Configuration

  • Root cause: robot/ci_nox_validation.robot ran nox --list during integration tests. When pabot executed all Robot suites in parallel, the nox --list stdout leaked into the sibling robot/actor_configuration.robot process, causing JSONDecodeError.
  • Fix: Added slow tag to interfering test cases; replaced hardcoded venv paths with python.

2026-02-12: Bugfix - CI security Job Failure (session name mismatch)

  • Root cause: CI referenced nox -s security_scan and nox -s dead_code, but the noxfile only had a session named security.
  • Fix: Renamed security to security_scan, added standalone dead_code session.

2026-02-12: Bugfix - CI unit_tests Failure: Rich Console Line-Wrapping

  • Root cause: Rich's Console wraps output at 80 columns. On CI, temp directory paths exceed 80 chars, splitting filenames across lines.
  • Fix: Collapse Rich line-wraps via clean_output.replace("\n", "") before assertions.

2026-02-12: Bugfix - CI integration_tests Failures (3 categories)

  • Issue 1: ModuleNotFoundError — hardcoded /app path doesn't exist on CI. Fixed with ${CURDIR}/.. relative paths.
  • Issue 2: --load-context help text assertion failures — Rich output goes to stderr when no TTY. Fixed with stderr=STDOUT merge.
  • Issue 3: FileNotFoundError: robot — pabot subprocess spawning exhausts container resources. Restored --exclude discovery flag.

2026-02-13: Bugfix - CI security Job Failure (missing build/ directory)

  • Root cause: build/ directory does not exist in fresh CI checkout. Bandit cannot create intermediate directories.
  • Fix: Added os.makedirs("build", exist_ok=True) before bandit invocation.

2026-02-13: Bugfix - CI integration_tests Failures (3 categories)

  • Issue 1: FileNotFoundError: robot after ~17 suites — container resource exhaustion. Capped pabot parallelism to min(default, 2), explicitly prepend venv bin/ to PATH.
  • Issue 2: Rich ANSI escape codes split --load-context string. Added env:NO_COLOR=1.
  • Issue 3: initial_next_command_test.robot exit code 3 — missing OPENAI_API_KEY. Tagged as slow.

2026-02-13: Bugfix - CI integration_tests Failures Round 2 (pabot resource exhaustion + ANSI codes)

  • robot is findable at session start yet pabot still fails after ~24 suites even with --processes 2.
  • Root cause confirmed: FileNotFoundError from subprocess.Popen on Linux occurs when the system cannot execve() due to exhausted PIDs or file descriptors, not because the binary is missing.
  • Fix: Replaced pabot with robot (sequential execution). CI values reliability over speed.
  • NO_COLOR=1 insufficient — Rich still embeds non-color ANSI sequences (bold, reset). Added env:TERM=dumb.

2026-02-13: Bugfix - CI integration_tests Failures Round 3 (venv PATH, Resource paths, hanging tests)

  • Issue 1: python -m cleveragents uses system Python — venv PATH line was accidentally removed. Restored.
  • Issue 2: Robot Resource file resolution fails on CI — bare relative references. Converted all to ${CURDIR}/ absolute paths across 30 robot files.
  • Issue 3: rxpy_route_validation.robot tests hang — missing timeout parameters on Run Process calls. Added timeout=30s to all 9 calls.

2026-02-13: Bugfix - CI integration_tests Failures Round 4 (duplicate Settings blocks, venv Python injection, CI debug)

  • Issue 1: Duplicate *** Settings *** blocks in 14 robot files created by sed-based conversion. Merged into single blocks.
  • Issue 2: Bare python in Run Process calls resolves to system Python on CI. Noxfile now passes --variable PYTHON:<venv-python-path> to robot. All 14 files updated to use ${PYTHON}.
  • Issue 3: Hardcoded /app/src path in system_prompt_template_rendering.robot. Removed (venv Python already has package installed).

2026-02-13: Bugfix - CI integration_tests Failures Round 5 (guard cleanup when OpenAI key missing)

  • Root cause: robot/scientific_paper_basic.robot suite setup skips on missing OPENAI_API_KEY, leaving ${CONTEXT_DIR} empty. Teardown's Remove Directory ${CONTEXT_DIR} recursive=True resolves to CWD, deleting the repo root (including .nox venv and robot/*.resource files), causing all downstream suites to fail.
  • Fix: Initialize ${CONTEXT_DIR} to ${TEMPDIR}/paper_basic_contexts and guard cleanup with empty check.

2026-02-13: Enhancement - Restore parallel Robot runs + silence discovery resource warning

  • Reintroduced pabot with conservative parallelism (max 2 processes) now that teardown bug is fixed.
  • Added robot/discovery_common.resource stub to silence non-fatal import warning.

Coverage and Security Enforcement [Brent]

2026-02-13: Task Q0-min-coverage In Progress - Coverage Threshold Enforcement

  • Enhanced coverage_report nox session: COVERAGE_THRESHOLD = 97 constant, generates HTML/XML/JSON reports before checking threshold, emits CI-parseable summary line.
  • Created docs/development/testing.md (180+ lines): full testing guide covering unit (Behave), integration (Robot), benchmarks (ASV), and coverage.
  • Created features/coverage_threshold_enforcement.feature (11 scenarios).
  • Verification: 2235 unit scenarios passed, 211 integration tests passed, 97.5% coverage.

2026-02-13: Task Q0-adv-security In Progress - Align Security Scans with Nox

  • Added Semgrep as Step 3 in security_scan session (between Bandit and Vulture).
  • Changed semgrep hook from wrapper script to direct invocation.
  • Updated quality-automation docs with revised thresholds (85->97%).

Plan Model Alignment [Luis]

2026-02-13: Stage A2b.beta Complete - Plan Model Alignment

  • Rewrote plan.py to align with spec: processing_state replaces action_state/state, project_links replaces project_ids, added action_name, automation_profile, invariants, actor fields, subplan hierarchy.
  • Rebased feature/m1-plan-model onto A2b.alpha commit. Resolved 30 merge conflicts. Key design decisions during merge:
    1. Plan field name: processing_state (per spec), NOT state.
    2. Invariant enum: InvariantSource (our naming), NOT InvariantScope.
    3. Action model: HEAD taken as-is (authoritative for action domain).
    4. No action_id on Action domain model — uses namespaced_name only.
    5. project_links used everywhere (NOT project_ids).
  • Fixed 6 post-rebase test failures.
  • Final results: 130 Behave features / 2246 scenarios / 9868 steps all pass, 206 Robot tests all pass, plan.py coverage 100%.

Persistence Layer (A5)

2026-02-09: Stage A5.3 + A5.4 Complete - v3 SQLAlchemy Models

  • Created LifecycleActionModel (table: actions_v3) and LifecyclePlanModel (table: lifecycle_plans) with full spec alignment.
  • Used __allow_unmapped__ = True on both models for from __future__ import annotations compatibility.
  • Legacy note: These v3 ORM models use action_id/project_ids and draft-style fields; they are superseded by the A5 rebaseline.

2026-02-09: Stage A5.6 Complete - ActionRepository (Luis tasks A5.6a-A5.6j)

  • Created ActionRepository with session-factory pattern, 10 methods, @database_retry on all public methods.
  • All mutating methods flush (do NOT commit) — caller/UnitOfWork handles commit.
  • Uses Any type hints for domain Action objects to avoid circular import issues.

Sandbox Infrastructure [Luis]

2026-02-09: Stage B3.1 + B3.2 + B3.5 + B3.6 + B3.7 + B3.8 Complete - Sandbox Infrastructure

  • Created src/cleveragents/infrastructure/sandbox/ package with 6 modules: protocol.py, no_sandbox.py, factory.py, manager.py, merge.py, __init__.py.
  • Design decisions:
    • Factory accepts raw strings rather than Resource objects (Resource model not yet implemented). This decouples the sandbox layer from the domain model.
    • SandboxStatus.assert_transition() provides a convenient guard that raises SandboxStateError on invalid transitions.
    • NoSandbox.rollback() always raises SandboxRollbackError rather than being a no-op, because silently swallowing rollback requests for unsandboxed resources would hide bugs.

2026-02-09: Stage A2.1 Already Complete

  • estimation_actor field already present in action model. review_actor field already present. Also has apply_actor (bonus field from earlier work).
  • safety_profile remains DEFERRED to post-30 per plan.

Security Hardening (SEC1) [Luis]

2026-02-09: Stage SEC1 Complete - Remove eval() Vulnerability

  • Audit results (7 matches in src/cleveragents/): 2 real vulnerabilities, 4 false positives.
  • Fixes applied: SimpleToolAgent — added named operation registry (_SAFE_OPERATIONS), legacy code blocks fully rejected. ReactiveStreamRouter — added named transform registry, legacy fn string expressions that are not in the registry fully rejected. No eval() or exec() calls remain.

2026-02-10: Stage SEC1.5 Complete - Security BDD tests

  • Created features/security_eval.feature with 8 scenarios covering code injection rejection, malicious transform rejection, valid config acceptance, custom registered operations.

Subplan Model [Luis]

2026-02-09: Stage E1 Complete - Subplan Model (Luis tasks E1.1-E1.5)

  • Added ExecutionMode, SubplanMergeStrategy enums, SubplanConfig, SubplanStatus, SubplanAttempt models, SubplanFailureHandler class.
  • Renamed from MergeStrategy to SubplanMergeStrategy to avoid collision with infrastructure-level MergeStrategy protocol.
  • Used Pydantic BaseModel instead of dataclass (per ADR-004) for SubplanAttempt and SubplanStatus.

Automation Levels [Luis]

2026-02-09: Stage A6 Complete - Automation Levels Foundation

  • Added AutomationLevel enum (MANUAL, REVIEW_BEFORE_APPLY, FULL_AUTOMATION).
  • Updated PlanLifecycleService with should_auto_progress(), auto_progress(), pause_plan(), resume_plan(), set_plan_automation_level().
  • complete_strategize() and complete_execute() now call auto_progress() automatically.

2026-02-10: Stage A6.5 Complete - Automation Levels BDD Tests

  • Created features/automation_levels.feature with 24 Behave scenarios.
  • Discovery: pydantic-settings BaseSettings does not accept constructor keyword overrides for fields with validation_alias; must set attribute after construction.

Action YAML Schema [Aditya]

2026-02-13: Stage A2b.gamma Complete - Action YAML Schema + Examples + Loader

  • Created docs/schema/action.schema.yaml, 5 example action YAMLs, ActionConfigSchema Pydantic model with from_yaml()/from_yaml_file() factory methods, camelCase->snake_case key normalization, ${ENV_VAR} interpolation, invariant normalization.
  • 31 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmark suites. Coverage: 98%.

Tool and Skill Domain Models [Jeff]

2026-02-13: Stage C0.domain Complete - Tool and Validation Domain Models

  • Created tool.py (624 lines) with 6 StrEnums, 5 Pydantic models, full validators, factory methods.
  • Key decision: YAML loader implemented as from_config() class methods (matches action.py pattern) rather than separate module.
  • Key decision: Validation extends Tool via Pydantic model inheritance.
  • Key decision: wraps field sets source to wrapped implicitly; transform required when wraps is set.

2026-02-13: Stage C0.skill.domain Complete - Skill Domain Model and Resolver

  • Created skill.py (475 lines) with 9 Pydantic models. SkillResolver.resolve() flattens includes, de-duplicates tools, rejects cycles with path traces.
  • Key decision: SkillResolver is a standalone class (not a method on Skill) to support registry-backed resolution.
  • Key decision: ResolvedToolEntry includes source tracking for compiler diagnostics.

Resource Registry [Jeff]

2026-02-13: Stage B1.core.db Complete - Resource Registry Database Tables

  • Created Alembic migration with 3 tables: resource_types, resources, resource_edges.
  • Key design decision: resource_types PK is the namespaced name string (not ULID) since types are schema-level definitions with stable identifiers.
  • Key design decision: link_type enum ('contains', 'references', 'derived_from') covers hierarchical containment, cross-layer references, and derived virtual resources.
  • Key design decision: auto_discovered flag on both resources and edges tracks provenance for cleanup/refresh operations.

Session Domain [Jeff]

2026-02-13: Stage A7.domain Complete - Session Domain Models and Contracts

  • Created session.py with MessageRole enum, SessionMessage, SessionTokenUsage, Session models, error classes, SessionService ABC.
  • Key decision: SessionService is an ABC (not Protocol) to enforce explicit inheritance and make contract violations visible at class definition time.
  • Key decision: Messages are ordered by sequence_number (not timestamp) to guarantee stable ordering across serialization boundaries.

CLI Spec Alignment [Jeff]

2026-02-14: Stage A4b.action Complete - Action CLI Spec Alignment

  • Rewrote action.py: config-only action create --config <file>, removed action available command, added filters to action list.

2026-02-14: Stage A4b.plan Complete - Plan CLI Flag Alignment

  • Updated plan.py: plan use now accepts multiple PROJECT args, --automation-profile, --invariant (repeatable), actor overrides, --arg name=value. Aligned plan list filters. Added --reason to plan cancel.

2026-02-14: Stage A4b.outputs Complete - CLI Output Format Stabilization

  • Created formatting.py: shared format_output() helper with OutputFormat enum, JSON/YAML/plain/table/rich serializers. Handles datetime, Enum, nested structures.

Tool Runtime and Built-ins [Jeff]

2026-02-14: Stage C0.runtime Complete - Tool Runtime Core

  • Created runtime.py (ToolSpec, ToolResult, ToolError), registry.py (thread-safe ToolRegistry with RLock), runner.py (ToolRunner with 4-stage lifecycle: discover/activate/execute/deactivate).
  • New tool modules have 100% coverage.

2026-02-14: Stage C0.files Complete - Built-in File Tools

  • Created src/cleveragents/tool/builtins/ package with 6 file tools (file-read/write/edit/delete/list/search), path traversal prevention, ChangeSetEntry/ChangeSet/ChangeSetCapture models.
  • 2026-02-14: Commit traceability sweep found no matching git log entries for skill schema/examples, skill CLI, provider actors, MCP config/adapter, actor compiler, or skill registry persistence; Section 2B traceability tasks remain open.

Tool and Session Persistence [Luis]

2026-02-14: Stage C1.tool.registry Complete - Tool Registry Persistence

  • Created Alembic migration with 3 tables: tools, tool_resource_bindings, validation_attachments.
  • Created ToolRegistryRepository (5 methods), ValidationAttachmentRepository (4 methods), ToolRegistryService (8 methods).
  • Key decision: ToolModel.to_domain() returns a dict (not a domain object) to decouple persistence from domain model until full integration is complete.
  • Key decision: ToolRegistryRepository.delete() pre-checks for active ValidationAttachmentModel rows before allowing deletion.
  • Key decision: Validation attachments are resource-scoped with optional project_name and plan_id narrowing.

2026-02-15: Stage A7.persistence Complete - Session Persistence and Repositories

  • Created Alembic migration with 2 tables: sessions, session_messages.
  • Created SessionRepository (5 methods), SessionMessageRepository (3 methods), PersistentSessionService (8 methods).
  • Key decision: PersistentSessionService extends the domain SessionService ABC.
  • Key decision: append_message() auto-calculates sequence number and updates parent session's updated_at timestamp.
  • Key decision: import_session() validates schema version and SHA-256 checksum before persisting, then generates fresh ULIDs to avoid ID collisions.
  • Key decision: update_token_usage() uses cumulative addition (+=) rather than replacement.

2026-02-15: Bugfix - Benchmark Unique Name Constraint [Luis]

  • ASV benchmark suites that create tools/resources now use unique names per iteration to avoid UNIQUE constraint failures when ASV runs multiple iterations.

Skill and Actor YAML [Aditya]

2026-02-16: C0.skill.schema Complete - Skill YAML Schema + Examples + Loader

  • Created docs/schema/skill.schema.yaml, 5 example skill YAMLs, SkillConfigSchema Pydantic model with from_yaml()/from_yaml_file() factory methods, camelCase->snake_case normalization, ${ENV_VAR} interpolation.
  • 36 Behave scenarios, 6 Robot smoke tests, 3 ASV benchmarks.

2026-02-17: Task C1.schema Complete - Actor YAML Schema Models

  • Created src/cleveragents/actor/schema.py (695 lines) with three actor types (LLM, TOOL, GRAPH), graph topology validation with cycle detection using DFS, type-specific field requirements, environment variable interpolation.
  • 5 example actor YAMLs, 50+ Behave scenarios, 11 Robot test cases, 8 ASV benchmark classes.

2026-02-17: C0.skill.cli Complete - Skill CLI Commands + Output Formatting

  • Created SkillService with dual-mode storage pattern (in-memory fallback, ready for UnitOfWork persistence).
  • Created skill.py Typer command group with 5 subcommands (skill add, remove, list, show, tools), all supporting 6 output formats.
  • Key decision: Module-level _service singleton (not DI container) matches the pattern in action.py CLI; _reset_skill_service() enables test isolation.

Bugfixes and Merges

2026-02-17: Bugfix - Unit Test and Benchmark Failures After Master Merge [Brent]

  • Unit test failure — Session mismatch in project repository steps. The NamespacedProjectRepository creates a new SQLAlchemy session per method call. Fix: Changed session factory to always return the same session instance.
  • Benchmark failures — 5 distinct issues across 7 files: missing parameterized setup params, renamed PlanPhase.APPLIED -> APPLY, per-iteration plan creation, batch counters for unique IDs across ASV iterations, simplified action reuse.

2026-02-18: Task C1.examples Complete - Actor YAML Examples and Documentation

  • Created docs/reference/actors_examples.md (1098 lines) with pattern-based organization (Strategist, Executor, Reviewer, Tool-Only, Validation-Node, Graph, Hierarchical).
  • 25 Behave scenarios, 16 Robot test cases, 7 ASV benchmarks.
  • Bug fixes: Missing description fields in test YAML strings, context_view step definition conflict, enhanced parallel execution detection.

Actor Loader [Aditya]

2026-02-19: Task C2.loader Complete - Actor Registry and Loader

  • Created src/cleveragents/actor/loader.py (277 lines) — ActorLoader class with thread-safe discovery, SHA-256 content-hash caching, and namespace management.
  • 23 Behave scenarios, 4 Robot test cases, 4 ASV benchmarks. Coverage: 98%.
  • Key design decisions: SHA-256 content hashing (not mtime) for deterministic cache invalidation, single consolidated ValidationError for duplicates/errors, tool reference resolution at load time via optional ToolRegistry, local/ default namespace applied during YAML parsing.

Plan Checklist Rebaseline

2026-02-13: Plan Update - Rebaseline A5 Persistence Checklist

  • Rewrote A5 plan persistence checklist to replace legacy tasks with spec-aligned rebaseline migrations, ORM mappings, repositories, and persistence tests.

2026-02-13: Plan Update - A5 Ownership + Validation Attachments

  • Reassigned A5.alpha migrations to Hamza to align DB/migration expertise.
  • Rebased tool registry tasks to make validation attachments explicitly resource-scoped with required/informational mode.

2026-02-13: Plan Update - Consolidate Skills/Actors/MCP + B0 DB Ownership

  • Retired duplicate C2/C3/C4 blocks; canonical tasks consolidated under C0.skill, C1/C2, C7.mcp, C8.providers.
  • Reassigned B0.db resource/project migrations to Hamza.

2026-02-13: Plan Update - Tool Runtime Naming Consolidation

  • Renamed M1 tool runtime block to C0 runtime core and M3 lifecycle block to C0.lifecycle to avoid duplicate naming.

2026-02-13: Plan Update - Spec Alignment Pass

  • Updated A4b CLI to use processing_state terminology.
  • Corrected skill registry dependency references.

2026-02-13: Plan Update - C1 Numbering Disambiguation

  • Renamed tool registry group to C1.tool to avoid collision with actor schema C1 group.

Change Tracking and Tool Router

2026-02-19: Stage C5.model COMMIT Complete - ChangeSet models and invocation tracker [Luis]

  • Added ToolInvocation Pydantic model, InvocationTracker Protocol and InMemoryInvocationTracker, per-change field validators, sorted_entries() and grouped_by_resource() methods, normalize_change_path() utility.
  • 21 BDD scenarios, 5 integration smoke tests, 4 ASV benchmark suites.

2026-02-19: Stage C4.git COMMIT Complete - Add git operation skills [Luis]

  • Created git_tools.py: 4 read-only git tools (status, diff, log, blame) with safe environment variables, path traversal guards, and user-friendly error mapping for 5 common git failure patterns.
  • 16 BDD scenarios, 5 integration smoke tests, 3 ASV benchmark suites.

2026-02-19: Combined merge branch fix - Alembic merge migration [Luis]

  • Created merge migration to resolve "Multiple head revisions" error between a7_001_session_tables and c0_001_skill_registry heads.

2026-02-19: Stage SEC1.eval COMMIT Complete - Remove eval-based config parsing [Luis]

  • Created config security scanner detecting 15 disallowed patterns across YAML/TOML/generic configs. Reports file path + line number + severity for each violation.
  • Extended security eval feature from 8 to 16 scenarios.

Execution Pipeline and Apply

2026-02-20: D0b.execute — Plan Execute via Actors (M1 Critical Blocker)

  • Created PlanExecutor orchestrator with StrategizeStubActor (read-only, produces decisions) and ExecuteStubActor (sandbox resources, routes tool calls).
  • Strategize phase is read-only. Execute phase uses sandbox resources and routes tool calls through ToolRunner.
  • Phase guards prevent Execute when Plan is not in Strategize COMPLETE state.
  • Streaming hooks emit interim status updates.

2026-02-20: D0b.apply Complete - Diff Review + Apply Integration [Jeff]

  • Created PlanApplyService with diff(), artifacts(), persist_apply_summary(), handle_merge_failure(), guard_empty_changeset().
  • Key decision: Used error_details dict on Plan as general metadata store for apply summary fields until a dedicated metadata field lands.
  • Key decision: Service accepts optional changeset_store — when None, builds stub SpecChangeSet from plan metadata only.

2026-02-20: C5.router Complete - Tool Call Router for LLM Providers [Jeff]

  • Created router.py (~890 lines) — ToolCallRouter translating between LLM provider tool call formats and internal ToolRunner: provider format detection, normalization, deterministic ID generation, error classification, schema adaptation, batch routing, streaming support.
  • Key discovery: ToolRunner.execute() catches handler exceptions internally and returns ToolResult(success=False) rather than raising, so streaming produces COMPLETE + failed result (not ERROR update) for tool failures.
  • Key discovery: Pyright strict mode requires all Pydantic model fields (even those with defaults via Field(default=...)) to be explicitly passed in constructor calls.
  • Key discovery: Behave step name collision with existing coverage_boost_extra_steps.py; renamed to avoid AmbiguousStep error.

A9.service: Config Service with Multi-Level Resolution

  • 2026-02-23 [Jeff]: Implemented ConfigService in src/cleveragents/application/services/config_service.py with:
    • Five-level resolution chain: CLI flag > env var > project-scoped > global config > default
    • Typed config key registry with 24 entries across 6 sections: core.* (7), plan.* (3), provider.* (4), sandbox.* (3), context.* (3), index.* (4)
    • TOML file management (~/.cleveragents/config.toml) with auto-created parent directories via tomlkit
    • ResolvedValue return type includes winning value, source level, and optional verbose chain
    • Env var interpolation following CLEVERAGENTS_<SECTION>_<KEY> convention
    • Validation: unknown keys raise ValueError with sample of valid keys; type mismatches raise TypeError with actionable messages
    • Boolean coercion from string: true/false, 1/0, yes/no (case-insensitive)
  • 2026-02-23 [Jeff]: Added docs/reference/config_resolution.md documenting all 24 config keys with types, defaults, env var mappings, and precedence rules
  • 2026-02-23 [Jeff]: Added features/config_resolution.feature with 31 Behave scenarios covering every resolution level, env var overrides, type coercion, unknown key rejection, TOML roundtrips, and verbose chain output
  • 2026-02-23 [Jeff]: Added robot/config_resolution.robot with 8 Robot Framework end-to-end tests exercising default/global/env/CLI resolution, unknown key rejection, type coercion, and env var convention
  • 2026-02-23 [Jeff]: Added benchmarks/config_resolution_bench.py with 6 ASV benchmark suites for resolution performance at each level, bulk operations, and validation throughput
  • 2026-02-23 [Jeff]: Exported ConfigService, ConfigEntry, ConfigLevel, ResolvedValue from application.services.__init__; updated vulture whitelist for new public API

CLI1.alpha: Plan/Action CLI Extensions

  • 2026-02-23 [Jeff]: Implemented CLI1.alpha. Key changes:
    • Added validate_namespaced_actor() to plan.py for strict namespace/name validation on actor override flags (--strategy-actor, --execution-actor, --estimation-actor, --invariant-actor)
    • Updated _plan_spec_dict() to include estimation_actor, invariant_actor, and invariants fields in all output formats (json, yaml, plain, table)
    • Updated _print_lifecycle_plan() rich panel to display invariants with source provenance tags
    • Updated plan_status summary table to include Profile and Invariants columns
    • Updated lifecycle_list_plans table to include Profile and Invariants columns
    • Extended action.py _action_spec_dict() to conditionally include estimation_actor, invariant_actor, and inputs_schema fields
    • Extended _print_action() rich panel to display estimation actor, invariant actor, invariants list, inputs_schema, and automation_profile
    • --automation-profile flag on plan use validates against BUILTIN_PROFILES and persists to plan metadata
    • --invariant flags on plan use are passed through to PlanLifecycleService.use_action() as PlanInvariant objects with InvariantSource.PLAN
    • Created docs/reference/plan_cli.md and docs/reference/action_cli.md reference documentation
    • Created features/cli_extensions.feature with 25 Behave scenarios covering all new functionality
    • Created features/steps/cli_extensions_steps.py with all step definitions
    • Created robot/cli_extensions.robot with 5 Robot Framework integration tests
    • Created benchmarks/cli_extensions_bench.py with 6 ASV benchmark suites
Reference in New Issue View Git Blame Copy Permalink