Commit Graph

812 Commits

Author SHA1 Message Date
brent.edwards 332eb142b3 Merge pull request 'test(e2e): verify M6 success criteria — Firefox-scale autonomous porting' (#457) from test/m6-e2e-verification into master
Reviewed-on: cleveragents/cleveragents-core#457
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 23:00:14 +00:00
brent.edwards ea060ec7a9 Merge branch 'master' into test/m6-e2e-verification 2026-02-27 22:16:51 +00:00
freemo 7235d46ade feat(skill): persist flattened tool sets
Add Alembic migration m4_002_skill_flattened_tools to extend the skills
table with five new columns: flattened_tools_json, includes_json,
capability_summary_json, yaml_text, and flattening_hash (SHA-256).  A
defence-in-depth uniqueness constraint (uq_skills_name) is also added.

Update SkillModel with the new column definitions and extend
SkillRepository with update_flattened_tools(), get_flattened_tools(),
needs_refresh(), recompute_flattening_hash(), and
invalidate_cached_summaries() methods.  The existing update() method
now nulls all cached fields on mutation (hash-based invalidation).

All new repository methods follow the session-factory pattern with
@database_retry and flush-but-don-t-commit semantics.  Structured
logging via structlog records cache updates and invalidations.

Database schema docs updated with the new skills table columns and a
persistence-field-to-domain-model mapping table.

Tests:
- 6 Behave scenarios covering create, invalidation, hash staleness,
  refresh recomputation, uniqueness constraint, and namespace filtering
- 2 Robot Framework smoke tests (round-trip and invalidation)
- 3 ASV benchmarks (persist, refresh check, namespace list)

ISSUES CLOSED: #166
2026-02-27 20:09:26 +00:00
brent.edwards 78bd3584e0 Merge branch 'master' into test/m6-e2e-verification 2026-02-27 20:03:33 +00:00
freemo c9abb45adf test(coverage): add Behave BDD scenarios for 9 under-covered modules
Added 246 new BDD scenarios across 9 feature files to improve unit test
coverage for modules that were either entirely untested or had significant
coverage gaps:

- lock_service_coverage.feature (27 scenarios): validation branches,
  TTL boundaries, re-entrant acquisition, rollback on exceptions
- plan_apply_service_coverage.feature (54 scenarios): operation labels,
  diff rendering (plain/rich/json), artifact building, validation gate,
  changeset resolution and cleanup
- plan_executor_coverage.feature (51 scenarios): step parsing, execute
  actor integration, strategize/execute guards, stub retry/recovery,
  decision tree construction
- skill_cli_coverage_r3.feature (22 scenarios): tools refresh, list/show
  JSON fallback, capability summary errors, remove confirmation
- changeset_repository_coverage.feature (39 scenarios): entry/tool repos
  validation, database error wrapping, domain conversion, SQLite store
  CRUD operations
- repositories_coverage.feature (20 scenarios): get_by_name/namespace
  errors, list_available filters, delete with ActionInUseError, plan
  update with invariants/processing_state/error_details
- sandbox_copy_on_write_coverage.feature (12 scenarios): create OSError
  wrapping, get_path state transitions, commit edge cases, rollback
  errors, cleanup with missing paths
- bridge_coverage.feature (8 scenarios): __del__ suppression, async task
  cancellation, execute_graph message type handling, stream config,
  state checkpointer
- plan_cli_coverage.feature (13 scenarios): legacy apply/list/cd paths,
  use-action with estimation/invariant actors, lifecycle-apply guards,
  status errors, error recovery display

All 246 scenarios (1105 steps) pass. Step definitions use unique prefixes
to prevent ambiguous step conflicts with existing tests.

ISSUES CLOSED: #467
2026-02-27 13:47:42 -05:00
freemo a074b4846f fix(provider): remove FakeListLLM defaults
Remove FakeListLLM as a silent fallback in agent graph constructors
(plan_generation.py, context_analysis.py, auto_debug.py). All three now
raise ValueError when llm=None, making missing-provider errors explicit.

Add Settings.mock_providers flag and validate_provider_availability()
method. Update container.get_ai_provider() to check Settings.mock_providers
first, with env-var fallback for backward compatibility.

Add resolve_provider_by_name() helper to the provider registry and export
it from cleveragents.providers. Add structlog trace logging to
ProviderRegistry.get_default_provider_type() to record selection reasoning.

Update all existing behave step files, robot tests, and benchmarks that
relied on the implicit FakeListLLM default to pass an explicit LLM
instance instead.

Add new BDD tests (features/provider_fixes.feature with 17 scenarios),
Robot Framework integration tests (robot/provider_detection_smoke.robot),
and ASV benchmarks (benchmarks/provider_selection_bench.py).

ISSUES CLOSED: #323
2026-02-27 09:47:10 -05:00
aditya fa1794dba9 Merge pull request 'feat(skill): add MCP refresh hooks' (#455) from feature/m4-skill-registry-refresh into master
Reviewed-on: cleveragents/cleveragents-core#455
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 13:17:20 +00:00
aditya 9af9ff8d95 Merge branch 'master' into feature/m4-skill-registry-refresh 2026-02-27 12:27:10 +00:00
aditya ddebcad872 Merge pull request 'feat(cli): add skill tools and refresh commands' (#454) from feature/m4-skill-registry-cli into master
Reviewed-on: cleveragents/cleveragents-core#454
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 12:15:07 +00:00
aditya 174d1dc334 Merge branch 'master' into feature/m4-skill-registry-cli 2026-02-27 10:40:42 +00:00
aditya 9fd29b2748 Merge pull request 'feat(skill): add agent skills loader' (#453) from feature/m3-agent-skills-loader into master
Reviewed-on: cleveragents/cleveragents-core#453
Reviewed-by: Hamza Khyari <hamza.khyari@cleverthis.com>
2026-02-27 10:22:01 +00:00
aditya 4c4bd74d94 Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-27 09:34:00 +00:00
aditya ab9732c2a1 Merge pull request 'feat(skill): add MCP adapter for external tools' (#434) from feature/m3-mcp-adapter into master
Reviewed-on: cleveragents/cleveragents-core#434
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 07:28:15 +00:00
aditya a0f56228e2 Merge branch 'master' into feature/m3-mcp-adapter 2026-02-27 06:40:15 +00:00
freemo 120274da99 feat(di): wire decision services
Create DecisionService application-layer service that wraps
DecisionRepository with structured logging and UnitOfWork transaction
management. Wire DecisionService and PlanLifecycleService into the DI
container as Factory providers.

Inject DecisionService into PlanLifecycleService so that phase
transitions automatically record decisions: start_strategize records a
strategy_choice decision and start_execute records an
implementation_choice decision. Decision recording is optional and
never blocks lifecycle transitions.

Add Behave feature (5 scenarios), Robot Framework smoke tests (2 test
cases), ASV benchmarks (3 benchmark classes), and DI reference
documentation.

ISSUES CLOSED: #173
2026-02-27 02:33:42 +00:00
brent.edwards f7b8186b3f Merge branch 'master' into test/m6-e2e-verification 2026-02-26 21:26:00 +00:00
hamza.khyari 13b1eb45a8 Merge pull request 'feat(security): enforce read-only actions' (#436) from feature/m4-security-readonly into master
Reviewed-on: cleveragents/cleveragents-core#436
Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-02-26 21:25:13 +00:00
khyari hamza 493e5cf8a1 fix(security): wire read_only through ExecuteStubActor and replace stubs with real tests
- Add read_only kwarg to ExecuteStubActor.execute(), propagated to
  ChangeSetCapture; PlanExecutor._run_execute_with_stub passes
  plan.read_only through the execution path (P1 #2 fix)
- Replace CLI fail-fast and Action-Skill stub scenarios with real
  ExecuteStubActor spy test and SkillContext.enforce_write_guard
  integration tests (P2 #1 fix)
- Update docs: remove unimplemented Layer 5 (Action-Skill), add
  ExecuteStubActor wiring description, fix test command (P2 #2 fix)

ISSUES CLOSED: #322
2026-02-26 20:33:20 +00:00
khyari hamza 2b09568cfa feat(security): enforce read-only actions
Tighten ToolRuntime._enforce_capabilities() to block ANY tool with
writes=True when plan_read_only is set, removing the not-cap.read_only
loophole that allowed certain write tools through. Tool name is now
always included in the ToolAccessDeniedError message.

Add read_only flag to ChangeSetCapture with ReadOnlyViolationError
raised when write-capable tools are wrapped on a read-only plan.

Add CLI fail-fast guards on plan execute and plan apply commands that
abort before calling the service layer if plan.read_only is True.

SkillContext.enforce_write_guard() already included tool name
correctly and required no changes.

Includes 18 Behave scenarios (90 steps), Robot integration tests,
ASV benchmarks, and docs/reference/read_only_actions.md.

ISSUES CLOSED: #322
2026-02-26 20:33:20 +00:00
freemo 4ed057fa58 Docs: Fixed refs in docs 2026-02-26 14:40:17 -05:00
freemo c88b432beb Docs: Daily timeline update 2026-02-26 14:32:59 -05:00
brent.edwards 1ec2200e34 test(e2e): verify M6 success criteria — Firefox-scale autonomous porting
Robot Framework E2E test suite exercising the complete M6 success
criteria verification sequence:

- Porting action creation from YAML config via CLI
- Plan use + execute via CLI with mocked lifecycle service
- Hierarchical decomposition: 4+ levels (root + L1-L4 = 21+ plans)
- Decision correction recomputes only affected subtree (CorrectionImpact)
- Parallel execution scales to 15 concurrent subplans (10+ required)
- Realistic porting task: full ACTION → STRATEGIZE → EXECUTE → APPLY
  lifecycle with 10 subplans completing autonomously
- Plan apply transitions to APPLIED terminal state
- SubplanFailureHandler retry/stop-others logic verification
- SubplanConfig supports all execution modes and merge strategies
- Decision tree structure: PROMPT_DEFINITION root with
  SUBPLAN_PARALLEL_SPAWN children and superseded_by flow

Ten subcommands in the Python helper, each printing a sentinel
string on success. All subcommands verified passing locally.

Closes #407
2026-02-26 19:27:53 +00:00
brent.edwards 93f2d396a1 Merge pull request 'test: consolidated Brent QA batch — issues #179, #180, #404, #405, #187' (#442) from develop-brent-5 into master
Reviewed-on: cleveragents/cleveragents-core#442
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
2026-02-26 16:54:21 +00:00
brent.edwards ffdd2f2b19 Merge branch 'master' into develop-brent-5 2026-02-26 16:14:07 +00:00
brent.edwards 32b81793a2 refactor(test): rename decision persistence files to avoid conflict with master
Rename our serialization-focused decision persistence suites to
*_serialization to avoid add/add conflicts with the repository-based
decision persistence suites that landed on master independently:

- decision_persistence.feature -> decision_persistence_serialization.feature
- decision_persistence_steps.py -> decision_persistence_serialization_steps.py
- decision_persistence_bench.py -> decision_persistence_serialization_bench.py
- decision_persistence.robot -> decision_persistence_serialization.robot
- helper_decision_persistence.py -> helper_decision_persistence_serialization.py

Updated robot helper path, step docstring, and testing.md references.
2026-02-26 16:13:44 +00:00
hamza.khyari 074f884f14 Merge pull request 'fix(security): close async resources and leaks' (#435) from feature/m4-security-async-cleanup into master
Reviewed-on: cleveragents/cleveragents-core#435
2026-02-26 15:19:16 +00:00
aditya 76db58414a Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-26 14:51:41 +00:00
aditya f8bc02869a Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-26 14:43:52 +00:00
aditya d0337a0881 fix(skill): eliminate Any from agent_skills_loader type signatures
Replace dict[str, Any] with a private _YamlMapping type alias
(dict[str, object]) in AgentSkillSpec Pydantic fields and all
internal helper function signatures. Add isinstance guard for
allowed-tools list iteration to satisfy pyright.

Refs: #160
2026-02-26 14:40:44 +00:00
khyari hamza c406781b86 fix(security): address PR #435 review feedback
P0: reject register() after close_all() with RuntimeError.
P1: catch CancelledError in close_all(), use WeakKeyDictionary for
cancellation_reasons to prevent memory leak, guard StateManager
update_state/reset/load_checkpoint/time_travel after close().
P2: contextlib.suppress in __del__ for partial construction, re-cancel
pending tasks in cleanup_tasks_async, handle late tasks added during
await window, guard AcpEventQueue.publish() after close with _is_closed
flag and is_closed property, fix ASV TimeRegisterBatch crash.
Tests: 5 new Behave scenarios (T1-T4 + is_closed), log handler and
event loop cleanup in after_scenario (T5-T6).
Docs: async_safety.md updated for register-after-close and state
mutation guards.

ISSUES CLOSED: #321
2026-02-26 14:30:43 +00:00
khyari hamza 5d5d3fde2c fix(security): close async resources and leaks
Add AsyncResourceTracker (core/async_cleanup.py) providing a central
registry for async resources with timeout-bounded close_all(), async
context manager support, and a __del__ finalizer that logs leaked
resources by name.

Enhance LangGraphBridge with cleanup_tasks_async() that awaits
in-flight tasks with a deadline instead of fire-and-forget cancel().
Add cancellation_reasons dict to trace why tasks were cancelled.

Add StateManager.close() to properly release checkpoint file handles
and complete the RxPY BehaviorSubject. Add AcpEventQueue.close() to
dispose all subscriptions.

Includes 14 Behave scenarios (67 steps), Robot integration tests,
ASV benchmarks, and docs/reference/async_safety.md.

ISSUES CLOSED: #321
2026-02-26 14:30:43 +00:00
aditya 7ad9f2bf4a chore: update CHANGELOG for feat(skill): add agent skills loader
ISSUE CLOSED #160
2026-02-26 14:28:09 +00:00
hamza.khyari db58a3bb62 Merge pull request 'feat(decision): add decision persistence layer' (#131) from feature/m4-decision-persistence into master
Reviewed-on: cleveragents/cleveragents-core#131
Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-02-26 14:25:27 +00:00
khyari hamza 7f2b1c61fc fix(decision): rebase migration onto merged changeset head
After rebasing onto master, m4_001_decision_tables and
d0_002_merge_changeset_and_locks were both leaf heads.  Update
down_revision so the decision migration chains after the merge
migration, restoring a single Alembic head.

ISSUES CLOSED: #171
2026-02-26 13:40:07 +00:00
khyari hamza 27bef69ddd feat(decision): add decision persistence
Add decision persistence layer with DecisionRepository, DecisionModel,
and Alembic migration. Includes tree queries (BFS traversal via deque,
path-to-root), superseded lookup, ordered decision path retrieval,
concrete Decision type annotations (via TYPE_CHECKING), and comprehensive
test coverage (Behave BDD, Robot Framework, ASV benchmarks). Updated
database_schema.md, CHANGELOG.md, and CONTRIBUTORS.md.

ISSUES CLOSED: #171
2026-02-26 13:40:07 +00:00
aditya 595cfd172f Merge branch 'master' into feature/m3-mcp-adapter 2026-02-26 13:30:29 +00:00
aditya e05afdcd27 Merge pull request 'feature/m2-actor-yaml' (#432) from feature/m2-actor-yaml into master
Reviewed-on: cleveragents/cleveragents-core#432
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-26 13:19:00 +00:00
aditya b25a886df8 feat(skill): add MCP refresh hooks
Implement SkillRegistry.refresh(name) and refresh_all() to recompute
flattened tool sets on demand. Wire MCPToolAdapter.dispatch_notification()
to MCPRefreshHook, which debounces rapid notifications/tools/list_changed
bursts into a single refresh_all() call (default 0.5 s window).

Add safeguard that skips tool-ref validation when no ToolRegistry is
configured, emitting a single WARNING with recovery steps. Introduce
immutable SkillRefreshResult (refreshed/failed/skipped counts) with
to_summary() and merge() for CLI and log output.

New files:
- src/cleveragents/skills/refresh.py        (SkillRefreshResult)
- src/cleveragents/mcp/refresh_hook.py      (MCPRefreshHook)
- features/skill_refresh.feature            (19 Behave scenarios)
- features/steps/skill_refresh_steps.py
- robot/skill_refresh.robot                 (10 Robot tests)
- robot/helper_skill_refresh.py
- benchmarks/skill_refresh_bench.py         (ASV benchmarks)
- docs/reference/skill_refresh.md

Modified:
- src/cleveragents/skills/registry.py       (refresh, refresh_all)
- src/cleveragents/mcp/adapter.py           (notification listener API)
- src/cleveragents/skills/__init__.py       (export SkillRefreshResult)
- src/cleveragents/mcp/__init__.py          (export MCPRefreshHook)
- CHANGELOG.md

ISSUES CLOSED #168
2026-02-26 12:29:49 +00:00
aditya c40ea014dc feat(cli): add skill tools and refresh commands
Implemented agents skill refresh command to recompute tool flattening and sync
MCP-backed skills. Enhanced skill list/show/tools outputs with capability
summary fields.

- agents skill refresh <name>|--all to recompute flattening
- Enhanced skill list/show/tools with capability summary and tool counts
- Added --format json/yaml schemas for refresh output
- CLI errors for nonexistent skills and MCP sync failures
- Behave tests (skill_cli.feature), Robot tests (skill_cli.robot)
- ASV benchmarks (skill_cli_bench.py) for CLI overhead baseline
- Updated docs/reference/skill_cli.md with refresh command examples
- Documented refresh side effects and caching behavior

ISSUES CLOSED: #167
2026-02-26 10:57:07 +00:00
aditya cb82fc51df feat(skill): add agent skills loader
Implemented AgentSkillSpec loader that parses SKILL.md frontmatter and
progressive disclosure sections (discover/activate/deactivate) into
structured SkillStep objects with stable 1-based ordering.

Mapped Agent Skills to AgentSkillToolDescriptor with namespaced naming
(namespace/short_name), source="agent_skill", read-only defaults, and
AgentSkillResourceSlot bindings for scripts/, references/, and assets/
directories. All resource slots are unconditionally read_only.

Added explicit validation for missing frontmatter fields (name,
description) and invalid namespace format with actionable error messages.

Added docs/reference/agent_skills.md covering folder layout, SKILL.md
parsing rules, progressive disclosure model, and tool mapping.

Added Behave scenarios covering valid/invalid SKILL.md parsing,
namespaced naming, step ordering, missing frontmatter errors, progressive
disclosure lifecycle, tool mapping, and resource binding slots.

Added Robot Framework integration tests using the deploy-to-staging
example skill folder (robot/agent_skills_loader.robot).

Added ASV benchmarks for parsing throughput, folder load, progressive
disclosure lifecycle, and resource listing
(benchmarks/agent_skills_loader_bench.py).

ISSUES CLOSED: #160
2026-02-26 09:10:35 +00:00
aditya 519e26fb02 Merge branch 'master' into feature/m3-mcp-adapter 2026-02-26 06:44:38 +00:00
aditya b8d7c4c893 Merge branch 'master' into feature/m2-actor-yaml 2026-02-26 06:21:39 +00:00
brent.edwards c53b880a5e Merge branch 'master' into develop-brent-5 2026-02-26 03:44:19 +00:00
freemo e3fcce413b feat(changeset): persist changesets and diff artifacts
Add SQLite persistence for ChangeSet entries and ToolInvocation records
via new changeset_repository module implementing the ChangeSetStore protocol.

- Alembic migration d0_001 creates changeset_entries and tool_invocations tables
- SqliteChangeSetStore, ChangeSetEntryRepository, ToolInvocationRepository
- PlanApplyService.cleanup_changeset() for cancel/failure cleanup
- Behave tests (17 scenarios), Robot tests (5 cases), ASV benchmarks
- Reference documentation in docs/reference/changeset.md

Closes #163
2026-02-26 02:47:14 +00:00
brent.edwards cf59f285fd Merge feature/m4-correction-subplan-smoke into develop-brent-5 2026-02-26 02:15:24 +00:00
brent.edwards a00c33c4c6 Merge test/m4-e2e-verification into develop-brent-5 2026-02-26 02:15:19 +00:00
brent.edwards 2e2ae3b9b8 Merge test/m3-e2e-verification into develop-brent-5 2026-02-26 02:15:14 +00:00
brent.edwards 3d3c8fda33 Merge feature/m4-decision-tests into develop-brent-5 2026-02-26 02:15:01 +00:00
brent.edwards b3f126b147 Merge feature/m3-decision-validation-smoke into develop-brent-5 2026-02-26 02:14:14 +00:00
brent.edwards 3c9a3efdf1 fix(test): correct mock targets, method names, ULIDs, and CLI args in M4 smoke suite
- Patch CorrectionService at its module path, not the local-import site
- Use request_correction/execute_correction/analyze_impact (real API)
- Replace invalid 27-char subplan_id values with valid 26-char ULIDs
- Add required --guidance flag to dry-run CLI invocation
- Fix feature file assertions to match actual CLI output

Resolves CI failures in unit_tests (job 4) and coverage (job 6) for
run 645 on PR #441.
2026-02-26 02:01:47 +00:00