Commit Graph

1526 Commits

Author SHA1 Message Date
brent.edwards 332eb142b3 Merge pull request 'test(e2e): verify M6 success criteria — Firefox-scale autonomous porting' (#457) from test/m6-e2e-verification into master
Reviewed-on: cleveragents/cleveragents-core#457
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 23:00:14 +00:00
brent.edwards c9782637d9 Merge branch 'master' into feature/m5-acms-smoke 2026-02-27 22:45:53 +00:00
brent.edwards c129f4c3f0 fix(test): address PR #470 review feedback
- Add CHANGELOG.md entry for M6 autonomy acceptance suite
- Split m6_autonomy_acceptance_steps.py (774 lines) into
  m6_facade_steps.py (398 lines) and m6_guardrails_steps.py (399 lines)
  to comply with the project's 500-line guideline

Closes #211
2026-02-27 22:33:47 +00:00
brent.edwards 75c628793b fix(test): correct M5 smoke patch targets and pattern matching
Fix 4 errored scenarios: patch get_container instead of the lazily-
imported ContextService (which is not a module-level attribute of
cleveragents.cli.commands.context).

Fix project context show (line 112): mock _get_namespaced_project_repo
to return a repo (not a tuple), mock get_container for session_factory,
and mock _read_policy to bypass DB access.

Fix context exclusion pattern (line 152): replace fnmatch.fnmatch with
PurePosixPath.match for proper ** glob handling and use a path with a
leading directory segment.

Also fix context list empty assertion to match actual CLI output
('No files in context' rather than 'No context'), and patch
Path.exists for the add command so the mock service is reached.
2026-02-27 22:23:13 +00:00
brent.edwards ea060ec7a9 Merge branch 'master' into test/m6-e2e-verification 2026-02-27 22:16:51 +00:00
brent.edwards f90cf24617 Merge branch 'master' into feature/m6-autonomy-smoke 2026-02-27 22:14:54 +00:00
brent.edwards 192fdd7705 Merge branch 'master' into test/m5-e2e-verification 2026-02-27 21:58:03 +00:00
brent.edwards 348c230bc5 test(e2e): add M6 autonomy acceptance suite
Add comprehensive M6 autonomy acceptance test suites covering the ACP
local-mode facade, autonomy guardrails, automation profile resolution,
event queue pub/sub, HTTP transport stub, and version negotiation.

Behave suite (52 scenarios):
- ACP facade dispatch for all 11 operations
- Guard enforcement (denylist, allowlist, budget, call limit, write/apply)
- Automation profile built-in validation and custom creation
- Profile resolution precedence (plan > action > project > global)
- Event queue lifecycle (publish, subscribe, unsubscribe, close)
- HTTP transport stub rejection in local mode
- ACP version negotiation (accept/reject)
- Model validation (AcpRequest, AcpResponse, AcpEvent, AcpErrorDetail)

Robot integration suite (11 tests):
- Facade session/plan lifecycle, unknown operation error
- Event queue publish/subscribe, transport stub, version negotiation
- Guard denylist/budget enforcement, profile resolution
- Fixture loading, full end-to-end flow

ASV benchmarks (5 suites):
- Facade dispatch, guard evaluation, profile resolution
- Event queue operations, fixture loading

Fixtures: acp_facade_flows.json, autonomy_guardrails.json,
automation_profiles.json

Closes #211
2026-02-27 20:31:15 +00:00
brent.edwards ece5e61725 test(e2e): add M5 ACMS + context suites 2026-02-27 20:20:13 +00:00
freemo 7235d46ade feat(skill): persist flattened tool sets
Add Alembic migration m4_002_skill_flattened_tools to extend the skills
table with five new columns: flattened_tools_json, includes_json,
capability_summary_json, yaml_text, and flattening_hash (SHA-256).  A
defence-in-depth uniqueness constraint (uq_skills_name) is also added.

Update SkillModel with the new column definitions and extend
SkillRepository with update_flattened_tools(), get_flattened_tools(),
needs_refresh(), recompute_flattening_hash(), and
invalidate_cached_summaries() methods.  The existing update() method
now nulls all cached fields on mutation (hash-based invalidation).

All new repository methods follow the session-factory pattern with
@database_retry and flush-but-don-t-commit semantics.  Structured
logging via structlog records cache updates and invalidations.

Database schema docs updated with the new skills table columns and a
persistence-field-to-domain-model mapping table.

Tests:
- 6 Behave scenarios covering create, invalidation, hash staleness,
  refresh recomputation, uniqueness constraint, and namespace filtering
- 2 Robot Framework smoke tests (round-trip and invalidation)
- 3 ASV benchmarks (persist, refresh check, namespace list)

ISSUES CLOSED: #166
2026-02-27 20:09:26 +00:00
brent.edwards 78bd3584e0 Merge branch 'master' into test/m6-e2e-verification 2026-02-27 20:03:33 +00:00
brent.edwards 742a8c8560 Merge branch 'master' into test/m5-e2e-verification 2026-02-27 20:03:22 +00:00
freemo c9abb45adf test(coverage): add Behave BDD scenarios for 9 under-covered modules
Added 246 new BDD scenarios across 9 feature files to improve unit test
coverage for modules that were either entirely untested or had significant
coverage gaps:

- lock_service_coverage.feature (27 scenarios): validation branches,
  TTL boundaries, re-entrant acquisition, rollback on exceptions
- plan_apply_service_coverage.feature (54 scenarios): operation labels,
  diff rendering (plain/rich/json), artifact building, validation gate,
  changeset resolution and cleanup
- plan_executor_coverage.feature (51 scenarios): step parsing, execute
  actor integration, strategize/execute guards, stub retry/recovery,
  decision tree construction
- skill_cli_coverage_r3.feature (22 scenarios): tools refresh, list/show
  JSON fallback, capability summary errors, remove confirmation
- changeset_repository_coverage.feature (39 scenarios): entry/tool repos
  validation, database error wrapping, domain conversion, SQLite store
  CRUD operations
- repositories_coverage.feature (20 scenarios): get_by_name/namespace
  errors, list_available filters, delete with ActionInUseError, plan
  update with invariants/processing_state/error_details
- sandbox_copy_on_write_coverage.feature (12 scenarios): create OSError
  wrapping, get_path state transitions, commit edge cases, rollback
  errors, cleanup with missing paths
- bridge_coverage.feature (8 scenarios): __del__ suppression, async task
  cancellation, execute_graph message type handling, stream config,
  state checkpointer
- plan_cli_coverage.feature (13 scenarios): legacy apply/list/cd paths,
  use-action with estimation/invariant actors, lifecycle-apply guards,
  status errors, error recovery display

All 246 scenarios (1105 steps) pass. Step definitions use unique prefixes
to prevent ambiguous step conflicts with existing tests.

ISSUES CLOSED: #467
2026-02-27 13:47:42 -05:00
freemo a074b4846f fix(provider): remove FakeListLLM defaults
Remove FakeListLLM as a silent fallback in agent graph constructors
(plan_generation.py, context_analysis.py, auto_debug.py). All three now
raise ValueError when llm=None, making missing-provider errors explicit.

Add Settings.mock_providers flag and validate_provider_availability()
method. Update container.get_ai_provider() to check Settings.mock_providers
first, with env-var fallback for backward compatibility.

Add resolve_provider_by_name() helper to the provider registry and export
it from cleveragents.providers. Add structlog trace logging to
ProviderRegistry.get_default_provider_type() to record selection reasoning.

Update all existing behave step files, robot tests, and benchmarks that
relied on the implicit FakeListLLM default to pass an explicit LLM
instance instead.

Add new BDD tests (features/provider_fixes.feature with 17 scenarios),
Robot Framework integration tests (robot/provider_detection_smoke.robot),
and ASV benchmarks (benchmarks/provider_selection_bench.py).

ISSUES CLOSED: #323
2026-02-27 09:47:10 -05:00
aditya fa1794dba9 Merge pull request 'feat(skill): add MCP refresh hooks' (#455) from feature/m4-skill-registry-refresh into master
Reviewed-on: cleveragents/cleveragents-core#455
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 13:17:20 +00:00
aditya 9af9ff8d95 Merge branch 'master' into feature/m4-skill-registry-refresh 2026-02-27 12:27:10 +00:00
aditya ddebcad872 Merge pull request 'feat(cli): add skill tools and refresh commands' (#454) from feature/m4-skill-registry-cli into master
Reviewed-on: cleveragents/cleveragents-core#454
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 12:15:07 +00:00
aditya 174d1dc334 Merge branch 'master' into feature/m4-skill-registry-cli 2026-02-27 10:40:42 +00:00
aditya 9fd29b2748 Merge pull request 'feat(skill): add agent skills loader' (#453) from feature/m3-agent-skills-loader into master
Reviewed-on: cleveragents/cleveragents-core#453
Reviewed-by: Hamza Khyari <hamza.khyari@cleverthis.com>
2026-02-27 10:22:01 +00:00
aditya 4c4bd74d94 Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-27 09:34:00 +00:00
aditya 131be21e80 fix(tests): update actor_examples count to 7 after subplan example
Refs: #198
2026-02-27 09:24:34 +00:00
aditya 2b5b73c689 Merge branch 'master' into feature/m5-subplan-actor 2026-02-27 08:43:40 +00:00
aditya b888afab71 feat(actor): add plan_subplan tool and decision emission
Add builtin/plan-subplan tool for strategy actors to emit SUBPLAN_SPAWN
or SUBPLAN_PARALLEL_SPAWN decisions when decomposing a plan into child
plans. Implements all acceptance criteria from issue #198:

- SubplanPayload (Pydantic) validates goal, resource_scopes/project_ref
  (at least one required), merge_strategy, max_parallel (1-50), parallel
  flag, dependencies, and context_view override.
- Defaults: merge_strategy=git_three_way, max_parallel=5, parallel=False,
  dependencies=[]. Omitted fields inherit sensible values automatically.
- make_plan_subplan_spec(decision_service=None) factory supports optional
  DecisionService injection for persistent decision recording.
- _build_rationale() generates human-readable rationale text (goal, scope,
  execution mode, dependencies, context_view) surfaced in plan explain.
- register_subplan_tool() added to tool/builtins/__init__.py for bulk
  registration. PLAN_SUBPLAN_SPEC exported as the default ready-to-use spec.
- Actor YAML example (examples/actors/strategy_with_subplan.yaml) with
  annotated serial and parallel spawn payload examples.
- Behave BDD: 20 scenarios, 70 steps covering validation, defaults,
  decision type, rationale, service injection, registry, and ToolRunner.
- Robot Framework: 9 smoke tests via robot/plan_subplan_tool.robot.
- ASV benchmarks: benchmarks/subplan_actor_tool_bench.py (5 suites).
- Coverage: 100% on subplan_tool.py. Lint, typecheck, and security clean.

ISSUES CLOSED: #198
2026-02-27 08:02:48 +00:00
aditya ab9732c2a1 Merge pull request 'feat(skill): add MCP adapter for external tools' (#434) from feature/m3-mcp-adapter into master
Reviewed-on: cleveragents/cleveragents-core#434
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
2026-02-27 07:28:15 +00:00
aditya a0f56228e2 Merge branch 'master' into feature/m3-mcp-adapter 2026-02-27 06:40:15 +00:00
freemo 120274da99 feat(di): wire decision services
Create DecisionService application-layer service that wraps
DecisionRepository with structured logging and UnitOfWork transaction
management. Wire DecisionService and PlanLifecycleService into the DI
container as Factory providers.

Inject DecisionService into PlanLifecycleService so that phase
transitions automatically record decisions: start_strategize records a
strategy_choice decision and start_execute records an
implementation_choice decision. Decision recording is optional and
never blocks lifecycle transitions.

Add Behave feature (5 scenarios), Robot Framework smoke tests (2 test
cases), ASV benchmarks (3 benchmark classes), and DI reference
documentation.

ISSUES CLOSED: #173
2026-02-27 02:33:42 +00:00
brent.edwards f7b8186b3f Merge branch 'master' into test/m6-e2e-verification 2026-02-26 21:26:00 +00:00
brent.edwards 8771b195f5 Merge branch 'master' into test/m5-e2e-verification 2026-02-26 21:25:47 +00:00
hamza.khyari 13b1eb45a8 Merge pull request 'feat(security): enforce read-only actions' (#436) from feature/m4-security-readonly into master
Reviewed-on: cleveragents/cleveragents-core#436
Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-02-26 21:25:13 +00:00
brent.edwards 8c5fa8ace7 Merge branch 'master' into test/m5-e2e-verification 2026-02-26 20:58:38 +00:00
khyari hamza 493e5cf8a1 fix(security): wire read_only through ExecuteStubActor and replace stubs with real tests
- Add read_only kwarg to ExecuteStubActor.execute(), propagated to
  ChangeSetCapture; PlanExecutor._run_execute_with_stub passes
  plan.read_only through the execution path (P1 #2 fix)
- Replace CLI fail-fast and Action-Skill stub scenarios with real
  ExecuteStubActor spy test and SkillContext.enforce_write_guard
  integration tests (P2 #1 fix)
- Update docs: remove unimplemented Layer 5 (Action-Skill), add
  ExecuteStubActor wiring description, fix test command (P2 #2 fix)

ISSUES CLOSED: #322
2026-02-26 20:33:20 +00:00
khyari hamza 2b09568cfa feat(security): enforce read-only actions
Tighten ToolRuntime._enforce_capabilities() to block ANY tool with
writes=True when plan_read_only is set, removing the not-cap.read_only
loophole that allowed certain write tools through. Tool name is now
always included in the ToolAccessDeniedError message.

Add read_only flag to ChangeSetCapture with ReadOnlyViolationError
raised when write-capable tools are wrapped on a read-only plan.

Add CLI fail-fast guards on plan execute and plan apply commands that
abort before calling the service layer if plan.read_only is True.

SkillContext.enforce_write_guard() already included tool name
correctly and required no changes.

Includes 18 Behave scenarios (90 steps), Robot integration tests,
ASV benchmarks, and docs/reference/read_only_actions.md.

ISSUES CLOSED: #322
2026-02-26 20:33:20 +00:00
freemo 4ed057fa58 Docs: Fixed refs in docs 2026-02-26 14:40:17 -05:00
freemo c88b432beb Docs: Daily timeline update 2026-02-26 14:32:59 -05:00
brent.edwards 1ec2200e34 test(e2e): verify M6 success criteria — Firefox-scale autonomous porting
Robot Framework E2E test suite exercising the complete M6 success
criteria verification sequence:

- Porting action creation from YAML config via CLI
- Plan use + execute via CLI with mocked lifecycle service
- Hierarchical decomposition: 4+ levels (root + L1-L4 = 21+ plans)
- Decision correction recomputes only affected subtree (CorrectionImpact)
- Parallel execution scales to 15 concurrent subplans (10+ required)
- Realistic porting task: full ACTION → STRATEGIZE → EXECUTE → APPLY
  lifecycle with 10 subplans completing autonomously
- Plan apply transitions to APPLIED terminal state
- SubplanFailureHandler retry/stop-others logic verification
- SubplanConfig supports all execution modes and merge strategies
- Decision tree structure: PROMPT_DEFINITION root with
  SUBPLAN_PARALLEL_SPAWN children and superseded_by flow

Ten subcommands in the Python helper, each printing a sentinel
string on success. All subcommands verified passing locally.

Closes #407
2026-02-26 19:27:53 +00:00
brent.edwards 93f2d396a1 Merge pull request 'test: consolidated Brent QA batch — issues #179, #180, #404, #405, #187' (#442) from develop-brent-5 into master
Reviewed-on: cleveragents/cleveragents-core#442
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
2026-02-26 16:54:21 +00:00
brent.edwards fd1612c38f test(e2e): verify M5 success criteria — ACMS and large-project context
Robot Framework E2E test suite exercising the complete M5 success
criteria verification sequence:

- Large project creation with 10,000+ simulated files
- Resource registration and linking to a project
- Indexing verification (resource linked, project shows correctly)
- Context tier management (hot/warm/cold) via ContextConfig
- ACMS v1 context policy set/show with persistence round-trip
- Phase view inheritance (default → strategize → execute → apply)
- Scoped context output per ACMS phase with narrowing size limits
- Policy clear and inheritance fallback
- ContextView validation rules (size limits, VALID_PHASES)

Nine subcommands in the Python helper, each printing a sentinel
string on success. All subcommands verified passing locally.

Closes #406
2026-02-26 16:32:01 +00:00
brent.edwards ffdd2f2b19 Merge branch 'master' into develop-brent-5 2026-02-26 16:14:07 +00:00
brent.edwards 32b81793a2 refactor(test): rename decision persistence files to avoid conflict with master
Rename our serialization-focused decision persistence suites to
*_serialization to avoid add/add conflicts with the repository-based
decision persistence suites that landed on master independently:

- decision_persistence.feature -> decision_persistence_serialization.feature
- decision_persistence_steps.py -> decision_persistence_serialization_steps.py
- decision_persistence_bench.py -> decision_persistence_serialization_bench.py
- decision_persistence.robot -> decision_persistence_serialization.robot
- helper_decision_persistence.py -> helper_decision_persistence_serialization.py

Updated robot helper path, step docstring, and testing.md references.
2026-02-26 16:13:44 +00:00
hamza.khyari 074f884f14 Merge pull request 'fix(security): close async resources and leaks' (#435) from feature/m4-security-async-cleanup into master
Reviewed-on: cleveragents/cleveragents-core#435
2026-02-26 15:19:16 +00:00
aditya 76db58414a Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-26 14:51:41 +00:00
aditya f8bc02869a Merge branch 'master' into feature/m3-agent-skills-loader 2026-02-26 14:43:52 +00:00
aditya d0337a0881 fix(skill): eliminate Any from agent_skills_loader type signatures
Replace dict[str, Any] with a private _YamlMapping type alias
(dict[str, object]) in AgentSkillSpec Pydantic fields and all
internal helper function signatures. Add isinstance guard for
allowed-tools list iteration to satisfy pyright.

Refs: #160
2026-02-26 14:40:44 +00:00
khyari hamza c406781b86 fix(security): address PR #435 review feedback
P0: reject register() after close_all() with RuntimeError.
P1: catch CancelledError in close_all(), use WeakKeyDictionary for
cancellation_reasons to prevent memory leak, guard StateManager
update_state/reset/load_checkpoint/time_travel after close().
P2: contextlib.suppress in __del__ for partial construction, re-cancel
pending tasks in cleanup_tasks_async, handle late tasks added during
await window, guard AcpEventQueue.publish() after close with _is_closed
flag and is_closed property, fix ASV TimeRegisterBatch crash.
Tests: 5 new Behave scenarios (T1-T4 + is_closed), log handler and
event loop cleanup in after_scenario (T5-T6).
Docs: async_safety.md updated for register-after-close and state
mutation guards.

ISSUES CLOSED: #321
2026-02-26 14:30:43 +00:00
khyari hamza 5d5d3fde2c fix(security): close async resources and leaks
Add AsyncResourceTracker (core/async_cleanup.py) providing a central
registry for async resources with timeout-bounded close_all(), async
context manager support, and a __del__ finalizer that logs leaked
resources by name.

Enhance LangGraphBridge with cleanup_tasks_async() that awaits
in-flight tasks with a deadline instead of fire-and-forget cancel().
Add cancellation_reasons dict to trace why tasks were cancelled.

Add StateManager.close() to properly release checkpoint file handles
and complete the RxPY BehaviorSubject. Add AcpEventQueue.close() to
dispose all subscriptions.

Includes 14 Behave scenarios (67 steps), Robot integration tests,
ASV benchmarks, and docs/reference/async_safety.md.

ISSUES CLOSED: #321
2026-02-26 14:30:43 +00:00
aditya 7ad9f2bf4a chore: update CHANGELOG for feat(skill): add agent skills loader
ISSUE CLOSED #160
2026-02-26 14:28:09 +00:00
hamza.khyari db58a3bb62 Merge pull request 'feat(decision): add decision persistence layer' (#131) from feature/m4-decision-persistence into master
Reviewed-on: cleveragents/cleveragents-core#131
Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-02-26 14:25:27 +00:00
khyari hamza 7f2b1c61fc fix(decision): rebase migration onto merged changeset head
After rebasing onto master, m4_001_decision_tables and
d0_002_merge_changeset_and_locks were both leaf heads.  Update
down_revision so the decision migration chains after the merge
migration, restoring a single Alembic head.

ISSUES CLOSED: #171
2026-02-26 13:40:07 +00:00
khyari hamza 27bef69ddd feat(decision): add decision persistence
Add decision persistence layer with DecisionRepository, DecisionModel,
and Alembic migration. Includes tree queries (BFS traversal via deque,
path-to-root), superseded lookup, ordered decision path retrieval,
concrete Decision type annotations (via TYPE_CHECKING), and comprehensive
test coverage (Behave BDD, Robot Framework, ASV benchmarks). Updated
database_schema.md, CHANGELOG.md, and CONTRIBUTORS.md.

ISSUES CLOSED: #171
2026-02-26 13:40:07 +00:00
aditya 595cfd172f Merge branch 'master' into feature/m3-mcp-adapter 2026-02-26 13:30:29 +00:00