49 Commits

Author	SHA1	Message	Date
freemo	2688c85769	feat(extensibility): implement Custom Sandbox Strategy Registration via SandboxStrategy Protocol ... Implement SandboxStrategyProtocol, a 9-method @runtime_checkable Protocol enabling third-party sandbox strategy registration. Includes: - SandboxStrategyProtocol with create/read/write/diff/commit/rollback/checkpoint/ restore_checkpoint/cleanup methods - SandboxRef (frozen dataclass) and DiffView/DiffEntry (Pydantic models) - SandboxStrategyRegistry with config-driven registration, Protocol validation, thread safety, and clear/list/has operations - BuiltInSandboxStrategyAdapter wrapping existing Sandbox implementations to conform to the new Protocol - CustomStrategyConfig for YAML/dict-based strategy registration - SandboxFactory integration with custom_registry parameter, has_custom_strategy() and get_custom_strategy_class() - 25 Behave BDD scenarios (85 steps) covering protocol, registry, adapter, config, and factory integration - 8 Robot Framework integration tests with real filesystem operations - ASV benchmarks for registry and adapter operations - Developer documentation ISSUES CLOSED: #586	2026-03-16 22:50:27 +00:00
freemo	ce722ed0ea	ops(ci): configure LLM API keys in Forgejo CI for integration test execution ... Updated CI pipeline to inject ANTHROPIC_API_KEY and OPENAI_API_KEY secrets as environment variables during Robot Framework integration test execution. Added CI setup documentation. ISSUES CLOSED: #701	2026-03-14 22:34:18 +00:00
freemo	d5b122d4a3	Docs: Updated to A2A and integrating rest standard	2026-03-11 13:19:55 -04:00
Brent E. Edwards	d88dad94f6	fix(test): resolve race condition in M4 validation integration test ... Three-pronged fix for intermittent pabot-parallel race condition in M4 validation integration tests: 1. Composable Setup Database Isolation keyword in common.resource gives each suite a unique CLEVERAGENTS_DATABASE_URL so concurrent pabot workers never contend on the same SQLite file. 2. Per-suite CLEVERAGENTS_HOME directories prevent shared temp directory cleanup from racing between workers. 3. Centralised reset_global_state() in robot/helpers_common.py clears Settings singleton, DI container, provider registry, and engine cache between chained CLI invocations in helper processes. Also: - Setup Test Environment now accepts optional mock_ai and auto_apply_migrations arguments (default TRUE) for backward compatibility while allowing suites to opt out. - Added Suite Teardown to cli_plan_context_commands.robot. - Fixed _COMMANDS typing in two helpers to eliminate type: ignore. - Updated docs/development/testing.md to reflect helpers_common delegation pattern. - Added timeout=30s to all Run Process calls in m4_e2e_verification.robot. Fixes: #563	2026-03-10 19:04:57 +00:00
freemo	74772280e6	perf(tests): optimize coverage instrumentation and reporting pipeline ... Replace coverage.py (sys.settrace-based) with slipcover (bytecode-based instrumentation) for significantly faster coverage collection: - Each behave-parallel worker runs under slipcover, producing per-feature JSON coverage files with unique UUIDs to avoid write contention. - After all workers finish, slipcover --merge combines per-worker data into a single build/coverage.json report. - XML report generated via slipcover --merge --xml for CI tooling. - Terminal report with --fail-under=97 threshold enforcement. - Robust JSON key-fallback logic handles both slipcover and coverage.py output formats. - CI workflow (ci.yml, nightly-quality.yml) updated with defensive key lookup instead of hardcoded coverage.py format. - Documentation updated to reflect slipcover as the coverage tool. - CHANGELOG.md updated. ISSUES CLOSED: #482	2026-03-02 02:01:27 +00:00
brent.edwards	38e75342a9	refactor: merge from master ... This code is a merge from master.	2026-02-28 05:49:26 +00:00
brent.edwards	348c230bc5	test(e2e): add M6 autonomy acceptance suite ... Add comprehensive M6 autonomy acceptance test suites covering the ACP local-mode facade, autonomy guardrails, automation profile resolution, event queue pub/sub, HTTP transport stub, and version negotiation. Behave suite (52 scenarios): - ACP facade dispatch for all 11 operations - Guard enforcement (denylist, allowlist, budget, call limit, write/apply) - Automation profile built-in validation and custom creation - Profile resolution precedence (plan > action > project > global) - Event queue lifecycle (publish, subscribe, unsubscribe, close) - HTTP transport stub rejection in local mode - ACP version negotiation (accept/reject) - Model validation (AcpRequest, AcpResponse, AcpEvent, AcpErrorDetail) Robot integration suite (11 tests): - Facade session/plan lifecycle, unknown operation error - Event queue publish/subscribe, transport stub, version negotiation - Guard denylist/budget enforcement, profile resolution - Fixture loading, full end-to-end flow ASV benchmarks (5 suites): - Facade dispatch, guard evaluation, profile resolution - Event queue operations, fixture loading Fixtures: acp_facade_flows.json, autonomy_guardrails.json, automation_profiles.json Closes #211	2026-02-27 20:31:15 +00:00
brent.edwards	ece5e61725	test(e2e): add M5 ACMS + context suites	2026-02-27 20:20:13 +00:00
brent.edwards	32b81793a2	refactor(test): rename decision persistence files to avoid conflict with master ... Rename our serialization-focused decision persistence suites to *_serialization to avoid add/add conflicts with the repository-based decision persistence suites that landed on master independently: - decision_persistence.feature -> decision_persistence_serialization.feature - decision_persistence_steps.py -> decision_persistence_serialization_steps.py - decision_persistence_bench.py -> decision_persistence_serialization_bench.py - decision_persistence.robot -> decision_persistence_serialization.robot - helper_decision_persistence.py -> helper_decision_persistence_serialization.py Updated robot helper path, step docstring, and testing.md references.	2026-02-26 16:13:44 +00:00
brent.edwards	3d3c8fda33	Merge feature/m4-decision-tests into develop-brent-5	2026-02-26 02:15:01 +00:00
brent.edwards	b3f126b147	Merge feature/m3-decision-validation-smoke into develop-brent-5	2026-02-26 02:14:14 +00:00
brent.edwards	b99510d016	fix(test): derive decision type count from enum instead of hard-coding 11 ... Replace hard-coded '11 decision types' with dynamic len(DecisionType) in the assertion and remove the literal from scenario names, step patterns, docstrings, and docs so the tests stay correct when new DecisionType values are added.	2026-02-26 01:19:28 +00:00
brent.edwards	93730067a2	fix(test): address review findings in M3 smoke tests ... - Replace dead after_scenario() hook with context.add_cleanup() calls so patchers are properly stopped (Behave only runs hooks from environment.py) - Fix invalid ULID constants: pad DECISION/CORRECTION to 26 chars, replace invalid 'I' with 'J' in INVARIANT_ULID - Add temp file cleanup for NamedTemporaryFile(delete=False) via context.add_cleanup() in behave steps and try/finally in robot helper - Remove decision-persistence doc sections from testing.md that belong in PR #438, not this PR	2026-02-26 01:13:16 +00:00
brent.edwards	c80cda2590	Merge feature/m4-correction-subplan-smoke into develop-brent-5	2026-02-25 22:11:33 +00:00
brent.edwards	917c2bc546	test(e2e): add M4 correction + subplan suites	2026-02-25 21:51:26 +00:00
brent.edwards	ace7311de4	test(e2e): add M3 decision + validation suites	2026-02-25 21:26:19 +00:00
brent.edwards	f4c4884d03	test(persistence): add decision persistence suites ... Add comprehensive persistence test suites for the Decision domain model covering serialization round-trips (model_dump/model_validate, JSON), context-snapshot persistence, correction-chain reconstruction, and decision-tree reconstruction from serialized data. Deliverables: - Behave: 21 scenarios in features/decision_persistence.feature - Robot Framework: 7 integration tests in robot/decision_persistence.robot - ASV benchmarks: 4 suites (17 benchmarks) in benchmarks/decision_persistence_bench.py - Documentation: Updated docs/development/testing.md ISSUES CLOSED: #180	2026-02-25 21:19:57 +00:00
brent.edwards	3b58432ecc	docs(merge): merge new docs into branch	2026-02-25 19:41:56 +00:00
CoreRasurae	57fcc880c3	feat(apply): run validation attachments ... ISSUES CLOSED: #176	2026-02-25 10:03:21 +00:00
brent.edwards	e230700857	Merge feature/m1-e2e-sourcecode into develop-brent-4	2026-02-25 06:36:45 +00:00
brent.edwards	dbc9be9fc0	Merge feature/m2-actor-tool-smoke into develop-brent-4	2026-02-25 06:35:44 +00:00
brent.edwards	0397a00eb6	test(e2e): add M1 source-code plan lifecycle suite	2026-02-25 04:50:43 +00:00
brent.edwards	9bf87a20d9	chore(pr): address code review feedback for PR #421 ... - Added CHANGELOG.md entry for M2 E2E test suite (CONTRIBUTING.md §6) - Added Brent E. Edwards to CONTRIBUTORS.md (CONTRIBUTING.md §8) - Fixed docs/development/testing.md: corrected M2 Behave scenario count from 11 to 10 to match actual feature file - Added standard explanatory comments to benchmark sys.path setup for consistency with project convention (advisory) - Replaced untyped lambda handler with typed _m2_noop_handler function in step definitions (advisory) All nox stages pass: lint, typecheck, format, unit_tests (280 features / 5846 scenarios / 0 failures), integration_tests (682 passed), coverage_report (97.2%). ISSUES CLOSED: #169	2026-02-25 04:13:55 +00:00
brent.edwards	093a74953f	test(e2e): add M2 actor + tool source smoke suite ... Add comprehensive E2E test suite for M2 (Actor Graphs + Tool Sources) epic: - Behave BDD: 10 scenarios covering actor YAML loading, skill registry, tool lifecycle (discover/activate/execute/deactivate), and MCP stub - Robot Framework: 6 integration tests via CLI helper script - ASV benchmarks: 12 benchmarks for actor loading, skill registry, tool lifecycle, and MCP stub performance baselines - MCP stub server mock: in-process fake with 3 tools (search/fetch/transform) - Fixtures: hierarchical graph actor YAML + skill pack with tool refs and inline tools - Docs: updated testing.md with M2 smoke suite section Closes #169	2026-02-24 21:55:24 +00:00
brent.edwards	2c21611ad5	test(cli): cover action and plan extensions ... Added comprehensive test coverage for CLI extension features from #325: - Behave scenarios for automation_profile resolution, invariant ordering, and actor override error cases - Output snapshot assertions for JSON/YAML/table formats - Robot integration test for action show with optional actors/invariants - ASV benchmark for extended CLI scenario runtime baseline - Updated testing.md with CLI extension test fixture documentation ISSUES CLOSED: #326	2026-02-24 20:11:42 +00:00
brent.edwards	a181b774c9	Merge branch 'feature/m6-perf-scale' into develop-brent-2 ... # Conflicts: # docs/development/testing.md	2026-02-20 16:31:01 +00:00
brent.edwards	5c323470e2	Merge branch 'feature/m6-validation-edge' into develop-brent-2	2026-02-20 16:30:19 +00:00
brent.edwards	4d41459800	Merge branch 'feature/m6-review-playbook' into develop-brent-2	2026-02-20 16:30:15 +00:00
brent.edwards	2cfa9eafbc	Merge branch 'feature/m5-subplan-tests' into develop-brent-2	2026-02-20 16:30:11 +00:00
brent.edwards	06c01a1d7b	Merge branch 'feature/m1-persistence-tests-robot' into develop-brent-2	2026-02-20 16:29:11 +00:00
brent.edwards	0dfdc02564	test(perf): add scale test fixtures	2026-02-20 03:31:43 +00:00
brent.edwards	558bcaf586	test(validation): add edge case suites	2026-02-20 02:54:08 +00:00
brent.edwards	f182148d82	test(persistence): add Robot persistence coverage	2026-02-20 00:28:15 +00:00
brent.edwards	4f90d7703b	test(cli): add Robot lifecycle CLI coverage	2026-02-19 22:44:34 +00:00
brent.edwards	2ba753c799	test(domain): add subplan model suites	2026-02-19 22:07:56 +00:00
brent.edwards	83dcfe35dc	docs(qa): add review playbook and priority matrix	2026-02-19 22:00:53 +00:00
brent.edwards	6f09f23d29	Merge branch 'feature/m1-persistence-tests' into develop-20260217 ... # Conflicts: # docs/development/testing.md	2026-02-18 02:43:58 +00:00
brent.edwards	f912a136b7	test(persistence): add plan/action persistence suites	2026-02-17 20:25:12 +00:00
brent.edwards	b8d2bfe252	test(cli): expand lifecycle command coverage	2026-02-17 20:01:18 +00:00
brent.edwards	89e6bdaf35	fix(qa): merge min-coverage into adv-security and resolve conflicts	2026-02-13 22:38:44 +00:00
brent.edwards	bb3c4ffaef	fix(qa): resolve merge conflicts with master and fix coverage threshold test ... Merged master into feature/q0-min-coverage. Resolved conflicts in: - benchmarks/coverage_report_bench.py: combined AST and regex benchmarks - docs/development/testing.md: merged both versions into comprehensive guide - robot/coverage_threshold.robot: merged test cases from both branches - implementation_plan.md: kept both Brent and Luis notes Fixed coverage_threshold_config test that failed because it expected a literal --fail-under=97 but our noxfile uses the COVERAGE_THRESHOLD constant via f-string. Updated step to resolve the constant via AST.	2026-02-13 22:25:16 +00:00
brent.edwards	75dc57845a	feat(qa): align security scans with nox	2026-02-13 19:24:24 +00:00
brent.edwards	35c74d1a52	feat(qa): enforce coverage >=97% in CI	2026-02-13 18:54:15 +00:00
freemo	9b30421b34	feat(db): add spec-aligned action and plan tables with migrations, ORM models, and benchmarks	2026-02-13 13:05:20 -05:00
brent.edwards	e801eb1ee8	feat(ci): add nox-based PR validation workflow ... - Rewrite .forgejo/workflows/ci.yml to route all jobs through nox sessions - Fix coverage_report nox session: serial behave mode replaces broken parallel mode (22% -> 97% accuracy), raise fail-under from 85% to 97% - Pass posargs through format nox session for CI --check support - Add 11 CI workflow validation scenarios (Behave) + Robot smoke test + ASV bench - Add 108 new Behave scenarios covering 6 largest coverage gaps to reach 97%: yaml_template_engine, actor/config, actor/registry, message_router, context_analysis, context_service - Update docs/development/ci-cd.md with nox-based CI docs and 97% threshold - Restore implementation_plan.md verbose style, check off completed CI tasks Verified: 1673 scenarios pass, 97% coverage, lint clean, typecheck clean	2026-02-12 22:01:51 +00:00
freemo	a4b56fa308	Docs: Cleaned up some minor issues like docs not being in nav	2026-02-12 11:08:54 -05:00
brent.edwards	7caac2714e	refactor: changes requested by Rui ... Removed `run-semgrep.sh` and automatically run setup-dev.sh	2026-02-11 00:59:58 +00:00
brent.edwards	395df78b94	test(qa): add validation fixtures, edge-case scenarios, and quality baseline ... Complete Brent QA tasks Q1.5, 10B.4, 10C.1, and 10C.4 on the Q0-quality-automation branch. - Add edge_case_plan_scenarios.feature (26 scenarios) covering concurrent plan execution, resource conflicts, validation failure chains, and rollback edge cases - Add validation_test_fixtures.feature (34 scenarios) covering AST security, lambda validation, content sanitization, project model validation, change-list coercion, and ActionArgument parsing - Add branch protection rules documentation in docs/development/ci-cd.md - Establish quality metrics baseline (96% coverage, 0 lint/type/security findings) - Fix missing langchain-anthropic dependency in pyproject.toml - Fix Rich table column wrapping in plan_lifecycle_cli_steps.py by patching console width during tests Suite: 107 features, 1673 scenarios, 7777 steps — all passing.	2026-02-10 22:19:21 +00:00
brent.edwards	7ef5ebb695	feat: Add Q0: Pre-commit hooks setup. ... This should automatically check for problems on build.	2026-02-10 16:04:17 +00:00