19 Commits

Author	SHA1	Message	Date
HAL9000	790eb6f001	test(data): introduce dynamic data generation and externalize test data in Behave and Robot Framework suites ... - Added Faker dependency to pyproject.toml for dynamic test data generation - Created features/test_data_factory.py with TestDataGenerator and ContextFragmentFactory classes for Behave tests - Created robot/helper_test_data_factory.py with RobotTestDataGenerator and factory classes for Robot Framework tests - Created features/test_data_loader.py to load externalized test data from JSON files - Created features/fixtures/test_data_samples.json with realistic test data samples - Updated robot/helper_acms_fusion.py to use dynamic test data generation instead of hardcoded values like "alpha" and "beta" - All quality gates passing: lint, typecheck, unit tests, integration tests, coverage ≥ 97% ISSUES CLOSED: #9048	2026-04-27 07:33:18 +00:00
hurui200320	8953449dc2	fix(actor): validate v3 YAML via ActorConfigSchema in agents actor add CLI ... - schema.py: provider field changed to Optional[str] with model validator validate_provider_required_for_llm_graph() that requires it only for LLM and GRAPH actor types; TOOL actors do not require provider - schema.py: is_v3_yaml() uses version_str == "3" or version_str.startswith("3.") to avoid false positives from "30" or "300" version strings - schema.py: tool namespace validation uses strict 2-part split to reject empty namespace or empty name (e.g. "/tool", "ns/", "a/b/c") - cli/commands/actor.py: schema_version extraction uses raw_version pattern (no # type: ignore[assignment]) for clean static typing - actor/__init__.py: is_v3_yaml removed from __all__ and _LAZY_IMPORTS since it is a module-private helper, not a public API - robot/actor_add_v3_schema_validation.robot: YAML fixtures for 'Reject v3 LLM Actor Without Model Field' and 'Reject v3 TOOL Actor Without Tools Field' now include required provider field (and model for TOOL fixture) - robot/helper_actor_add_v3_schema_validation.py: except clauses unified to catch (subprocess.TimeoutExpired, FileNotFoundError) in both add_actor() and update_actor() functions - features/actor_add_v3_schema_validation.feature: 'Update a v3 actor with valid YAML succeeds' scenario now includes 'And the actor should be validated via ActorConfigSchema'; error assertions tightened to exact messages (e.g. "Input should be 'llm', 'tool' or 'graph'", "Node ID must be alphanumeric", "must be namespaced") - features/steps/actor_add_v3_schema_validation_steps.py: step_run_actor_update now spies on ActorConfigSchema.model_validate; failure paths assert isinstance(result.exception, SystemExit) ISSUES CLOSED: #5869	2026-04-17 16:28:45 +08:00
CoreRasurae	007af498b8	refactor(autonomy): rename automation profile task flags to spec names ... Renamed all 11 task-type confidence threshold fields in AutomationProfile from phase-transition semantics to spec-defined task-type semantics. Updated all 8 built-in profiles, CLI formatting, YAML schema, services, and all Behave/Robot tests referencing the old field names. Post-review fixes: - Fixed 24 stale old field names in M6 fixture files (automation_profiles.json, autonomy_guardrails.json) - Added model_validator(mode='before') to detect legacy field names and raise actionable ValueError with rename mapping - Added semantic bridge comments in PlanLifecycleService mapping task-type thresholds to phase-transition gates - Added threshold_field to structured log messages for observability - Restored categorised CLI automation-profile show output to match spec (Phase Transitions / Decision Automation / Self-Repair / Execution Controls) instead of flat list - Added missing access_network field to spec show output examples (Rich, Plain, JSON, YAML variants) - Aligned ADR-017 profile fields table to all 11 fields with descriptions matching spec Automatable Tasks table - Aligned automation_profiles.md threshold descriptions with spec - Added spec section references in phase_reversion.md, error_recovery.md, and plan_execute.md for field naming context - Extended repository roundtrip test to assert all 11 threshold fields - Fixed benchmark _make_profile() passing safety fields as top-level kwargs instead of via SafetyProfile sub-model (incompatible with extra="forbid") - Aligned CLI JSON/YAML output structure for automation-profile show with the specification grouped format (phase_transitions, decision_automation, self_repair, execution_controls) - Moved safety boolean fields into the Execution Controls section of Rich output per spec examples - Reverted auto profile description to "Fully automatic except apply" per specification (line 16703, line 28406) - Improved bridge comments in test steps with semantic context for threshold-to-gate mappings ISSUES CLOSED: #902	2026-03-30 13:18:07 +01:00
brent.edwards	b88bc0ec1b	feat(perf): large project scaling tests (#984) ... ## Summary Add large project scaling benchmarks and tests at production scale (10K–100K files). ### New ASV Benchmarks **IndexingScalingSuite** (`large_project_scaling_bench.py`): - `time_walk_and_index` at 1K/10K/50K/100K files - `time_incremental_refresh` (1% modified files) - `track_indexed_file_count`, `track_tokens_per_second` **ContextAssemblyScalingSuite** (`context_assembly_scaling_bench.py`): - `time_full_pipeline` at 100/1K/5K/10K fragments - `time_tiered_strategy`, `time_recency_strategy` - `track_assembled_tokens`, `track_fragments_per_second` **ExecutionThroughputSuite** (`execution_throughput_bench.py`): - `time_sequential_plans` at 10/50/100 plans - `time_executor_construction`, `time_decision_tree_scaling` ### Scale Fixture Updates - Added `xlarge` (50K files) and `xxlarge` (100K files) profiles to `scale_metadata.json` - Added 50K/100K thresholds to `baseline_thresholds.json` - Added `context_assembly` and `execution_throughput` threshold sections ### Tests & Documentation - 15 Behave scenarios validating profiles, thresholds, monotonicity, memory budgets - 6 Robot integration tests including live 1K-file indexing throughput check - `docs/reference/scaling_baselines.md` documenting all baseline metrics ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,910 scenarios) | | `nox -s integration_tests` | PASS (1,526 tests) | | `nox -s coverage_report` | 97% (>= 97%) | Closes #859 Reviewed-on: #984 Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>	2026-03-21 04:46:45 +00:00
freemo	ec0b7631d0	refactor(a2a): rename ACP module and symbols to A2A standard ... Renamed src/cleveragents/acp/ to src/cleveragents/a2a/ and all 13 Acp* classes to A2a* per ADR-047 (A2A Standard Adoption). Updated all imports, structlog event names (acp.* → a2a.*), field names (acp_version → a2a_version), and test references across the entire codebase. This is a cosmetic rename only — no behavioral changes. ISSUES CLOSED: #688	2026-03-12 14:38:57 +00:00
CoreRasurae	7e6f6fae37	test(validation): add semantic validation suites ... Add comprehensive semantic validation test suites covering five new fixture categories: language porting mismatches, dependency graph violations, API surface changes, cross-file symbols, and circular import detection. New BDD scenarios (38) exercise all six built-in rules against fixture-driven inputs. Robot Framework integration tests (11) validate end-to-end rule execution via the SemanticValidationService. ASV benchmarks (6 suites) establish performance baselines for batch validation throughput and per-rule latency. Files added: - features/fixtures/validation/{language_porting_mismatches, dependency_graph_violations, api_surface_changes, cross_file_symbols, circular_import_detection}.json - features/semantic_validation_suite.feature - features/steps/semantic_validation_suite_steps.py - robot/semantic_validation_suite.robot - robot/helper_semantic_validation_suite.py - benchmarks/semantic_validation_suite_bench.py - docs/reference/semantic_validation_coverage.md All 11 nox sessions pass (lint, format, typecheck, security_scan, dead_code, unit_tests, integration_tests, docs, build, benchmark, coverage_report). Coverage remains at 97%. ISSUES CLOSED: #316	2026-03-02 22:05:53 +00:00
brent.edwards	38e75342a9	refactor: merge from master ... This code is a merge from master.	2026-02-28 05:49:26 +00:00
brent.edwards	348c230bc5	test(e2e): add M6 autonomy acceptance suite ... Add comprehensive M6 autonomy acceptance test suites covering the ACP local-mode facade, autonomy guardrails, automation profile resolution, event queue pub/sub, HTTP transport stub, and version negotiation. Behave suite (52 scenarios): - ACP facade dispatch for all 11 operations - Guard enforcement (denylist, allowlist, budget, call limit, write/apply) - Automation profile built-in validation and custom creation - Profile resolution precedence (plan > action > project > global) - Event queue lifecycle (publish, subscribe, unsubscribe, close) - HTTP transport stub rejection in local mode - ACP version negotiation (accept/reject) - Model validation (AcpRequest, AcpResponse, AcpEvent, AcpErrorDetail) Robot integration suite (11 tests): - Facade session/plan lifecycle, unknown operation error - Event queue publish/subscribe, transport stub, version negotiation - Guard denylist/budget enforcement, profile resolution - Fixture loading, full end-to-end flow ASV benchmarks (5 suites): - Facade dispatch, guard evaluation, profile resolution - Event queue operations, fixture loading Fixtures: acp_facade_flows.json, autonomy_guardrails.json, automation_profiles.json Closes #211	2026-02-27 20:31:15 +00:00
brent.edwards	ece5e61725	test(e2e): add M5 ACMS + context suites	2026-02-27 20:20:13 +00:00
brent.edwards	c80cda2590	Merge feature/m4-correction-subplan-smoke into develop-brent-5	2026-02-25 22:11:33 +00:00
brent.edwards	917c2bc546	test(e2e): add M4 correction + subplan suites	2026-02-25 21:51:26 +00:00
brent.edwards	ace7311de4	test(e2e): add M3 decision + validation suites	2026-02-25 21:26:19 +00:00
brent.edwards	e230700857	Merge feature/m1-e2e-sourcecode into develop-brent-4	2026-02-25 06:36:45 +00:00
brent.edwards	0397a00eb6	test(e2e): add M1 source-code plan lifecycle suite	2026-02-25 04:50:43 +00:00
brent.edwards	093a74953f	test(e2e): add M2 actor + tool source smoke suite ... Add comprehensive E2E test suite for M2 (Actor Graphs + Tool Sources) epic: - Behave BDD: 10 scenarios covering actor YAML loading, skill registry, tool lifecycle (discover/activate/execute/deactivate), and MCP stub - Robot Framework: 6 integration tests via CLI helper script - ASV benchmarks: 12 benchmarks for actor loading, skill registry, tool lifecycle, and MCP stub performance baselines - MCP stub server mock: in-process fake with 3 tools (search/fetch/transform) - Fixtures: hierarchical graph actor YAML + skill pack with tool refs and inline tools - Docs: updated testing.md with M2 smoke suite section Closes #169	2026-02-24 21:55:24 +00:00
brent.edwards	a181b774c9	Merge branch 'feature/m6-perf-scale' into develop-brent-2 ... # Conflicts: # docs/development/testing.md	2026-02-20 16:31:01 +00:00
brent.edwards	0dfdc02564	test(perf): add scale test fixtures	2026-02-20 03:31:43 +00:00
brent.edwards	558bcaf586	test(validation): add edge case suites	2026-02-20 02:54:08 +00:00
freemo	3cf6a812ae	feat: ported in run behavior as a subcommand of actor	2026-02-04 14:43:08 -05:00