2746 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
aditya	d9e51d98f8	fix(tui,cli,tests): harden persona/input modes and stabilize parallel test execution ... - Add shell safety controls for REPL/TUI (`looks_dangerous`, confirmation gate, timeout handling, and env-based shell disable guard). - Secure persona workflows with strict name/path validation, safe import/export resolution, atomic+locked registry writes, and malformed YAML resilience. - Unify persona models and wiring by reusing canonical TUI schema/registry, adding DI providers, and lazy-loading TUI exports to avoid circular imports. - Improve reference discovery with ignored-directory filtering, symlink-safe walking, and TTL caching for CLI/TUI reference catalogs. - Expand Behave/Robot coverage for safety/error paths and parallel-isolation behavior; add shared `features/mocks/fake_repl_input.py` helper. - Fix parallel-run flakiness via deterministic cleanup/reload patterns and watchdog polling fallback when inotify limits are reached. ISSUES CLOSED: #695	2026-03-18 10:09:06 +00:00
aditya	c02f3842ae	feat(tui): implement persona system and reference/command input modes ... - Add `agents tui` command wiring and Textual app scaffolding for interactive TUI startup. - Implement TUI persona schema/registry/state with local YAML persistence and per-session persona binding. - Add three input-mode flows: Normal (`@` references), Command (`/` slash commands), and Shell (`!` passthrough). - Introduce TUI widgets/overlays for prompt, persona bar, reference picker, and slash command interactions. - Extend REPL command routing to support persona/session commands and reference/shell dispatch behavior. - Add test coverage: Behave features + step definitions, Robot TUI smoke test, and ASV fuzzy-reference benchmark. ISSUES CLOSED: #695	2026-03-18 10:09:05 +00:00
hurui200320	48ecf4c00c	fix(cli): add --execution-env-priority flag to plan use (#972) ... ## Summary Adds the missing `--execution-env-priority` flag to the `agents plan use` command, aligning the CLI with the specification (spec line 12501). The flag accepts `fallback` (default) or `override` and controls execution environment routing precedence per ADR-043: - **`override`**: The specified execution environment always wins, bypassing devcontainer auto-detection. - **`fallback`**: The specified environment defers to auto-detected devcontainers or project-level overrides. ### Changes - **Domain model** (`cleveragents.domain.models.core.plan`): - Added `ExecutionEnvPriority` StrEnum with `FALLBACK`/`OVERRIDE` values. - Changed `execution_env_priority` field type to `ExecutionEnvPriority | None` (leverages Pydantic enum validation). - Added `@model_validator` enforcing that `execution_env_priority` requires `execution_environment` (domain-level fail-fast invariant). Verified `validate_assignment=True` is set on `Plan.model_config`. - Updated `Plan.as_cli_dict()` to include `execution_environment` and `execution_env_priority`, with fallback default of `"fallback"` for pre-migration data where `execution_env_priority` is `None`. - **CLI** (`cleveragents.cli.commands.plan`): - Added `--execution-env-priority` parameter to `use_action`. - Validation: priority requires `--execution-environment`, enum value validation, case-insensitive input. - Defaults to `"fallback"` when `--execution-environment` is set without explicit priority. - Updated `_print_lifecycle_plan` and `_plan_spec_dict` to display the priority, defaulting to `"fallback"` for pre-migration data. - Updated `_plan_spec_dict` docstring to mention new keys. - Hoisted `ExecutionEnvPriority` import to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan` (no longer conditional on execution environment being set). - Guarded `service.save_plan(plan)` with `has_overrides` flag so it is only called when CLI overrides were actually applied. - **Persistence** (`cleveragents.infrastructure.database`): - Added `execution_environment` (`String(255)`, nullable) and `execution_env_priority` (`String(20)`, nullable) columns to `LifecyclePlanModel`. - Updated `from_domain()`/`to_domain()` for round-trip serialization including `ExecutionEnvPriority` enum reconstruction. Uses direct attribute access (`plan.execution_environment`) instead of defensive `getattr` — Plan fields always exist on the Pydantic BaseModel. - Updated `LifecyclePlanRepository.update()` to persist both fields using direct attribute access. - Added Alembic migration `m4_003_plan_env_columns` adding both columns to the `v3_plans` table. Uses `String(255)` for `execution_environment` to accommodate namespaced resource names. Descends from `m6_005_profile_guards_json`. - **Service** (`cleveragents.application.services.plan_lifecycle_service`): - Added `save_plan()` public convenience method for callers that need to re-persist after post-creation mutations. - **Tests**: - 18 Behave scenarios covering: - CLI acceptance criteria (valid values, defaults, validation errors, output display, case-insensitive input, service invocation with `call_args` verification). - Domain model validator invariant: construction with priority but no environment raises `ValueError`; construction with both fields succeeds. - `ExecutionEnvPriority` enum: values verification, `StrEnum` subclass assertion. - `Plan.as_cli_dict()`: includes both fields when set, omits both when `None`, defaults priority to `"fallback"` for pre-migration data. - DB round-trip serialization: `from_domain()` → `to_domain()` preserves both fields; preserves `None` values. - 5 Robot Framework integration tests. - Updated pre-existing `SimpleNamespace`-based plan test fixtures in `database_models_lifecycle_coverage_steps`, `database_models_new_coverage_steps`, `database_models_coverage_r2_steps`, and `repositories_error_handling_coverage_steps` to include `execution_environment` and `execution_env_priority` attributes. - Simplified Robot helper `sys.path` pattern to standard approach. - **Changelog**: Updated per CONTRIBUTING.md requirements. ### Review Fixes (Brent Edwards, Review #2384) - **P2 #1 — Defensive `getattr`**: Replaced `getattr(plan, "execution_environment", None)` with direct `plan.execution_environment` access in both `LifecyclePlanModel.from_domain()` and `LifecyclePlanRepository.update()`. Plan is a Pydantic BaseModel with `default=None`, so the field always exists. Updated 4 pre-existing `SimpleNamespace`-based test fixtures to include the new attributes. - **P2 #2 — Late conditional import**: Hoisted `ExecutionEnvPriority` import from inside conditional blocks to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan`. - **P3 — Migration naming**: Acknowledged as deferred (cosmetic only). Updated `m4_003` to descend from `m6_005_profile_guards_json` (post-rebase chain fix). - **Rebase**: Branch rebased onto latest `master` with merge conflict in `CHANGELOG.md` resolved. ### Deferred Items - **Partial failure atomicity** (#8 from review): If `save_plan()` fails after `use_action()` succeeds, the plan exists in the DB without CLI overrides. This requires service-layer restructuring beyond the scope of this ticket. - **Alembic migration naming** (#11 from review): Migration `m4_003` depends on `m6_005`, creating non-sequential naming. Renaming an existing migration risks breaking the chain for anyone who has already applied it. ### Quality Gates | Session | Result | |---------|--------| | lint | PASS | | typecheck | PASS (0 errors) | | unit_tests | PASS (11,125 scenarios, 0 failures) | | integration_tests | PASS (1,562 tests, 0 failures) | | coverage_report | 97% (threshold: 97%) | Closes #886 Reviewed-on: #972 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-18 08:13:56 +00:00
aditya	20efab9021	Merge pull request 'feature/m6-estimation-actor-yaml' (#975) from feature/m6-estimation-actor-yaml into master ... Reviewed-on: #975 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-18 07:23:14 +00:00
aditya	2764fcef5c	fix(actor,preflight,tests): resolve PR #975 review findings and stabilize full-suite coverage runs ... Address review-driven fixes across actor schema, preflight guardrails, docs/examples, and Behave/Robot coverage: unify preflight warning behavior with shared role-warning logic, resolve actor-name to config payloads in production preflight flow, harden response_format validation/coercion edge cases, extract duplicated helper logic, and expand negative-path test coverage. Also fix cross-scenario patcher leakage in step modules to eliminate full-run-only coverage failures.	2026-03-18 06:58:39 +00:00
aditya	60aeeb718f	feat(actor): update the changelog file aligning it with previous commit	2026-03-18 06:58:28 +00:00
aditya	26ad778aee	feat(estimation): add estimation actor YAML template and role-aware registration validation ... - Add `role_hint` and `response_format` support to actor schema. - Add non-fatal estimation-role compatibility warnings in actor registration CLI flows. - Add preflight warning path when `estimation` actor is missing `response_format`. - Add `examples/actors/estimator.yaml` and update actor examples documentation/tests. - Update integration helper expectations (m1/m2/m3/m6) for missing provider config in local test env. ISSUES CLOSED: #650	2026-03-18 06:46:14 +00:00
hamza.khyari	ab1fd19bcd	Merge pull request 'feat(resource): add deferred virtual resource types' (#663) from feature/post-resource-types-virtual into master ... Reviewed-on: #663 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-18 02:40:31 +00:00
hamza.khyari	5d6cb099ad	feat(resource): add deferred virtual resource types ... Add 3 deferred virtual resource types (remote, submodule, symlink) with equivalence metadata for physical-to-virtual resource linking. Depends on: #662 (child_types reference types introduced by #662) - Type definitions extracted to _resource_registry_virtual_deferred.py for consistency with _resource_registry_virtual.py (#329) - YAML configs with equivalence criteria per spec, spec reference comments - Bootstrap registration via BUILTIN_TYPES spread, hidden from resource add scaffolding (user_addable: false) - Equivalence structural validation in ResourceTypeSpec model validator: criteria must be a non-empty list of non-empty strings; virtual types must have sandbox_strategy=none, user_addable=false, handler=None, all capabilities false - Behave tests (52 scenarios), Robot tests (7), ASV benchmarks - DB roundtrip tests verifying virtual types survive bootstrap persistence - Negative tests: missing equivalence/name/kind, manual add rejection for all 3 virtual types (register_resource guard), invalid criteria elements (non-string, empty string) ISSUES CLOSED: #331	2026-03-18 02:30:00 +00:00
hamza.khyari	3618bf4f7e	Merge pull request 'feat(resource): add virtual core resource types' (#661) from feature/post-resource-types-virtual-core into master ... Reviewed-on: #661 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-18 01:52:25 +00:00
hamza.khyari	c14ce65d61	feat(resource): add virtual core resource types ... Add 6 built-in virtual resource types (file, directory, commit, branch, tag, tree) with equivalence metadata for content-hash and git-object identity matching. - YAML configs under examples/resource-types/ - Bootstrap registration with virtual types hidden from resource add scaffolding - Equivalence criteria per spec (content_hash, merkle_hash, git SHA identity) - Behave tests (83 scenarios), Robot tests, ASV benchmarks - Documentation in docs/reference/resource_types_builtin.md ISSUES CLOSED: #329	2026-03-18 01:45:25 +00:00
freemo	758dafd8fa	Docs: daily update to timeline	2026-03-17 14:45:41 -04:00
freemo	c65e8a5285	feat(resource): add cloud infrastructure resources ... Implement cloud resource types (aws, gcp, azure) with credential fields, region/tenant metadata, and stubbed sandbox strategies. Credential resolution uses environment variables and profile names with no secrets logged. Key changes: - Add CloudResourceHandler with aws/gcp/azure type definitions - Add credential resolution from env vars and profile names - Add stubbed sandbox strategies (validate config, raise NotImplementedError) - Register cloud types in bootstrap_builtin_types - Credential masking via existing redaction patterns - Add Behave BDD tests, Robot integration tests, ASV benchmarks ISSUES CLOSED: #343	2026-03-17 13:14:37 -04:00
CoreRasurae	ff2d824f17	fix(cli): share PlanLifecycleService instance between CLI handler and PlanExecutor ... _get_plan_executor() created a second PlanLifecycleService Factory instance with its own in-memory _plans cache. After the executor's run_strategize() advanced the plan to execute/queued (via auto_progress), the CLI handler's separate service instance returned stale strategize/queued state from its cache, causing spurious "Plan is not in an executable state" errors. Fix: _get_plan_executor() now accepts an optional lifecycle_service parameter; the plan execute handler passes its own service instance so both share the same cache. Also addressed review feedback: - Improved type safety: lifecycle_service parameter typed as PlanLifecycleService | None instead of Any | None. - Added BDD regression test verifying the lifecycle service is shared between the CLI handler and the executor. - Updated reference documentation to reflect the type annotation change. ISSUES CLOSED: #1026	2026-03-17 12:22:32 +00:00
Luis Mendes	f0bdc3c651	fix(cli): load persisted actions in start_strategize and run execute phase inline ... start_strategize() built its action_registry from the in-memory _actions dict only, so fresh CLI processes (e.g. `plan execute` after a separate `plan use`) failed with PreflightRejection: "Action not found in registry". The PreflightRejection (extending bare Exception, not CleverAgentsError) escaped the CLI error handler, producing an opaque "Error [500] INTERNAL: An unexpected error occurred" message. Additionally, `plan execute` only transitioned the plan to execute/queued without running the execute phase, leaving the plan stuck and making `lifecycle-apply` fail with PlanNotReadyError. Changes: - start_strategize() loads the plan's action from the persistence layer (via get_action()) before building the preflight action_registry. - plan execute CLI catches PreflightRejection for user-friendly errors. - plan execute CLI runs the execute phase inline via PlanExecutor so the plan progresses through execute/queued -> execute/complete. - lifecycle-apply CLI handles plans already auto-progressed to apply/queued by complete_execute()'s auto_progress() call. ISSUES CLOSED: #746	2026-03-17 10:39:33 +00:00
Luis Mendes	cb583021df	test(e2e): E2E acceptance criteria for M6 (v3.5.0) — autonomy hardening ... Implemented Robot Framework E2E test suite for M6 autonomy hardening acceptance criteria. Tests exercise the real CleverAgents CLI with zero mocking, covering session lifecycle, automation profiles, project setup, plan lifecycle via A2A facade, guard enforcement, and a full autonomy acceptance flow. LLM-dependent tests use Skip If No LLM Keys for graceful degradation when API keys are unavailable. Hardened shared E2E keywords (common_e2e.resource): - Safe JSON parsing with rfind-bounded extraction and error wrapping (C1, M1) - Multi-object fallback: last-line reverse scan for multi-JSON output (M5) - Moved Safe Parse Json Field to common_e2e.resource for reuse (L1) - Migrated deprecated Run Keyword If to IF/ELSE blocks (M4) - Added cwd parameter to Run CleverAgents Command (M6) - API key protection via inline evaluation instead of RF variables (S1) - Git return-code assertions in Create Temp Git Repo (L1) - Removed unused Collections library import (L2) - Warning log on directory removal failure instead of silent ignore (L6) Hardened m6_acceptance.robot: - Force Tags E2E instead of per-test [Tags] (L3) - Per-test [Teardown] for resource cleanup, including Init test (M2, M9, L5) - Collision-safe uuid4 hex suffix instead of randint (L7) - Initialized session_id to EMPTY before test body for safe teardown (L2) - ELSE branches with WARN log on conditional assertions (M3, M4, M6) - Guard enforcement checks specific automation-profile fields (M5) - Strengthened ci assertions with case-sensitive matching and JSON field parsing for config get verification (M9) - Eliminated Python code injection via string interpolation (H2) - Added assertions to Full Flow Apply Step keyword (H4) - Removed redundant config set in Guard test (M10) - Accurate CHANGELOG entry describing actual test scope (C2) Post-review fixes applied: - Verify all 8 built-in profiles (manual, review, supervised, cautious, trusted, auto, ci, full-auto) instead of 4 (M1) - Session delete confirms removal via re-list (M2) - Full Flow Apply Step verifies plan phase transition (M3) - Plan execute output asserts plan_id presence + phase parsing (M4) - Safe Parse Json Field gains two-strategy approach: outer-bracket extraction then last-line reverse scan fallback (M5) - Removed redundant automation profile reset in Config test (L1) - JSON-quoted assertions ("ci", "auto") prevent false-positive substring matches on short profile names (L3) New E2E tests covering remaining acceptance criteria: - Guard enforcement with custom profile: registers a profile with explicit denylist, budget cap, and tool-call limits, then verifies all guard fields via automation-profile show (AC-4 / H1) - Profile precedence resolution: sets global profile to "review", creates plan with --automation-profile trusted, asserts plan output shows "trusted" not "review" (AC-5 / C2) - Event queue via plan lifecycle transitions: creates plan, captures initial state, executes, verifies state transition proving domain event bus delivered and processed events (AC-3 / C1) - Hierarchical decomposition via plan tree: creates plan with full-auto profile, executes, runs plan tree --format json, verifies decision nodes and children structure (AC-6 / C3) Robot Framework uses dots as hierarchy separators in suite names (e.g. "E2E.M6 Acceptance"). The E2E Suite Setup keyword replaced spaces with underscores but preserved dots, producing directory names like "E2E.M6_Acceptance". The CLI init command derives the project name from Path.cwd().name, and Project.validate_name() rejects dots ("Name must be alphanumeric with hyphens, underscores, or spaces"). Added a second Replace String call to convert dots to underscores so the sanitized suite name passes Project name validation. Also increased subprocess timeouts in m3_e2e_verification.robot (60s->120s) and m4_e2e_verification.robot (30s->120s) to prevent flaky CI failures from Python startup overhead. The plan lifecycle E2E tests call "plan use local/code-review" but actions are user-defined entities that must be explicitly registered via "action create --config <yaml>" before use. Without the action, plan use failed with "Action 'local/code-review' not found" and all LLM-dependent tests skipped. Added action registration to M6 Suite Setup that dynamically selects the actor matching the available API key (anthropic/claude-sonnet-4 when ANTHROPIC_API_KEY is set, openai/gpt-4o otherwise) and creates the action YAML inline. ISSUES CLOSED: #746	2026-03-17 10:13:23 +00:00
CoreRasurae	23fd06bbd0	fix: Protect sensitive values from being exposed ... There are events being sent that include sensitive data like passwords and need to be protected. Refs: 746	2026-03-17 10:06:54 +00:00
Luis Mendes	f07d3475f8	fix(cli): persist plan overrides and run strategize inline in plan execute ... The `plan use` CLI set the --automation-profile, actor, and execution-environment overrides on the in-memory Plan object after use_action() had already persisted it to the database. Subsequent CLI invocations (separate processes) loaded the plan from the database without the overrides, so `plan execute` could never see the automation profile. Additionally, `plan execute` required the plan to be in Strategize/complete state, but nothing ran the strategize phase between `plan use` and `plan execute`. The auto_strategize field existed on the AutomationProfile model but was never implemented — plans created with auto_strategize=0.0 (ci, full-auto, trusted, etc.) stayed in Strategize/queued indefinitely. Changes: - Call service._commit_plan(plan) after applying post-creation overrides in the `plan use` CLI command so that automation-profile, actor, and execution-environment changes survive across process boundaries. - In the `plan execute` CLI command, detect plans still in Strategize/queued and run the strategize phase inline via PlanExecutor.run_strategize() before transitioning to Execute. After inline strategize, if auto_progress already advanced the plan to Execute (e.g. when auto_execute=0.0), skip the explicit execute_plan() call and report the current state. - Widen the auto-select query to include both QUEUED and COMPLETE plans (QUEUED plans are now eligible because strategize will run inline). - Update the "no plans ready" test to use an empty plan list (the old test returned a QUEUED plan which is now eligible). - Add BDD scenarios covering inline strategize, auto-progress handling, and automation-profile persistence. Refs: #746	2026-03-17 10:06:42 +00:00
CoreRasurae	09d3b0aa20	style: fix style and type checks	2026-03-17 09:54:04 +00:00
Luis Mendes	22580752d2	fix(service): add database fallback to PlanLifecycleService.list_plans() ... list_plans() only read from the in-memory self._plans dict, but the DI container creates PlanLifecycleService via providers.Factory (a new instance per call), so every CLI invocation started with an empty dict. Plans created by "plan use" were persisted to the database via UnitOfWork.transaction() but "plan lifecycle-list" could never find them because it never queried the database. Added a database query path to list_plans() that calls LifecyclePlanRepository.list_all() when persistence is enabled, mirroring the existing database fallback in get_plan(). Also added the list_all() method to LifecyclePlanRepository. Refs: #746	2026-03-17 09:54:01 +00:00
Luis Mendes	ab911dbdc4	fix(cli): write machine-readable formats directly to stdout bypassing Rich line-wrapping ... The format_output() function returned a string that callers passed to Rich console.print(), which wraps long lines at terminal width. This injected literal newline characters into JSON string values (e.g. in definition_of_done fields), producing invalid JSON that downstream parsers could not decode (JSONDecodeError: Invalid control character). For machine-readable formats (json, yaml, plain), format_output() now writes the rendered output directly to sys.stdout and returns an empty string. This preserves the exact serialization from json.dumps/ yaml.dump without Rich text processing artifacts. Refs: #746	2026-03-17 09:53:57 +00:00
Luis Mendes	2acb957d83	fix(cli): replace in-memory automation-profile repository with database-backed persistence ... The automation-profile CLI commands used an _InMemoryProfileRepository (a Python dict) that lost all data between CLI process invocations. Profiles created with "automation-profile add" were invisible to subsequent "automation-profile show" or "list" calls because each CLI command is a separate process with a fresh empty dict. Changes: - Replaced _InMemoryProfileRepository with the real AutomationProfileRepository from the infrastructure layer, wired via the DI container following the same pattern as tool.py and session.py. - Added auto_commit support to AutomationProfileRepository (matching the existing SessionRepository pattern) so that CLI commands running outside a UnitOfWork commit each operation automatically. - Added safety_json and guards_json Text columns to the automation_profiles table (Alembic migration m6_005) for full-fidelity round-trip of the AutomationGuard and SafetyProfile sub-models. Previously, guards and several safety fields (max_cost_per_plan, max_retries_per_step, etc.) were silently dropped on persistence. - Updated _from_domain, _to_domain, and _update_row to serialize and deserialize the full guard and safety sub-models via JSON, with backward-compatible fallback to legacy scalar columns. Refs: #746	2026-03-17 09:53:53 +00:00
brent.edwards	c0658c2acf	Merge pull request 'fix(test): remove eager tdd_test_helpers import from mocks __init__' (#985) from fix/benchmark-tdd-import into master ... Reviewed-on: #985	2026-03-16 23:29:51 +00:00
brent.edwards	288246d9b5	fix(test): remove eager tdd_test_helpers import from mocks __init__ ... The re-export of make_mock_scenario from features/mocks/__init__.py caused ASV benchmark discovery to fail because tdd_test_helpers imports behave.model.Status, which is unavailable in the ASV benchmark virtualenv. All callers already import via the full module path (features.mocks.tdd_test_helpers), so the re-export was unnecessary. ISSUES CLOSED: #628	2026-03-16 23:22:05 +00:00
brent.edwards	c19c2b2e2c	Merge pull request 'feat(testing): implement @tdd_expected_fail tag handling in Robot Framework' (#673) from feature/m5-robot-tdd-tags into master ... Reviewed-on: #673 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-16 23:13:30 +00:00
brent.edwards	c13a62a2f2	Merge branch 'master' into feature/m5-robot-tdd-tags	2026-03-16 23:06:19 +00:00
brent.edwards	d0ca129d90	Merge branch 'master' into feature/m5-robot-tdd-tags	2026-03-16 23:05:43 +00:00
freemo	2688c85769	feat(extensibility): implement Custom Sandbox Strategy Registration via SandboxStrategy Protocol ... Implement SandboxStrategyProtocol, a 9-method @runtime_checkable Protocol enabling third-party sandbox strategy registration. Includes: - SandboxStrategyProtocol with create/read/write/diff/commit/rollback/checkpoint/ restore_checkpoint/cleanup methods - SandboxRef (frozen dataclass) and DiffView/DiffEntry (Pydantic models) - SandboxStrategyRegistry with config-driven registration, Protocol validation, thread safety, and clear/list/has operations - BuiltInSandboxStrategyAdapter wrapping existing Sandbox implementations to conform to the new Protocol - CustomStrategyConfig for YAML/dict-based strategy registration - SandboxFactory integration with custom_registry parameter, has_custom_strategy() and get_custom_strategy_class() - 25 Behave BDD scenarios (85 steps) covering protocol, registry, adapter, config, and factory integration - 8 Robot Framework integration tests with real filesystem operations - ASV benchmarks for registry and adapter operations - Developer documentation ISSUES CLOSED: #586	2026-03-16 22:50:27 +00:00
hurui200320	a5de448856	feat(testing): implement @tdd_expected_fail tag handling in Robot Framework ... Implements the three-tag TDD bug-capture system in Robot Framework via a Listener v3 module, paralleling the Behave implementation. Tests tagged tdd_expected_fail that fail have their result inverted to PASS (bug still exists); tests that unexpectedly pass are inverted to FAIL with guidance. Addresses all 15 findings from code review (PR !673, reviewer hamza.khyari): P2 fixes: - Added idempotency guard (_processed_tests set) to prevent double-inversion when the listener is loaded twice in the same process. - Rewrote normal-test-unaffected check to run alongside a tdd_expected_fail fixture in a single Robot invocation, proving the listener is loaded and selectively applies rather than being a tautological pass. P3 fixes: - Added output.xml existence guard with clear diagnostics in _run_fixture. - Documented intentional use of data.tags (static definition) vs result.tags (runtime-modifiable) in end_test docstring. - Added SKIP status test fixture and integration test case. - Added message content assertion in cmd_expected_fail_inverted. - Tightened substring assertions to match specific error text. - Added tdd_expected_fail-alone fixture (both companions missing). - Added close() hook to clear _validation_errors and _processed_tests. - Simplified _run_fixture return type to tuple[str, str]. - Changed listener path resolution from CWD-relative to __file__-relative in noxfile.py (integration_tests, slow_integration_tests, e2e_tests). P4 fixes: - Added __all__ declaration to helper module. - Changed module docstring from "mirroring" to "paralleling". - Added comment documenting accepted XML parsing risk (self-generated XML). Additional fixes: - Increased M4 E2E plan-tree test timeout from 30s to 120s (pre-existing timeout failure unrelated to this feature). Quality gates (post-rebase onto latest master): - nox -s lint: PASS - nox -s typecheck: PASS (0 errors) - nox -s unit_tests: PASS (10,700 scenarios) - nox -s integration_tests: PASS (1,505 tests) - nox -s coverage_report: PASS (97.9% >= 97% threshold) - nox -s benchmark: PASS - nox -s docs: PASS - nox -s build: PASS - nox -s security_scan: PASS - nox -s dead_code: PASS ISSUES CLOSED: #628	2026-03-16 22:45:55 +00:00
freemo	05503712ae	Docs: Daily update to timeline	2026-03-16 17:40:17 -04:00
hamza.khyari	028cf150b7	Merge pull request 'feat(acms): implement UKO Layer 3 Technology Vocabularies (uko-py, uko-ts, uko-rs, uko-java)' (#660) from feature/m6-uko-layer3-technology-vocabularies into master ... Reviewed-on: #660 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-16 12:18:06 +00:00
hamza.khyari	89eaee008d	feat(acms): implement UKO Layer 3 Technology Vocabularies (uko-py, uko-ts, uko-rs, uko-java) ... Implement Layer 3 technology-specific UKO vocabulary extensions for Python, TypeScript, Rust, and Java with language-specific classes, properties, and DetailLevelMap insertions. - 4 OWL/Turtle ontology files with language-specific semantic classes - DetailLevelMap insertion logic with correct integer reassignment - Provenance contract (5 required fields per spec) - Full 4-layer chain resolution (Layer 3 -> Layer 2 -> Layer 1 -> Layer 0) - Comprehensive Behave test suite (63 scenarios) ISSUES CLOSED: #576	2026-03-16 12:11:08 +00:00
brent.edwards	3b6b1d2414	Merge pull request 'test(plan): TDD failing tests for checkpoint real rollback (bug #822)' (#929) from tdd/m6-checkpoint-real-rollback into master ... Reviewed-on: #929 Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>	2026-03-16 03:31:43 +00:00
brent.edwards	3119383529	Merge branch 'master' into tdd/m6-checkpoint-real-rollback	2026-03-16 03:15:10 +00:00
brent.edwards	3eecb79003	test(plan): TDD failing tests for checkpoint real rollback (bug #822) ... TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839	2026-03-16 01:45:50 +00:00
brent.edwards	2d4b12df6a	Merge pull request 'test(plan): TDD failing tests for subplan spawn orchestration (bug #823)' (#930) from tdd/m6-subplan-spawn-orchestration into master ... Reviewed-on: #930	2026-03-16 01:34:55 +00:00
brent.edwards	b67dc63eda	test(plan): TDD failing tests for subplan spawn orchestration (bug #823) ... Write Behave scenario and Robot Framework test proving that SubplanService.spawn() only creates metadata (SubplanStatus records and SpawnMetadata) without creating real child Plan domain objects or triggering lifecycle progression. Tests are tagged @tdd_expected_fail so CI passes via result inversion. ISSUES CLOSED: #838	2026-03-16 01:12:24 +00:00
freemo	dfa05a6909	fix(cli): wire real LLM actors into plan executor for production execution ... The `plan execute` CLI command only performed phase transitions (Strategize → Execute) without ever invoking the `PlanExecutor` to drive the strategize or execute actors. `PlanExecutor.__init__` unconditionally created `StrategizeStubActor()` and `ExecuteStubActor()` which parse text locally and return empty changesets — no real LLM call was made. Added `_get_plan_executor()` helper that resolves `ProviderRegistry` from the DI container and constructs `LLMStrategizeActor` / `LLMExecuteActor` for real LLM invocations via LangChain. Updated `execute_plan` CLI command to detect plan phase/state and automatically run the appropriate actor: - Strategize/queued → run strategize actor → transition to Execute - Strategize/complete → phase transition only (backward compat) - Execute/queued → run execute actor → mark complete New `llm_actors.py` module provides `LLMStrategizeActor` (task decomposition into numbered steps) and `LLMExecuteActor` (code generation with FILE: blocks). Both resolve `provider/model` actor names (e.g. `openai/gpt-4`) to live LangChain LLM instances. `PlanExecutor.__init__` now accepts optional `strategize_actor` and `execute_actor` parameters, falling back to stubs when None. Existing mock-based BDD tests remain backward-compatible via duck- typing fallback (MagicMock.phase is not a PlanPhase, so the legacy `service.execute_plan()` path is taken). Includes Behave BDD scenarios testing custom actor injection into PlanExecutor. ISSUES CLOSED: #960	2026-03-15 19:59:38 -04:00
freemo	2d423bdfcd	fix(cli): add missing --format flag to action create command ... The `action create` CLI command was the only action subcommand missing the `--format`/`-f` parameter. All other action subcommands (`list`, `show`, `archive`) already accepted `--format` and routed through `_print_action()`. Running `action create --config action.yaml --format plain` failed with a Typer unrecognized-option error. Added the `fmt` parameter (with `--format`/`-f` aliases, defaulting to `rich`) to the `create()` function signature and passed it through to the existing `_print_action()` helper which already handles all output formats. Added Behave BDD scenarios for `--format plain` and `--format json` to `action_cli_spec_alignment.feature`. ISSUES CLOSED: #959	2026-03-15 19:33:11 -04:00
freemo	065171f21c	test(e2e): E2E acceptance criteria for M2 (v3.1.0) — actor compiler and LLM integration ... Add Robot Framework E2E test suite robot/e2e/m2_acceptance.robot exercising M2 acceptance criteria with zero mocking. Test creates a temp git repo with sample project files, registers a custom actor via CLI, sets up resource and project, creates an action referencing the actor, and runs the full plan lifecycle (use → execute strategize → execute → diff → apply). Validates actor YAML compilation, skill registry, tool lifecycle, and LLM integration through real CLI invocations with real provider API keys. Uses flexible structural assertions and expected_rc=None for LLM-dependent commands. ISSUES CLOSED: #742	2026-03-15 19:32:44 -04:00
freemo	5f07316641	fix: wire DI persistence and plan execute/apply for M1 lifecycle ... Fixed 5 bugs preventing the M1 E2E acceptance test from passing: 1. _get_lifecycle_service() in action.py and plan.py bypassed the DI container, creating PlanLifecycleService without UnitOfWork. All plan/action data was in-memory only and lost between subprocess calls. Now uses container.plan_lifecycle_service() for DB persistence. 2. `plan execute` CLI only called service.execute_plan() (a pure state transition) without running PlanExecutor phase processing. Rewrote to detect the plan's current phase/state and dispatch synchronously: Strategize/queued → run_strategize(), Strategize/complete → transition + run_execute(), Execute/queued → run_execute(). 3. `plan apply` CLI had no plan_id argument. Added optional positional plan_id with _lifecycle_apply_with_id() that drives the plan through Apply/queued → Apply/processing → Apply/applied. 4. Preflight guardrail in start_strategize() built action_registry from the in-memory _actions dict only. Added get_action(plan.action_name) call to load the action from DB into cache before the guardrail check. 5. Robot Framework Create File syntax used continuation lines producing 9 arguments instead of 1. Fixed to use Catenate SEPARATOR=\n then pass single variable to Create File. Also fixed --branch main to --branch master (git init default). update mocks for execute_plan CLI changes across unit and integration tests The new execute_plan() command calls _get_plan_executor() and service.get_plan(plan_id) for phase/state detection. Existing tests only mocked _get_lifecycle_service, so MagicMock defaults caused phase/state comparisons to fail. Changes across 14 files: - Patch _get_plan_executor in all test setups that invoke the CLI execute command (Behave step files + Robot helper scripts) - Set service.get_plan.return_value to real Plan objects with correct phase/state so the execute_plan dispatch logic works - Fix error-path tests to use STRATEGIZE/COMPLETE plans so the error side_effects are actually reached - Fix "Multiple plans eligible" → "Multiple plans ready" message text to match existing test expectations increase Robot Framework subprocess timeouts for CI resource contention Three integration tests were timing out in CI due to resource contention when pabot runs multiple test suites in parallel. All three pass locally and the timeouts were simply too tight for constrained CI environments. - tdd_session_create_di.robot: 30s → 90s (DI container init + DB setup) - database_integration.robot: 60s → 120s (Run Python Script keyword) - m3_e2e_verification.robot: 60s → 120s (correction-live-revert spawns 3 sequential CLI subprocesses with full container initialization) ISSUES CLOSED: #789	2026-03-15 20:50:02 +00:00
freemo	cb3b7aab44	test(e2e): E2E acceptance criteria for M1 (v3.0.0) — minimal plan execution flow ... Added Robot Framework E2E test suite for M1 milestone acceptance criteria. Tests the complete plan lifecycle (action create → resource add → project create → plan use → plan execute strategize → plan execute → plan diff → plan apply) with real LLM API keys and no mocking. Key implementation details: - Uses openai/gpt-4o-mini as strategy/execution actor (cost-effective) - Simple definition_of_done: "Create a file called HELLO.md" - Creates isolated temp git repo via Create Temp Git Repo keyword - Extracts plan ID via ULID regex from plain-text output - Uses expected_rc=None for LLM-dependent steps (execute, diff, apply) to handle non-deterministic LLM behavior gracefully - Flexible structural assertions: checks rc, output presence, git log - Skips gracefully when no LLM API keys (ANTHROPIC/OPENAI) are set - Tagged [E2E] so it runs only in nox -s e2e_tests session ISSUES CLOSED: #741	2026-03-15 20:50:02 +00:00
freemo	21a8e672a3	Docs: Contributing now enforces 97% coverage	2026-03-14 21:20:43 -04:00
freemo	ce722ed0ea	ops(ci): configure LLM API keys in Forgejo CI for integration test execution ... Updated CI pipeline to inject ANTHROPIC_API_KEY and OPENAI_API_KEY secrets as environment variables during Robot Framework integration test execution. Added CI setup documentation. ISSUES CLOSED: #701	2026-03-14 22:34:18 +00:00
freemo	af6340e732	Docs: Daily update to timeline	2026-03-14 18:26:20 -04:00
freemo	67291b4614	Docs: Updated chat room	2026-03-14 18:26:16 -04:00
freemo	09f1d621bf	Docs: Daily update to timeline	2026-03-14 18:26:12 -04:00
freemo	ec450e9085	fix(test): fix tolerant exit code and missing RC check in resource CLI test ... - Replace `rc == 0 or rc == 1` with strict `rc == 0` in resource type list test so failures are no longer silently accepted - Capture and assert the return code of the Suite Setup database schema creation to fail fast if the setup itself is broken apply strict RC checks to resource_cli.robot - Capture return value of Run Process in Suite Setup and assert rc==0 - Replace tolerant RC check (rc==0 or rc==1) with strict Should Be Equal As Integers check for resource type list test case broaden exception handling in resource CLI commands Add catch-all `except Exception` handler after each `except CleverAgentsError` block in all 14 resource CLI command handlers. This ensures unexpected exceptions (e.g. sqlalchemy.exc.OperationalError) are caught and displayed gracefully instead of producing raw tracebacks. re-raise typer.Abort/Exit in broad exception handlers The `except Exception` handlers added in the previous commit inadvertently caught typer.Abort and typer.Exit, which are subclasses of Exception (via click.exceptions). This turned successful CLI exits into aborts and double-handled already-caught errors, breaking integration tests that rely on normal typer exit behaviour. Add an isinstance guard to re-raise typer.Abort and typer.Exit before the catch-all handler runs. Fixes #896	2026-03-14 15:47:03 -04:00
freemo	447328a92d	fix(test): remove retry masking and output filtering in database integration test	2026-03-14 03:56:18 +00:00
freemo	9ef8502570	fix(database): reset session factory after engine disposal in init_database	2026-03-14 03:56:18 +00:00