730 Commits

Author	SHA1	Message	Date
hurui200320	554d6889cc	fix(cli): add --skill flag to actor run command (#971) ... ## Summary Add the missing `--skill` repeatable flag to `actor run` and `actor-run` CLI commands, aligning the implementation with the specification (CLI Synopsis line 277). The flag enables ad-hoc skill injection at runtime without modifying YAML configuration. Closes #887 ## Changes ### DI Container - **`container.py`**: Added `_build_skill_service()` factory and `skill_service` Singleton provider, following the established `_build_*` pattern. Falls back to in-memory `SkillService()` when the database is unavailable. Exception handling narrowed to `(ImportError, OperationalError, DatabaseError, OSError)` with `exc_info=True` for traceability. ### CLI Layer - **`actor.py`**: Added `--skill` Typer option (`list[str] | None`, repeatable, `metavar="NAME"`). Help text notes that skills only augment tool-bearing agents. Wrapped constructor in the existing `try/except` block so `CleverAgentsException` from skill resolution is properly caught. - **`actor_run.py`**: Same `--skill` option with `metavar="NAME"`. Exception handler catches `CleverAgentsException` (matching master — not broadened to `CleverAgentsError`). - **`skill.py`**: Removed module-level `_service` cache. `_get_skill_service()` now always delegates to `get_container().skill_service()` so that `reset_container()` correctly invalidates the cached instance. `_reset_skill_service()` now overrides the container's provider via `providers.Object()`. Removed dead `validate_skill_names()` function. ### Runtime Layer - **`application.py`** (438 lines, down from 625): `ReactiveCleverAgentsApp` gains `skill_names` parameter with automatic deduplication via `dict.fromkeys`. `_resolve_skills()` obtains `SkillService` from the DI container (no CLI layer import). Separate `except KeyError` and `except ValueError` produce distinct error messages (`"not found in registry"` vs `"resolution failed: {exc}"`). Skill tools are only injected into agents that already have tools (`if self._resolved_skill_tools and tools:`), preventing LLM agents from being converted to pass-through `SimpleToolAgent` instances. When skill tools are skipped for tool-less agents, `logger.debug` emits a diagnostic message. `_sanitize_skill_name()` validates skill name format with tightened regex: `^[\w.-]{1,127}/[\w.-]{1,127}$` with `re.ASCII` flag. Zero-tool skill warning now uses `logger.warning` (not `print(stderr)`), ensuring structured log output and proper log-level filtering. - **`graph_executor.py`** (334 lines): Extracted graph execution logic. Type annotations improved. ### Tests - 24+ Behave scenarios across feature files covering: single/multiple/unknown skill flags, skill+context combined, duplicate deduplication, skill resolution, ValueError path, zero-tool resolution, error handling, tool merging, default behavior, overrides, LLM agent guard, `_sanitize_skill_name` edge cases (empty string, too-long name, ANSI escape codes, disallowed characters), `_build_skill_service` happy+fallback paths, `_get_skill_service` container delegation. - CLI "unknown skill" tests for **both** `actor.py` and `actor_run.py` exercise the real error chain (mock only `get_container()`, not the entire `ReactiveCleverAgentsApp`), testing `_resolve_skills()` → `CleverAgentsException` → `except CleverAgentsException` → exit code 2 end-to-end. - Combined skill+context tests assert `ContextManager` was instantiated and `exists()` was called in dedicated **Then** steps. - `@coverage` tags added to all new scenarios. - **Robot Framework smoke tests** added (`robot/skill_actor_run.robot` + `robot/helper_skill_actor_run.py`): unknown-skill error path and valid-skill acceptance path. ### Changelog - Added entry under `## Unreleased` in `CHANGELOG.md`. ## Review Fixes Applied (Brent Edwards, Rounds 1 & 2) | # | Finding | Resolution | |---|---------|------------| | **P1-1** | `print(stderr)` for zero-tool skill warning | **Fixed** — replaced with `logger.warning("Skill '%s' resolved to zero tools", name)`, removed unused `import sys` | | **P2-2** | Skill tools silently skipped for tool-less agents | **Fixed** — added `logger.debug` when skipped; updated `--skill` help text to note "only augments tool-bearing agents" | | **P2-3** | `container.py` at 739 lines | **Acknowledged** — pre-existing growth (+59 lines for `_build_skill_service`); extracting factories is a separate refactoring task | | **P2-4↑** | `CleverAgentsException` → `CleverAgentsError` broadens catch scope | **Fixed** — reverted `actor_run.py` to `except CleverAgentsException` matching master | | **P3-5** | No Robot Framework smoke test for `--skill` | **Fixed** — added `skill_actor_run.robot` with 2 test cases (unknown-skill error, valid-skill acceptance) | | **P3-6** | `GraphExecutor._follow_chained_edges` static-calling-static | **Acknowledged** — cosmetic pattern that doesn't affect correctness; can address in a follow-up | ## Known Limitations / Deferred Items | Item | Reason | |------|--------| | `actor.py` at 679 lines (500-line guideline) | Pre-existing (670 on master), +9 lines for `--skill`. Refactoring the shared `_execute()` closure is a separate task. | | `container.py` at 739 lines (500-line guideline) | Was 680 lines on master, +59 lines for `_build_skill_service()` and `skill_service` provider. Refactoring into sub-modules is a separate task. | | Code duplication between `actor.py` and `actor_run.py` `run()` | ~47 lines identical code. Coupled with the line-count issue above — both require extracting shared execution logic into a helper module. | | `SimpleToolAgent` only executes `tools[0]` | Deferred to #974. Pre-existing architectural limitation, not introduced by this PR. | | `GraphExecutor._follow_chained_edges` static-calling-static pattern | Cosmetic, doesn't affect behavior. | ## Quality Gates - `nox -s lint`: ✅ PASS - `nox -s typecheck`: ✅ PASS (0 errors) - `nox -s unit_tests`: ✅ PASS (11,130 scenarios, 0 failures) - `nox -s integration_tests`: ✅ PASS (1,559 tests, 0 failures) - `nox -s coverage_report`: ✅ 97% (meets threshold) - Branch rebased onto latest `master` (`ab1fd19b`) Reviewed-on: #971 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 09:30:35 +00:00
hurui200320	2434253c1a	feat(cli): implement full output rendering framework (#812) ... ## Summary Implement the 6 missing element handle types (Tree, Text, Code, Diff, Separator, ActionHint) and ensure all 6 materialization strategies (rich, color, table, plain, json, yaml) support all 10 element types. This completes the output rendering framework per the specification (§25417-27276). Closes #550 ## Changes ### Source (restructured into smaller modules) #### `handles/` package (was `handles.py` — 1062 lines → 5 files, all ≤500 lines) - **`_models.py`**: All Pydantic data models, constants (MAX_TREE_DEPTH, MAX_ELEMENTS_PER_SESSION, MAX_TABLE_ROWS), exceptions (ElementClosedError), event classes, ElementSnapshot union. P2-1: `DiffLine.type` renamed to `line_type` with backward-compatible alias. - **`_base.py`**: Generic `ElementHandle[E]` base class with thread-safe `element_copy()` (lock-protected). - **`_panel_table.py`**: PanelHandle, TableHandle, StatusHandle. - **`_concrete.py`**: ProgressHandle (P1-1: validates `total` non-negative), TreeHandle, TextHandle (P3-2: close() delegates to super()), CodeHandle, DiffHandle, SeparatorHandle, ActionHintHandle. P3-8: `increment(delta=0)` now rejected. - **`__init__.py`**: Re-exports all public symbols. #### `_renderers.py` — plain renderers, sanitization, and shared helpers - Plain render functions for all 10 element types. - Terminal escape sanitization (`strip_terminal_escapes`). - P2-3: `_sort_table_rows` now consults `ColumnDef.col_type` for numeric sorting. - P2-6: `compute_column_widths()` shared helper extracted (DRY fix). #### `_color_renderers.py` — ANSI colour renderers - Colour-coded render functions for all 10 element types. - P2-2: TextBlock now gets color treatment (`_render_text_color`) instead of falling through to plain. #### `_boxdraw.py` — box-drawing renderers - P2-5: Upgraded from ASCII `+-|` to Unicode box-drawing `╭─╮│╰╯` with rounded corners per spec §26821. #### `_ids.py` — ID generation helpers - P3-1: Session and handle IDs now use separate counters, making IDs monotonic within their namespace. #### `materializers.py` — strategy protocol and 6 concrete strategies - P1-2: `_snapshot_to_dict` now includes `timing` field in JSON/YAML output per spec §27022. - P2-4: `_column_def_to_dict` always includes all fields unconditionally for stable JSON schemas. #### `selection.py` — materializer selection with fallback - P1-4: `NO_COLOR` environment variable now respected (https://no-color.org/). When set, all visual formats fall back to plain. Precedence: explicit flag > NO_COLOR > terminal capability fallback. #### `session.py` - P1-1: `session.progress()` factory validates `total >= 0`. - P1-2: `snapshot()` includes `timing` when available. #### `__init__.py` — package docstring - P1-3: SD-29 corrected to reflect actual Table → Color → Plain fallback chain. - SD-14 marked as implemented (NO_COLOR support added). ### Spec Deviations (Documented) 28 deliberate deviations documented in `__init__.py` module docstring (SD-1 through SD-29, with SD-14 now implemented). SD-29 corrected. ### Tests (updated) - **+23 new BDD scenarios** covering: P1-1 total validation, P1-4 NO_COLOR, P2-1 DiffLine.line_type alias, P2-2 text color, P2-3 numeric sorting, P2-4 ColumnDef serialization, P2-5 Unicode box-drawing, P3-3 add_rows limit, P3-5 10-thread stress test, P3-6 summary truncation, P3-7 explicit format for color/table, P3-8 zero delta. - **Robot tests**: Updated box-drawing assertion for Unicode chars. ## Verification | Check | Result | |-------|--------| | Pyright | 0 errors, 1 pre-existing warning | | Ruff lint | All passed | | Unit tests | 393 features, 11,344 scenarios, 0 failures | | Integration tests | All passed | | E2E tests | All passed | | Coverage | 97% overall (threshold: 97%) | ## Review Fixes Applied (Luis Review #2412) | ID | Severity | Fix | |----|----------|-----| | P1-1 | High | `set_progress()` and `session.progress()` validate `total >= 0` | | P1-2 | High | `_snapshot_to_dict` includes `timing` field | | P1-3 | High | SD-29 documentation corrected | | P1-4 | High | `NO_COLOR` env var respected | | P2-1 | Medium | `DiffLine.type` → `line_type` with alias | | P2-2 | Medium | TextBlock gets color treatment | | P2-3 | Medium | Numeric column sorting | | P2-4 | Medium | ColumnDef always serializes all fields | | P2-5 | Medium | Unicode box-drawing characters | | P2-6 | Medium | Shared `compute_column_widths()` helper | | P3-1 | Low | Separate ID counters | | P3-2 | Low | TextHandle.close() delegates to super() | | P3-3 | Low | add_rows batch limit test | | P3-5 | Low | 10-thread stress test | | P3-6 | Low | Summary truncation test | | P3-7 | Low | Explicit format tests for color/table | | P3-8 | Low | Zero delta rejected | ### Deferred Items | ID | Reason | |----|--------| | P3-9 | snapshot() lock scope — acceptable correctness trade-off | | P3-10 | CLEVERAGENTS_FORMAT env var — documented as SD-15, requires CLI framework changes | Reviewed-on: #812 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 08:46:46 +00:00
hurui200320	3837327564	feat(plan): enforce decision type phase-gating at recording time (#973) ... ## Summary Adds phase-gating validation to `DecisionService.record_decision()` that enforces the specification's constraint: certain decision types are only valid during specific plan phases. This prevents invalid decisions (e.g., `tool_invocation` during Strategize, `strategy_choice` during Execute) from being persisted. ### Changes - **Exception** (`cleveragents.core.exceptions`): Added `DecisionPhaseViolationError(BusinessRuleViolation)` with `decision_type`, `plan_phase`, and `allowed_types` attributes. - **Phase constants** (`cleveragents.domain.models.core.decision`): - `resource_selection` added to `EXECUTE_TYPES` — now phase-agnostic (Strategize or Execute) per ADR-007 L72 and ADR-033 L74. - `subplan_spawn` / `subplan_parallel_spawn` in both sets; code comment documents divergence from ADRs per M4 subplan model (ticket #931). - `USER_INTERVENTION` remains phase-agnostic (both sets). - Module-level docstring table updated to match actual assignments. - `is_any_phase_type` property updated to check membership in both sets dynamically (was hardcoded to `USER_INTERVENTION` only). - **Phase-gating module** (`cleveragents.application.services.phase_gating`): - Extracted from `DecisionService` to reduce `decision_service.py` line count (1010 → 913) and isolate the phase-gating concern. - `PHASE_ALLOWED_TYPES` typed as `Mapping[PlanPhase, frozenset[DecisionType]]`. - `resolve_plan_phase()` helper: supports explicit parameter, DB lookup, and graceful skip. - `validate_phase_gating()` enforcement raises `DecisionPhaseViolationError`. - Exception narrowing: DB lookup catches `(DatabaseError, OperationalError, OSError)` instead of bare `except Exception` — only absorbs infrastructure failures, not programming errors. - `# TODO(pg-migration):` marker on TOCTOU race documentation for future PostgreSQL migration. - **Decision service** (`cleveragents.application.services.decision_service`): - Added `plan_phase` parameter to `record_decision()`. - Invalid `plan_phase` string now raises `ValidationError` (was uncaught `ValueError`). - Imports and delegates to `phase_gating` module for all phase-gating logic. - `PHASE_ALLOWED_TYPES` re-exported in `__all__` for backward compatibility. - **CHANGELOG**: Added behavioral change entry for `resource_selection` reclassification. - **Backward compatibility**: Phase-gating is opt-in — when neither `plan_phase` is provided nor a UnitOfWork is wired, validation is skipped, preserving all existing callers. - **Unrelated drive-by reverted**: Removed `ULID_PATTERN` from `decision.py` `__all__` (was an unrelated export addition). - **Tests**: - 36 Behave scenarios covering valid/invalid types per phase, phase-agnostic acceptance, DB-based resolution (Strategize and Execute plans), unknown plan in DB, PlanPhase enum pass-through, error attributes, and ungated phases. - 11 new Behave scenarios for `is_any_phase_type`: 4 dual-phase types (true) + 7 single-phase types (false), including `prompt_definition` root test. - 6 Robot Framework integration tests with stderr assertions. - Updated `consolidated_decision.feature` for new `EXECUTE_TYPES` member count (8 members). - Test cleanup now calls `uow.engine.dispose()` before file deletion. - `tempfile.mktemp()` replaced with `tempfile.mkstemp()`. - Inline imports moved to module top-level per CONTRIBUTING.md. - Flaky concurrency test timing increased in `subplan_execution_steps.py`. ### Review Round 1 + 2 Fixes | # | Finding | Resolution | |---|---------|------------| | P1-1 | `except Exception` too broad in `_resolve_plan_phase` | Narrowed to `(DatabaseError, OperationalError, OSError)` — matches codebase pattern | | P2-2 | `decision_service.py` at 1010 lines | Extracted to `phase_gating.py` module (1010 → 913 lines) | | P2-3 | TOCTOU race — no programmatic guard | Added `# TODO(pg-migration):` marker with actionable guidance | | P2-4 | `resource_selection` reclassification needs CHANGELOG | Added CHANGELOG entry documenting behavioral change | | P2-7 | `is_any_phase_type` BDD gap for dual-phase types | Added 11 parametrized scenarios covering all 4 dual-phase + 7 single-phase types | | P3-5 | `ULID_PATTERN` export is unrelated drive-by | Reverted — removed from `decision.py` `__all__` | | P3-6 | `decision.py` at 514 lines (now 513) | No action — reviewer accepted as marginally over | ### Quality Gates | Session | Result | |---------|--------| | lint | PASS | | typecheck | PASS (0 errors) | | unit_tests | PASS (11,153 scenarios, 0 failures) | | integration_tests | PASS (1,563 tests, 0 failures) | | e2e_tests | PASS (16 tests, 0 failures) | | coverage_report | 97% (threshold: 97%) | Closes #931 Reviewed-on: #973 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 07:53:43 +00:00
hurui200320	0b415a6e5e	test: add TDD bug-capture test for #969 — plan correct plan_id handling (#1051) ... ## Summary Add TDD bug-capture tests for bug #969 (`plan correct` expects `decision_id` but M3 acceptance test passes `plan_id`). These tests prove the bug exists and will serve as regression guards once the fix in #969 is merged. ### Changes - **Behave test** (`features/tdd_plan_correct_plan_id.feature`): Two scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_969` — one for `--mode revert` and one for `--mode append` — that invoke `plan correct <plan_id>` (without `--plan` flag) and assert the command resolves the plan_id to its root decision as `target_decision_id`. Both modes are tested because the bug affects `target_decision_id` resolution **before** mode-specific branching. - **Step definitions** (`features/steps/tdd_plan_correct_plan_id_steps.py`): Mock setup for DI container (DecisionService), CorrectionService, and `_resolve_active_plan_id`. Uses `tpcpid` step prefix per project conventions. - **Shared fixtures** (`features/mocks/tdd_plan_correct_plan_id_fixtures.py`): Centralised constants, patch targets, mock builders (`make_decision_ns`, `make_mock_container`, `make_correction_svc`, `make_default_decisions`, `make_default_container`), and `build_cli_args` helper. Both the Behave steps and Robot helper import from this shared module, eliminating code duplication and drift risk. - **Robot test** (`robot/tdd_plan_correct_plan_id.robot`): Two integration-level tests (revert + append) with `tdd_expected_fail tdd_bug tdd_bug_969` tags, exercising the same code paths via the helper script. - **Robot helper** (`robot/helper_tdd_plan_correct_plan_id.py`): Standalone helper that exits 0 with sentinel when the bug is fixed, exits 1 when the bug is present. Imports shared fixtures from `features/mocks/`. - **Changelog** (`CHANGELOG.md`): Added entry under Unreleased for #979. ### Bug Description The `correct_decision` function in `cleveragents.cli.commands.plan` declares `decision_id` as its first positional argument. When the M3 acceptance test calls `plan correct <plan_id> --mode revert --guidance "..."`, the plan_id is captured as `decision_id` and used directly as `target_decision_id` in `svc.request_correction()`. Since the plan_id is not a valid decision_id, the correction service cannot find the targeted decision. The same bug path is exercised by `--mode append`. ### How TDD Expected-Fail Works - The `@tdd_expected_fail` tag causes the test framework to invert the result: the test passes CI when the underlying assertion fails (proving the bug exists) and fails CI if the assertion passes (bug was fixed without removing the tag). - When bug #969 is fixed, the developer removes the `@tdd_expected_fail` tag, and the test runs normally as a regression guard. ### Quality Gates All nox sessions pass: - `nox -s lint` ✅ - `nox -s typecheck` ✅ (0 errors) - `nox -s unit_tests` ✅ (387 features, 11121 scenarios, 0 failures) - `nox -s integration_tests` ✅ (1561 tests, 0 failures) - `nox -s e2e_tests` ✅ (16 tests, 0 failures) - `nox -s coverage_report` ✅ (≥97%) Closes #979 Reviewed-on: #1051 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 07:31:12 +00:00
hurui200320	c741329f70	test: add TDD bug-capture test for #968 — plan explain plan_id handling (#1052) ... ## Summary Add TDD bug-capture tests for bug #968: the `plan explain` CLI command fails with rc=1 when given a plan_id because `explain_decision_cmd` treats its argument as a decision_id and `svc.get_decision(plan_id)` raises `DecisionNotFoundError`. These tests capture the exact failure condition described in #968 and are tagged with `@tdd_expected_fail`, `@tdd_bug`, and `@tdd_bug_968`. The `@tdd_expected_fail` tag inverts the test result so that CI passes while the bug is unfixed. Once bug #968 is fixed, the `@tdd_expected_fail` tag will be removed and the tests will serve as permanent regression tests. Closes #978 ## Changes ### Behave Tests (`features/tdd_plan_explain_plan_id.feature`) - **Scenario 1: Plan explain succeeds when given a plan_id with decisions** — Mocks `DecisionService` so `get_decision(plan_id)` raises `DecisionNotFoundError` (the bug) and `list_decisions(plan_id)` returns decisions. Invokes `plan explain <plan_id>` via CliRunner and asserts rc=0 with decision details visible. Verifies `list_decisions` was called with the correct `plan_id`. - **Scenario 2: Plan explain with plan_id shows root decision question** — Same setup, asserts the root decision question appears in the output. Both scenarios currently fail (rc=1, proving the bug exists), but the `@tdd_expected_fail` tag makes CI pass. ### Robot Tests (`robot/tdd_plan_explain_plan_id.robot`) - **TDD Plan Explain Succeeds With Plan ID** — Integration test using a helper script that records decisions via `DecisionService`, invokes `plan explain <plan_id>` via subprocess, and asserts rc=0. - **TDD Plan Explain With Plan ID Shows Root Question** — Same setup, asserts the root decision question appears in the output. Both tests carry `tdd_expected_fail`, `tdd_bug`, and `tdd_bug_968` tags. ### Step Definitions (`features/steps/tdd_plan_explain_plan_id_steps.py`) Step implementations for the Behave feature, following the established patterns from `plan_explain_cli_coverage_steps.py`. Includes `list_decisions.assert_called_once_with(plan_id)` verification to ensure the fix exercises the fallback lookup path. ### Helper Script (`robot/helper_tdd_plan_explain_plan_id.py`) Python helper for Robot tests, following the established pattern from `helper_tdd_checkpoint_real_rollback.py`. Uses a shared `_run_plan_explain()` helper to avoid subprocess invocation duplication, sets `NO_COLOR=1` in subprocess environment to prevent ANSI escape codes. ### CHANGELOG Added entry under `## Unreleased` describing the TDD bug-capture tests for bug #968. ## Review Fixes Applied (Iteration 1) 1. **Imports moved to module level** (Major) — Three imports in `_setup_plan_with_decisions()` moved to module level with `# noqa: E402`, consistent with all other `robot/helper_tdd_*.py` files. 2. **Mock accurately models real behavior** (Minor) — `svc.get_decision.return_value = None` changed to `svc.get_decision.side_effect = DecisionNotFoundError(...)` to match the real `DecisionService.get_decision()` which raises `DecisionNotFoundError`, not returns `None`. 3. **Strengthened assertion** (Minor) — `assert "decision_id" in output or "question" in output` changed to AND for a more precise assertion that verifies both fields are present. 4. **Removed `# type: ignore[arg-type]`** (Minor) — Changed `kwargs: dict[str, object]` to `kwargs: dict` and removed the `# type: ignore[arg-type]` directive, per CONTRIBUTING.md type safety rules. 5. **Handled `subprocess.TimeoutExpired`** (Minor) — Both `subprocess.run()` calls wrapped in `try/except subprocess.TimeoutExpired` with descriptive `_fail()` messages. 6. **Resolved timeout race condition** (Minor) — Inner `subprocess.run` timeout reduced from 60s to 45s so the helper catches timeout before Robot's 60s outer timeout kills it. 7. **Defensive setup check** (Nit, addressed) — Added `list_decisions()` verification after `record_decision()` to distinguish setup failures from the actual bug. ## Review Fixes Applied (Iteration 2) 1. **Added CHANGELOG.md entry** (Major) — Added entry under `## Unreleased` describing the TDD bug-capture tests for bug #968, per CONTRIBUTING.md §"Pull Request Process". 2. **Fixed Gherkin step text and feature comments** (Minor) — Renamed step from "returns None" to "raises DecisionNotFoundError" in feature file and step decorator. Updated feature comment from "returns None" to "raises DecisionNotFoundError". Step text now accurately describes the mock behavior. 3. **Fixed Robot helper output assertion logic** (Minor) — Changed `if "decision" not in combined.lower() and "question" not in combined.lower()` to `or` so both keywords must be present, matching the Behave test's AND-based validation. 4. **Added `NO_COLOR=1` to subprocess environment** (Minor) — Added `_make_subprocess_env()` helper that copies `os.environ` and sets `NO_COLOR=1`, following the established pattern from `robot/helper_e2e_common.py`. 5. **Extracted shared subprocess helper** (Nit, addressed) — Deduplicated subprocess invocation blocks into `_run_plan_explain(plan_id)` function, reducing code duplication across the two test subcommands. 6. **Added `list_decisions` mock verification** (Nit, addressed) — Added `svc.list_decisions.assert_called_once_with(plan_id)` in the Behave step that checks decision details, making the test contract clearer. 7. **Rebased on current master** (Nit, addressed) — Branch rebased on `origin/master` per CONTRIBUTING.md §"Branch Hygiene". ## Quality Gates - ✅ `nox -e lint` — All checks passed - ✅ `nox -e typecheck` — 0 errors - ✅ `nox -e unit_tests` — 391 features, 11175 scenarios, 0 failed - ✅ `nox -e integration_tests` — All tests passed (including Tdd Plan Explain Plan Id) - ✅ `nox -e e2e_tests` — 16 tests, 0 failed - ✅ `nox -e coverage_report` — 97% coverage (≥97% threshold met) Reviewed-on: #1052 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 07:21:11 +00:00
hurui200320	cbf8bcc993	test(e2e): E2E acceptance criteria for M5 (v3.4.0) — ACMS v1 and context scaling (#811) ... ## Summary Add `robot/e2e/m5_acceptance.robot` with **21 zero-mock E2E test cases** (in addition to existing M5 test suite) covering all M5 (v3.4.0) acceptance criteria: 1. **Context Assembly** — add/list/show/clear files in the context pipeline 2. **Context Scaling** — 10,000+ file project setup with simulate plumbing *(structural)* 3. **Context Policy Configuration** — per-view include/exclude paths, file-size limits 4. **Budget Enforcement** — max_file_size / max_total_size constraint storage *(structural)* 5. **Context Analysis** — ACMS pipeline inspect (tier schema) and simulate (JSON schema) *(structural)* 6. **Plan Execution** — real LLM calls via `openai/gpt-4o-mini` (`plan use` + `plan resume`) ### Structural vs. Behavioural Scope Tests in sections 1b–4 that use `project context simulate` or `inspect` are **structural / plumbing validations** — they verify CLI execution, JSON serialization, and stored configuration but do **not** exercise actual ACMS indexing or budget enforcement because the `ContextTierService` is an in-memory singleton that starts empty per CLI process. Each affected test has a `[Documentation]` note explaining this limitation. Behavioural ACMS validation is deferred until the full indexing pipeline is wired. ### Production Bug Fixes | Fix | File | Description | |-----|------|-------------| | `session.flush()` → `session.commit()` | `project_context.py` | Policy changes silently lost on `session.close()` | | `contextlib.suppress` rollback wrapper | `project_context.py` | Prevents rollback failure from masking original commit exception | | Add `session_factory` DI provider | `container.py` | `project context` commands hit `AttributeError` | | `providers.Factory` → `providers.Singleton` | `container.py` | Avoid creating duplicate engines per call | | Add Gemini API key pattern | `redaction.py` | `AIzaSy...` keys now redacted in logs | ### Review Feedback Addressed (Tenth Pass — @CoreRasurae Review #2410) | # | Severity | Finding | Fix | |---|----------|---------|-----| | P3-1 | Medium | "Clear Context" test tautological — never asserts files were present before clearing | Added `Should Contain ${list_before.stdout} config.py` precondition check after `context-load` and before `clear` | | P3-2 | Medium | Policy/budget verification uses substring matching (`Should Contain 262144`) | Replaced with `Extract JSON From Stdout` + `$rv.get('max_file_size') == 262144` parsed JSON assertions using `resolved_view` dict access | | P3-3 | Medium | Plan resume doesn't verify `phase` value, only existence | Added `Should Not Be Equal As Strings ${phase} queued` assertion to verify plan transitioned from queued | | P3-4 | Medium | Plan JSON extraction inconsistency (`rindex` vs `Extract JSON From Stdout`) | Replaced fragile `rindex`-based extraction with `Extract JSON From Stdout` keyword for consistency | | P3-6 | Medium | Context show summary weak content assertions | Added `Should Not Contain` guards against traceback/error output to reject false positives | | P3-8 | Medium | `_SafeSession` singleton may accumulate dirty state after rollback | Changed `_SafeSession.close()` from pure no-op to `real.rollback()` to reset session state between calls | | P3-14 | Medium | No test for `_save_policy_json` rollback path | Added BDD scenario "Save policy rollback re-raises after commit failure" with monkey-patched commit | | P3-15 | Medium | No test for `_save_policy_json` on nonexistent project | Added BDD scenario "Save policy on nonexistent project row updates zero rows" verifying silent 0-row behavior | | P4-1 | Low | Plan resume TRY/EXCEPT swallows assertion details | Moved field assertions outside TRY block; TRY only guards JSON extraction | | P4-2 | Low | `Safe Parse Json Field` logs stale error context | Fixed to track and report both Strategy 1 and Strategy 2 error contexts separately | | P4-4 | Low | SQLite WAL/SHM files not cleaned in regression test | Added cleanup loop for `-wal` and `-shm` suffixes alongside `.db` file | ### Deferred Items (Out of Scope) | ID | Severity | Reason | |----|----------|--------| | P2-1 | High | `execution_environment` silently dropped on subsequent `context set` — pre-existing production code bug in `_write_policy()`, not introduced by this PR | | P2-2 | High | Unhandled `ValidationError` on corrupt policy blob — pre-existing `_read_policy()` code, not changed by this PR | | P2-3 | High | Silent no-op UPDATE when `ns_projects` row missing — pre-existing `_save_policy_json` logic; this PR only changed error handling | | P3-5 | Medium | Structural tests cannot detect regressions — already honestly documented in every affected test's `[Documentation]` block | | P3-7 | Medium | View inheritance/override behavior not tested — nice-to-have, not in ticket acceptance criteria | | P3-9 | Medium | `context_set` double-writes when `execution_environment` set — pre-existing production logic | | P3-10 | Medium | `budget_tokens=0` silently replaced by default (falsy `or`) — pre-existing production code bug | | P3-11 | Medium | `context set` replaces entire view instead of merging — pre-existing design choice | | P3-12 | Medium | GEMINI_API_KEY propagated but potentially unused — security-first: propagating for redaction testing | | P3-13 | Medium | `reset_container()` doesn't dispose Singleton resources — pre-existing container lifecycle issue | | M5 | Medium | `_build_session_factory` engine never disposed — production code architecture, out of scope for testing ticket | | M6 | Medium | Missing `check_same_thread`/`isolation_level` — production code architecture, out of scope for testing ticket | | L1 | Low | `plan resume` not in spec CLI synopsis — informational | | L2 | Low | Context summary assertions depend on exact CLI wording — acceptable stability risk | | L3 | Low | Gemini regex minimum length slightly loose — acceptable security-first trade-off | | L4 | Low | Missing Google OAuth2 credential patterns — out of scope for this PR | | P4-3 | Low | `Run CLI` keyword duplicated — different purpose (uses `${WS}` as default cwd), not a true duplicate | | P4-5–P4-9 | Low | Various additional E2E coverage gaps — nice-to-have, not in ticket acceptance criteria | ### Quality Gates | Gate | Result | |------|--------| | lint | PASS | | typecheck | PASS (0 errors) | | unit_tests | **393/393** features, 11,210 scenarios | | integration_tests | **1,576/1,576** | | e2e_tests | **37/37** (21 M5 + 12 M6 + 2 smoke + 2 M1) | | coverage_report | **97%** (threshold: 97%) | ### Files Changed | File | Change | |------|--------| | `robot/e2e/m5_acceptance.robot` | **NEW** — 21 E2E test cases with honest structural documentation, parsed JSON assertions, prerequisite skip guards on all sections, safe assertion messages | | `robot/e2e/common_e2e.resource` | `on_timeout=kill` + return code checks + safe key evaluation via `os.environ.get` + fixed stale error logging in `Safe Parse Json Field` | | `robot/e2e/m1_acceptance.robot` | `on_timeout=kill` on git log | | `robot/e2e/m2_acceptance.robot` | `on_timeout=kill` + return code checks + safe assertion messages (no stderr embedding) | | `src/cleveragents/application/container.py` | Add `_build_session_factory` + `session_factory` Singleton | | `src/cleveragents/cli/commands/project_context.py` | `flush()` → `commit()` + `contextlib.suppress` rollback | | `src/cleveragents/shared/redaction.py` | Add Gemini API key pattern | | `noxfile.py` | Propagate `GEMINI_API_KEY` in e2e_tests | | `CHANGELOG.md` | 4 entries for #745 | | `features/application_container_coverage_boost.feature` | Updated title + 3 scenarios | | `features/steps/application_container_coverage_boost_steps.py` | Step defs for `_build_session_factory` | | `features/consolidated_security.feature` | 2 Gemini API key redaction scenarios | | `features/project_context_cli_coverage_boost.feature` | `flush()→commit()` regression test + rollback path + nonexistent project tests | | `features/steps/project_context_cli_coverage_boost_steps.py` | Separate engines for regression test + `try/finally` cleanup + `_SafeSession.close()` state reset + rollback/nonexistent test steps + WAL/SHM cleanup | Closes #745 ISSUES CLOSED: #745 Reviewed-on: #811 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 06:53:06 +00:00
hurui200320	774dfedc6b	test: add TDD bug-capture test for #967 — plan execute phase processing (#1050) ... ## Summary This PR adds TDD bug-capture tests for bug #967 — `plan execute` only transitions state without running strategize or execute phase processing. ### Motivation Bug #967 describes that the `plan execute` CLI command originally only called `service.execute_plan(plan_id)`, which is a state transition only (Strategize/COMPLETE → Execute/QUEUED). When a plan was in Strategize/QUEUED state (immediately after `plan use`), the command failed because `execute_plan()` requires Strategize/COMPLETE. The CLI should detect the plan's current phase and run `PlanExecutor.run_strategize()` before transitioning. Per the project's TDD Bug Fix Workflow (`CONTRIBUTING.md`), the first step in fixing any bug is to write a test that captures the buggy behavior. Since the fix for #967 is already present in the codebase (the CLI handler already orchestrates properly), the `@tdd_expected_fail` tags have been removed and these tests serve as **permanent regression guards** ensuring the fix is never reverted. ### Design Approach **All `@tdd_expected_fail` scenarios were rewritten to exercise the CLI orchestration layer** — the actual code path affected by bug #967. This satisfies AC4: "The test is specific enough that it will pass normally (without the tag) only when the bug is genuinely fixed." - **Scenarios 1, 2, 4** (previously `@tdd_expected_fail`): Use Typer's `CliRunner` with mocked services to invoke the `plan execute` CLI command handler directly. This tests the orchestration logic in `plan.py` — the exact code that was buggy. - **Scenario 3** (positive control): Uses real `PlanLifecycleService` (in-memory) and `PlanExecutor` (stub actors) to demonstrate that proper service-level orchestration works. - **Robot tests**: Replicate the CLI orchestration logic using real services to verify at the integration level. ### Changes #### Behave Unit Tests - `features/tdd_plan_execute_phase_processing.feature` — 4 scenarios - `features/steps/tdd_plan_execute_phase_processing_steps.py` — Step definitions using CliRunner and mocked services **Scenarios:** 1. **CLI execute command handles plan in Strategize/QUEUED state** — Invokes `plan execute` via CliRunner on a QUEUED plan. Verifies the CLI succeeds and the plan reaches Execute phase. 2. **CLI execute command orchestrates full lifecycle for QUEUED plan** — Verifies `run_strategize()` and `run_execute()` are both called by the CLI handler. 3. **Positive control — proper orchestration transitions QUEUED plan to Execute** — Demonstrates that `run_strategize()` → `execute_plan()` works correctly at the service level. Always passes. 4. **CLI auto-discovery finds plans in Strategize/QUEUED state** — Invokes `plan execute` with no plan_id. Verifies the auto-discovery filter includes QUEUED plans. #### Robot Integration Tests - `robot/tdd_plan_execute_phase_processing.robot` — 4 test cases matching the Behave scenarios - `robot/helper_tdd_plan_execute_phase_processing.py` — Helper script replicating CLI orchestration logic #### CHANGELOG - `CHANGELOG.md` — Added entry under "## Unreleased" describing the new tests. ### Quality Gates - `nox -e lint`: ✅ passed - `nox -e typecheck`: ✅ passed (0 errors) - `nox -e unit_tests`: ✅ passed (391 features, 11177 scenarios, 0 failures) - `nox -e integration_tests`: ✅ passed (1572 tests, 0 failures) - `nox -e e2e_tests`: ✅ passed (16 tests, 0 failures) - `nox -e coverage_report`: ✅ 97% coverage ### Review Cycle 2 Fixes - **Critical #1**: Rewrote `@tdd_expected_fail` scenarios to exercise the CLI orchestration layer via CliRunner instead of testing service/executor APIs that are correct by design. Removed `@tdd_expected_fail` tags since the bug fix is already in the codebase. - **Major #2**: Added CHANGELOG entry. - **Major #3**: Rebased onto current `master`. - **Minor #4**: Added `ProcessingState.QUEUED` assertion to positive control scenario. - **Minor #5**: Narrowed exception handler from bare `except Exception` to `except (PlanError, PlanNotReadyError)`. - **Minor #7**: Changed Robot suite to use `Setup Test Environment With Database Isolation`. - **Minor #8**: Added `on_timeout=kill` to all Robot `Run Process` calls. - **Minor #12**: Added docstring to `_fail()` helper. - **Minor #13**: Removed redundant `Settings()` instantiation. ### Known Limitations - The `@tdd_expected_fail` tags were removed because the bug fix for #967 is already in the codebase. If this PR is merged before the #967 fix PR, the tags would need to be re-added. However, the CHANGELOG and existing CLI code confirm the fix is already on `master`. Closes #977 Reviewed-on: #1050 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 06:41:26 +00:00
hurui200320	747d8d3c9a	test(cli): TDD failing tests for init --yes non-interactive (bug #783) (#1049) ... ## Summary Adds TDD bug-capture tests proving that `agents init --yes` fails to bypass the migration approval prompt in a TTY environment (bug #783). These tests follow the mandatory TDD bug-fix workflow defined in CONTRIBUTING.md §Bug Fix Workflow. ### Changes - **Behave feature** (`features/tdd_init_yes_no_input.feature`): Two scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_783` that invoke `agents init --yes` in a fresh environment with the migration prompt simulated as declining (TTY with no input). The tests verify the command exits successfully and does not display the "Apply migrations now?" prompt. - **Behave steps** (`features/steps/tdd_init_yes_no_input_steps.py`): Step definitions that create an isolated temp directory, clear auto-apply and database-URL environment variables, patch `sys.stdin` on the real `sys` module (correct mock target per project convention), and replace `MigrationRunner._default_prompt_for_migration` with a function that returns `False` (simulating a TTY prompt where the user declines). Temp directory and env var cleanup is registered via `context.add_cleanup()`. - **Robot Framework test** (`robot/tdd_init_yes_no_input.robot`): Two integration test cases tagged `tdd_bug`, `tdd_bug_783`, `tdd_expected_fail` matching the Behave scenarios. - **Robot helper** (`robot/helper_tdd_init_yes_no_input.py`): Helper script that exercises the same code path using the same mock strategy as the Behave steps. ### Bug Reproduction Mechanism The core challenge is that `CliRunner` replaces `sys.stdin` during `invoke()`, making direct `isatty()` patching ineffective for controlling the prompt path. The mock strategy addresses this with a two-pronged approach: 1. **`patch.object(sys, "stdin", mock_stdin)`** — Patches `sys.stdin` directly on the real `sys` module (correct mock target per the project's established pattern in `features/steps/migration_runner_steps.py`). Documents the intent of simulating a TTY. 2. **`patch.object(MigrationRunner, "_default_prompt_for_migration", ...)`** — Replaces the prompt function with one that returns `False` (simulating a TTY user declining migration). This is the mechanism that actually exercises the bug path, since CliRunner's stdin replacement bypasses the `isatty()` check. 3. **`CLEVERAGENTS_DATABASE_URL` with non-template filename** — Sets the database URL to a path whose filename does not match the `before_scenario` template-DB prefixes, forcing the real Alembic migration path instead of the test fast-path. With bug #783 present, `require_confirmation=True` is hardcoded in `UnitOfWork._ensure_database_initialized()`, so the prompt fires, returns `False`, causing `MigrationNotApprovedError` and a non-zero exit code. After the fix, `--yes` should bypass the prompt entirely. ### Root Cause (for the bug fix developer) `init_command()` in `cleveragents.cli.commands.project` receives the `--yes` flag but only uses it to control output format. It does not forward `yes` to the migration runner. The migration runner's `init_or_upgrade()` is called via `unit_of_work._ensure_database_initialized()` with `require_confirmation=True` hardcoded. ### Scenario 2 Limitation While the bug is present, Scenario 2's first assertion (exit code 0) fails and Behave skips subsequent steps. The "contains Initialized" assertion is only evaluated after the bug is fixed, providing distinct post-fix regression value. ### Quality Gate Results - `nox -s lint` — ✅ passed - `nox -s typecheck` — ✅ passed (0 errors) - `nox -s unit_tests` — ✅ passed (387 features, 11121 scenarios, 0 failed) - `nox -s integration_tests` — ✅ passed (1561 tests, 0 failed) - `nox -s e2e_tests` — ✅ passed (16 tests, 0 failed) - `nox -s coverage_report` — ✅ passed (97% coverage) Closes #842 Reviewed-on: #1049 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-19 06:31:20 +00:00
freemo	ad98d41d61	feat(a2a): implement _cleveragents/ extension method routing ... Add spec-aligned _cleveragents/ prefixed extension method routing to the A2A local facade per ADR-047. The facade now supports 42 total operations: 31 new extension methods across 6 families plus 11 legacy proprietary names retained for backward compatibility. Extension method families implemented: - _cleveragents/plan/* (13 methods): use, execute, apply, cancel, status, tree, explain, correct, diff, artifacts, prompt, rollback, list. Plan operations delegate to PlanLifecycleService when wired; new operations (cancel, tree, explain, correct, artifacts, prompt, rollback, list) have stub handlers returning safe defaults. - _cleveragents/registry/* (6 methods): tool/list, resource/list, actor/list, skill/list, action/list, project/list. Tool and resource list delegate to existing services; entity lists are stubs. - _cleveragents/context/* (4 methods): show (delegates to existing handler), inspect, simulate, set (stubs). - _cleveragents/health/* (2 methods): check, diagnostics/run. - _cleveragents/sync/* (3 methods): pull, push, status (stubs). - _cleveragents/namespace/* (3 methods): list, show, members (stubs). Legacy proprietary names (session.create, plan.create, etc.) continue to work via the same handler map, marked deprecated. Updated existing test assertions for the expanded operation count (11 -> 42). ISSUES CLOSED: #876	2026-03-18 17:33:30 +00:00
freemo	21e9a65c33	fix(plan): implement end-to-end subplan execution orchestration ... Replace the metadata-only subplan spawn with real child Plan domain object creation. SubplanService.spawn() now creates full Plan instances with proper PlanIdentity (linking parent_plan_id and root_plan_id), sets them to PlanPhase.STRATEGIZE / ProcessingState.QUEUED, and returns them in the SpawnResult.child_plans list for downstream lifecycle orchestration. Key changes: - SubplanService.spawn() now creates Plan domain objects for each spawn entry with complete PlanIdentity linking, inheriting the parent plan's namespace, actors, project links, definition_of_done, and access settings. - SpawnResult dataclass extended with child_plans: list[Plan] field to carry the created child plans alongside the existing metadata and statuses. - Parent plan's subplan_statuses list is updated with SubplanStatus entries tracking each child's lifecycle state. - Fixed Pyright type errors: added missing definition_of_done, reusable, read_only, and server parameters to Plan and NamespacedName constructors. - Removed @tdd_expected_fail tags from TDD test files since the bug is fixed. - Added Behave scenarios and Robot integration tests verifying child plan creation, lifecycle phase/state, and parent tracking. ISSUES CLOSED: #823	2026-03-18 17:07:07 +00:00
freemo	ec4c39aecf	fix(plan): implement real checkpoint rollback via git reset ... Replace the simulated rollback in CheckpointService.rollback_to_checkpoint() with real git operations. The method now executes git reset --hard <sandbox_ref> followed by git clean -fd inside the sandbox working directory, reverting all tracked file changes and removing untracked files added after the checkpoint. Key changes: - CheckpointService.rollback_to_checkpoint() now calls _git_reset_hard() and _git_clean() via subprocess against the sandbox path, enforcing sandbox boundary by confining all git operations to cwd=sandbox_path. - Added _resolve_sandbox_path() to extract sandbox validation into a dedicated method, supporting both lifecycle-service and in-memory fallback paths. - Added _validate_sandbox() to verify the sandbox path is a directory containing a .git subdirectory before executing git operations. - Added _git_changed_paths() to compute the diff between HEAD and the target ref before reset, providing accurate restored file counts in the result. - Domain event emission (CHECKPOINT_RESTORED) added via the optional event_bus, following the same EventBus protocol pattern used by other services. - Updated existing Robot Framework helper (helper_checkpoint_rollback.py) to use a real temporary git workspace instead of a fake sandbox path, since _validate_sandbox() now enforces that the path is a real git repository. - Removed @tdd_expected_fail tags from TDD test files since the bug is now fixed. - Added new Behave scenarios and Robot integration tests for real file reversion, file removal after rollback, domain event emission, and sandbox boundary enforcement. ISSUES CLOSED: #822	2026-03-18 16:51:14 +00:00
aditya	d9e51d98f8	fix(tui,cli,tests): harden persona/input modes and stabilize parallel test execution ... - Add shell safety controls for REPL/TUI (`looks_dangerous`, confirmation gate, timeout handling, and env-based shell disable guard). - Secure persona workflows with strict name/path validation, safe import/export resolution, atomic+locked registry writes, and malformed YAML resilience. - Unify persona models and wiring by reusing canonical TUI schema/registry, adding DI providers, and lazy-loading TUI exports to avoid circular imports. - Improve reference discovery with ignored-directory filtering, symlink-safe walking, and TTL caching for CLI/TUI reference catalogs. - Expand Behave/Robot coverage for safety/error paths and parallel-isolation behavior; add shared `features/mocks/fake_repl_input.py` helper. - Fix parallel-run flakiness via deterministic cleanup/reload patterns and watchdog polling fallback when inotify limits are reached. ISSUES CLOSED: #695	2026-03-18 10:09:06 +00:00
aditya	c02f3842ae	feat(tui): implement persona system and reference/command input modes ... - Add `agents tui` command wiring and Textual app scaffolding for interactive TUI startup. - Implement TUI persona schema/registry/state with local YAML persistence and per-session persona binding. - Add three input-mode flows: Normal (`@` references), Command (`/` slash commands), and Shell (`!` passthrough). - Introduce TUI widgets/overlays for prompt, persona bar, reference picker, and slash command interactions. - Extend REPL command routing to support persona/session commands and reference/shell dispatch behavior. - Add test coverage: Behave features + step definitions, Robot TUI smoke test, and ASV fuzzy-reference benchmark. ISSUES CLOSED: #695	2026-03-18 10:09:05 +00:00
hurui200320	48ecf4c00c	fix(cli): add --execution-env-priority flag to plan use (#972) ... ## Summary Adds the missing `--execution-env-priority` flag to the `agents plan use` command, aligning the CLI with the specification (spec line 12501). The flag accepts `fallback` (default) or `override` and controls execution environment routing precedence per ADR-043: - **`override`**: The specified execution environment always wins, bypassing devcontainer auto-detection. - **`fallback`**: The specified environment defers to auto-detected devcontainers or project-level overrides. ### Changes - **Domain model** (`cleveragents.domain.models.core.plan`): - Added `ExecutionEnvPriority` StrEnum with `FALLBACK`/`OVERRIDE` values. - Changed `execution_env_priority` field type to `ExecutionEnvPriority | None` (leverages Pydantic enum validation). - Added `@model_validator` enforcing that `execution_env_priority` requires `execution_environment` (domain-level fail-fast invariant). Verified `validate_assignment=True` is set on `Plan.model_config`. - Updated `Plan.as_cli_dict()` to include `execution_environment` and `execution_env_priority`, with fallback default of `"fallback"` for pre-migration data where `execution_env_priority` is `None`. - **CLI** (`cleveragents.cli.commands.plan`): - Added `--execution-env-priority` parameter to `use_action`. - Validation: priority requires `--execution-environment`, enum value validation, case-insensitive input. - Defaults to `"fallback"` when `--execution-environment` is set without explicit priority. - Updated `_print_lifecycle_plan` and `_plan_spec_dict` to display the priority, defaulting to `"fallback"` for pre-migration data. - Updated `_plan_spec_dict` docstring to mention new keys. - Hoisted `ExecutionEnvPriority` import to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan` (no longer conditional on execution environment being set). - Guarded `service.save_plan(plan)` with `has_overrides` flag so it is only called when CLI overrides were actually applied. - **Persistence** (`cleveragents.infrastructure.database`): - Added `execution_environment` (`String(255)`, nullable) and `execution_env_priority` (`String(20)`, nullable) columns to `LifecyclePlanModel`. - Updated `from_domain()`/`to_domain()` for round-trip serialization including `ExecutionEnvPriority` enum reconstruction. Uses direct attribute access (`plan.execution_environment`) instead of defensive `getattr` — Plan fields always exist on the Pydantic BaseModel. - Updated `LifecyclePlanRepository.update()` to persist both fields using direct attribute access. - Added Alembic migration `m4_003_plan_env_columns` adding both columns to the `v3_plans` table. Uses `String(255)` for `execution_environment` to accommodate namespaced resource names. Descends from `m6_005_profile_guards_json`. - **Service** (`cleveragents.application.services.plan_lifecycle_service`): - Added `save_plan()` public convenience method for callers that need to re-persist after post-creation mutations. - **Tests**: - 18 Behave scenarios covering: - CLI acceptance criteria (valid values, defaults, validation errors, output display, case-insensitive input, service invocation with `call_args` verification). - Domain model validator invariant: construction with priority but no environment raises `ValueError`; construction with both fields succeeds. - `ExecutionEnvPriority` enum: values verification, `StrEnum` subclass assertion. - `Plan.as_cli_dict()`: includes both fields when set, omits both when `None`, defaults priority to `"fallback"` for pre-migration data. - DB round-trip serialization: `from_domain()` → `to_domain()` preserves both fields; preserves `None` values. - 5 Robot Framework integration tests. - Updated pre-existing `SimpleNamespace`-based plan test fixtures in `database_models_lifecycle_coverage_steps`, `database_models_new_coverage_steps`, `database_models_coverage_r2_steps`, and `repositories_error_handling_coverage_steps` to include `execution_environment` and `execution_env_priority` attributes. - Simplified Robot helper `sys.path` pattern to standard approach. - **Changelog**: Updated per CONTRIBUTING.md requirements. ### Review Fixes (Brent Edwards, Review #2384) - **P2 #1 — Defensive `getattr`**: Replaced `getattr(plan, "execution_environment", None)` with direct `plan.execution_environment` access in both `LifecyclePlanModel.from_domain()` and `LifecyclePlanRepository.update()`. Plan is a Pydantic BaseModel with `default=None`, so the field always exists. Updated 4 pre-existing `SimpleNamespace`-based test fixtures to include the new attributes. - **P2 #2 — Late conditional import**: Hoisted `ExecutionEnvPriority` import from inside conditional blocks to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan`. - **P3 — Migration naming**: Acknowledged as deferred (cosmetic only). Updated `m4_003` to descend from `m6_005_profile_guards_json` (post-rebase chain fix). - **Rebase**: Branch rebased onto latest `master` with merge conflict in `CHANGELOG.md` resolved. ### Deferred Items - **Partial failure atomicity** (#8 from review): If `save_plan()` fails after `use_action()` succeeds, the plan exists in the DB without CLI overrides. This requires service-layer restructuring beyond the scope of this ticket. - **Alembic migration naming** (#11 from review): Migration `m4_003` depends on `m6_005`, creating non-sequential naming. Renaming an existing migration risks breaking the chain for anyone who has already applied it. ### Quality Gates | Session | Result | |---------|--------| | lint | PASS | | typecheck | PASS (0 errors) | | unit_tests | PASS (11,125 scenarios, 0 failures) | | integration_tests | PASS (1,562 tests, 0 failures) | | coverage_report | 97% (threshold: 97%) | Closes #886 Reviewed-on: #972 Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-18 08:13:56 +00:00
aditya	2764fcef5c	fix(actor,preflight,tests): resolve PR #975 review findings and stabilize full-suite coverage runs ... Address review-driven fixes across actor schema, preflight guardrails, docs/examples, and Behave/Robot coverage: unify preflight warning behavior with shared role-warning logic, resolve actor-name to config payloads in production preflight flow, harden response_format validation/coercion edge cases, extract duplicated helper logic, and expand negative-path test coverage. Also fix cross-scenario patcher leakage in step modules to eliminate full-run-only coverage failures.	2026-03-18 06:58:39 +00:00
aditya	26ad778aee	feat(estimation): add estimation actor YAML template and role-aware registration validation ... - Add `role_hint` and `response_format` support to actor schema. - Add non-fatal estimation-role compatibility warnings in actor registration CLI flows. - Add preflight warning path when `estimation` actor is missing `response_format`. - Add `examples/actors/estimator.yaml` and update actor examples documentation/tests. - Update integration helper expectations (m1/m2/m3/m6) for missing provider config in local test env. ISSUES CLOSED: #650	2026-03-18 06:46:14 +00:00
hamza.khyari	5d6cb099ad	feat(resource): add deferred virtual resource types ... Add 3 deferred virtual resource types (remote, submodule, symlink) with equivalence metadata for physical-to-virtual resource linking. Depends on: #662 (child_types reference types introduced by #662) - Type definitions extracted to _resource_registry_virtual_deferred.py for consistency with _resource_registry_virtual.py (#329) - YAML configs with equivalence criteria per spec, spec reference comments - Bootstrap registration via BUILTIN_TYPES spread, hidden from resource add scaffolding (user_addable: false) - Equivalence structural validation in ResourceTypeSpec model validator: criteria must be a non-empty list of non-empty strings; virtual types must have sandbox_strategy=none, user_addable=false, handler=None, all capabilities false - Behave tests (52 scenarios), Robot tests (7), ASV benchmarks - DB roundtrip tests verifying virtual types survive bootstrap persistence - Negative tests: missing equivalence/name/kind, manual add rejection for all 3 virtual types (register_resource guard), invalid criteria elements (non-string, empty string) ISSUES CLOSED: #331	2026-03-18 02:30:00 +00:00
hamza.khyari	c14ce65d61	feat(resource): add virtual core resource types ... Add 6 built-in virtual resource types (file, directory, commit, branch, tag, tree) with equivalence metadata for content-hash and git-object identity matching. - YAML configs under examples/resource-types/ - Bootstrap registration with virtual types hidden from resource add scaffolding - Equivalence criteria per spec (content_hash, merkle_hash, git SHA identity) - Behave tests (83 scenarios), Robot tests, ASV benchmarks - Documentation in docs/reference/resource_types_builtin.md ISSUES CLOSED: #329	2026-03-18 01:45:25 +00:00
freemo	c65e8a5285	feat(resource): add cloud infrastructure resources ... Implement cloud resource types (aws, gcp, azure) with credential fields, region/tenant metadata, and stubbed sandbox strategies. Credential resolution uses environment variables and profile names with no secrets logged. Key changes: - Add CloudResourceHandler with aws/gcp/azure type definitions - Add credential resolution from env vars and profile names - Add stubbed sandbox strategies (validate config, raise NotImplementedError) - Register cloud types in bootstrap_builtin_types - Credential masking via existing redaction patterns - Add Behave BDD tests, Robot integration tests, ASV benchmarks ISSUES CLOSED: #343	2026-03-17 13:14:37 -04:00
Luis Mendes	cb583021df	test(e2e): E2E acceptance criteria for M6 (v3.5.0) — autonomy hardening ... Implemented Robot Framework E2E test suite for M6 autonomy hardening acceptance criteria. Tests exercise the real CleverAgents CLI with zero mocking, covering session lifecycle, automation profiles, project setup, plan lifecycle via A2A facade, guard enforcement, and a full autonomy acceptance flow. LLM-dependent tests use Skip If No LLM Keys for graceful degradation when API keys are unavailable. Hardened shared E2E keywords (common_e2e.resource): - Safe JSON parsing with rfind-bounded extraction and error wrapping (C1, M1) - Multi-object fallback: last-line reverse scan for multi-JSON output (M5) - Moved Safe Parse Json Field to common_e2e.resource for reuse (L1) - Migrated deprecated Run Keyword If to IF/ELSE blocks (M4) - Added cwd parameter to Run CleverAgents Command (M6) - API key protection via inline evaluation instead of RF variables (S1) - Git return-code assertions in Create Temp Git Repo (L1) - Removed unused Collections library import (L2) - Warning log on directory removal failure instead of silent ignore (L6) Hardened m6_acceptance.robot: - Force Tags E2E instead of per-test [Tags] (L3) - Per-test [Teardown] for resource cleanup, including Init test (M2, M9, L5) - Collision-safe uuid4 hex suffix instead of randint (L7) - Initialized session_id to EMPTY before test body for safe teardown (L2) - ELSE branches with WARN log on conditional assertions (M3, M4, M6) - Guard enforcement checks specific automation-profile fields (M5) - Strengthened ci assertions with case-sensitive matching and JSON field parsing for config get verification (M9) - Eliminated Python code injection via string interpolation (H2) - Added assertions to Full Flow Apply Step keyword (H4) - Removed redundant config set in Guard test (M10) - Accurate CHANGELOG entry describing actual test scope (C2) Post-review fixes applied: - Verify all 8 built-in profiles (manual, review, supervised, cautious, trusted, auto, ci, full-auto) instead of 4 (M1) - Session delete confirms removal via re-list (M2) - Full Flow Apply Step verifies plan phase transition (M3) - Plan execute output asserts plan_id presence + phase parsing (M4) - Safe Parse Json Field gains two-strategy approach: outer-bracket extraction then last-line reverse scan fallback (M5) - Removed redundant automation profile reset in Config test (L1) - JSON-quoted assertions ("ci", "auto") prevent false-positive substring matches on short profile names (L3) New E2E tests covering remaining acceptance criteria: - Guard enforcement with custom profile: registers a profile with explicit denylist, budget cap, and tool-call limits, then verifies all guard fields via automation-profile show (AC-4 / H1) - Profile precedence resolution: sets global profile to "review", creates plan with --automation-profile trusted, asserts plan output shows "trusted" not "review" (AC-5 / C2) - Event queue via plan lifecycle transitions: creates plan, captures initial state, executes, verifies state transition proving domain event bus delivered and processed events (AC-3 / C1) - Hierarchical decomposition via plan tree: creates plan with full-auto profile, executes, runs plan tree --format json, verifies decision nodes and children structure (AC-6 / C3) Robot Framework uses dots as hierarchy separators in suite names (e.g. "E2E.M6 Acceptance"). The E2E Suite Setup keyword replaced spaces with underscores but preserved dots, producing directory names like "E2E.M6_Acceptance". The CLI init command derives the project name from Path.cwd().name, and Project.validate_name() rejects dots ("Name must be alphanumeric with hyphens, underscores, or spaces"). Added a second Replace String call to convert dots to underscores so the sanitized suite name passes Project name validation. Also increased subprocess timeouts in m3_e2e_verification.robot (60s->120s) and m4_e2e_verification.robot (30s->120s) to prevent flaky CI failures from Python startup overhead. The plan lifecycle E2E tests call "plan use local/code-review" but actions are user-defined entities that must be explicitly registered via "action create --config <yaml>" before use. Without the action, plan use failed with "Action 'local/code-review' not found" and all LLM-dependent tests skipped. Added action registration to M6 Suite Setup that dynamically selects the actor matching the available API key (anthropic/claude-sonnet-4 when ANTHROPIC_API_KEY is set, openai/gpt-4o otherwise) and creates the action YAML inline. ISSUES CLOSED: #746	2026-03-17 10:13:23 +00:00
Luis Mendes	ab911dbdc4	fix(cli): write machine-readable formats directly to stdout bypassing Rich line-wrapping ... The format_output() function returned a string that callers passed to Rich console.print(), which wraps long lines at terminal width. This injected literal newline characters into JSON string values (e.g. in definition_of_done fields), producing invalid JSON that downstream parsers could not decode (JSONDecodeError: Invalid control character). For machine-readable formats (json, yaml, plain), format_output() now writes the rendered output directly to sys.stdout and returns an empty string. This preserves the exact serialization from json.dumps/ yaml.dump without Rich text processing artifacts. Refs: #746	2026-03-17 09:53:57 +00:00
Luis Mendes	2acb957d83	fix(cli): replace in-memory automation-profile repository with database-backed persistence ... The automation-profile CLI commands used an _InMemoryProfileRepository (a Python dict) that lost all data between CLI process invocations. Profiles created with "automation-profile add" were invisible to subsequent "automation-profile show" or "list" calls because each CLI command is a separate process with a fresh empty dict. Changes: - Replaced _InMemoryProfileRepository with the real AutomationProfileRepository from the infrastructure layer, wired via the DI container following the same pattern as tool.py and session.py. - Added auto_commit support to AutomationProfileRepository (matching the existing SessionRepository pattern) so that CLI commands running outside a UnitOfWork commit each operation automatically. - Added safety_json and guards_json Text columns to the automation_profiles table (Alembic migration m6_005) for full-fidelity round-trip of the AutomationGuard and SafetyProfile sub-models. Previously, guards and several safety fields (max_cost_per_plan, max_retries_per_step, etc.) were silently dropped on persistence. - Updated _from_domain, _to_domain, and _update_row to serialize and deserialize the full guard and safety sub-models via JSON, with backward-compatible fallback to legacy scalar columns. Refs: #746	2026-03-17 09:53:53 +00:00
brent.edwards	c13a62a2f2	Merge branch 'master' into feature/m5-robot-tdd-tags	2026-03-16 23:06:19 +00:00
brent.edwards	d0ca129d90	Merge branch 'master' into feature/m5-robot-tdd-tags	2026-03-16 23:05:43 +00:00
freemo	2688c85769	feat(extensibility): implement Custom Sandbox Strategy Registration via SandboxStrategy Protocol ... Implement SandboxStrategyProtocol, a 9-method @runtime_checkable Protocol enabling third-party sandbox strategy registration. Includes: - SandboxStrategyProtocol with create/read/write/diff/commit/rollback/checkpoint/ restore_checkpoint/cleanup methods - SandboxRef (frozen dataclass) and DiffView/DiffEntry (Pydantic models) - SandboxStrategyRegistry with config-driven registration, Protocol validation, thread safety, and clear/list/has operations - BuiltInSandboxStrategyAdapter wrapping existing Sandbox implementations to conform to the new Protocol - CustomStrategyConfig for YAML/dict-based strategy registration - SandboxFactory integration with custom_registry parameter, has_custom_strategy() and get_custom_strategy_class() - 25 Behave BDD scenarios (85 steps) covering protocol, registry, adapter, config, and factory integration - 8 Robot Framework integration tests with real filesystem operations - ASV benchmarks for registry and adapter operations - Developer documentation ISSUES CLOSED: #586	2026-03-16 22:50:27 +00:00
hurui200320	a5de448856	feat(testing): implement @tdd_expected_fail tag handling in Robot Framework ... Implements the three-tag TDD bug-capture system in Robot Framework via a Listener v3 module, paralleling the Behave implementation. Tests tagged tdd_expected_fail that fail have their result inverted to PASS (bug still exists); tests that unexpectedly pass are inverted to FAIL with guidance. Addresses all 15 findings from code review (PR !673, reviewer hamza.khyari): P2 fixes: - Added idempotency guard (_processed_tests set) to prevent double-inversion when the listener is loaded twice in the same process. - Rewrote normal-test-unaffected check to run alongside a tdd_expected_fail fixture in a single Robot invocation, proving the listener is loaded and selectively applies rather than being a tautological pass. P3 fixes: - Added output.xml existence guard with clear diagnostics in _run_fixture. - Documented intentional use of data.tags (static definition) vs result.tags (runtime-modifiable) in end_test docstring. - Added SKIP status test fixture and integration test case. - Added message content assertion in cmd_expected_fail_inverted. - Tightened substring assertions to match specific error text. - Added tdd_expected_fail-alone fixture (both companions missing). - Added close() hook to clear _validation_errors and _processed_tests. - Simplified _run_fixture return type to tuple[str, str]. - Changed listener path resolution from CWD-relative to __file__-relative in noxfile.py (integration_tests, slow_integration_tests, e2e_tests). P4 fixes: - Added __all__ declaration to helper module. - Changed module docstring from "mirroring" to "paralleling". - Added comment documenting accepted XML parsing risk (self-generated XML). Additional fixes: - Increased M4 E2E plan-tree test timeout from 30s to 120s (pre-existing timeout failure unrelated to this feature). Quality gates (post-rebase onto latest master): - nox -s lint: PASS - nox -s typecheck: PASS (0 errors) - nox -s unit_tests: PASS (10,700 scenarios) - nox -s integration_tests: PASS (1,505 tests) - nox -s coverage_report: PASS (97.9% >= 97% threshold) - nox -s benchmark: PASS - nox -s docs: PASS - nox -s build: PASS - nox -s security_scan: PASS - nox -s dead_code: PASS ISSUES CLOSED: #628	2026-03-16 22:45:55 +00:00
hamza.khyari	89eaee008d	feat(acms): implement UKO Layer 3 Technology Vocabularies (uko-py, uko-ts, uko-rs, uko-java) ... Implement Layer 3 technology-specific UKO vocabulary extensions for Python, TypeScript, Rust, and Java with language-specific classes, properties, and DetailLevelMap insertions. - 4 OWL/Turtle ontology files with language-specific semantic classes - DetailLevelMap insertion logic with correct integer reassignment - Provenance contract (5 required fields per spec) - Full 4-layer chain resolution (Layer 3 -> Layer 2 -> Layer 1 -> Layer 0) - Comprehensive Behave test suite (63 scenarios) ISSUES CLOSED: #576	2026-03-16 12:11:08 +00:00
brent.edwards	3119383529	Merge branch 'master' into tdd/m6-checkpoint-real-rollback	2026-03-16 03:15:10 +00:00
brent.edwards	3eecb79003	test(plan): TDD failing tests for checkpoint real rollback (bug #822) ... TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839	2026-03-16 01:45:50 +00:00
brent.edwards	b67dc63eda	test(plan): TDD failing tests for subplan spawn orchestration (bug #823) ... Write Behave scenario and Robot Framework test proving that SubplanService.spawn() only creates metadata (SubplanStatus records and SpawnMetadata) without creating real child Plan domain objects or triggering lifecycle progression. Tests are tagged @tdd_expected_fail so CI passes via result inversion. ISSUES CLOSED: #838	2026-03-16 01:12:24 +00:00
freemo	065171f21c	test(e2e): E2E acceptance criteria for M2 (v3.1.0) — actor compiler and LLM integration ... Add Robot Framework E2E test suite robot/e2e/m2_acceptance.robot exercising M2 acceptance criteria with zero mocking. Test creates a temp git repo with sample project files, registers a custom actor via CLI, sets up resource and project, creates an action referencing the actor, and runs the full plan lifecycle (use → execute strategize → execute → diff → apply). Validates actor YAML compilation, skill registry, tool lifecycle, and LLM integration through real CLI invocations with real provider API keys. Uses flexible structural assertions and expected_rc=None for LLM-dependent commands. ISSUES CLOSED: #742	2026-03-15 19:32:44 -04:00
freemo	5f07316641	fix: wire DI persistence and plan execute/apply for M1 lifecycle ... Fixed 5 bugs preventing the M1 E2E acceptance test from passing: 1. _get_lifecycle_service() in action.py and plan.py bypassed the DI container, creating PlanLifecycleService without UnitOfWork. All plan/action data was in-memory only and lost between subprocess calls. Now uses container.plan_lifecycle_service() for DB persistence. 2. `plan execute` CLI only called service.execute_plan() (a pure state transition) without running PlanExecutor phase processing. Rewrote to detect the plan's current phase/state and dispatch synchronously: Strategize/queued → run_strategize(), Strategize/complete → transition + run_execute(), Execute/queued → run_execute(). 3. `plan apply` CLI had no plan_id argument. Added optional positional plan_id with _lifecycle_apply_with_id() that drives the plan through Apply/queued → Apply/processing → Apply/applied. 4. Preflight guardrail in start_strategize() built action_registry from the in-memory _actions dict only. Added get_action(plan.action_name) call to load the action from DB into cache before the guardrail check. 5. Robot Framework Create File syntax used continuation lines producing 9 arguments instead of 1. Fixed to use Catenate SEPARATOR=\n then pass single variable to Create File. Also fixed --branch main to --branch master (git init default). update mocks for execute_plan CLI changes across unit and integration tests The new execute_plan() command calls _get_plan_executor() and service.get_plan(plan_id) for phase/state detection. Existing tests only mocked _get_lifecycle_service, so MagicMock defaults caused phase/state comparisons to fail. Changes across 14 files: - Patch _get_plan_executor in all test setups that invoke the CLI execute command (Behave step files + Robot helper scripts) - Set service.get_plan.return_value to real Plan objects with correct phase/state so the execute_plan dispatch logic works - Fix error-path tests to use STRATEGIZE/COMPLETE plans so the error side_effects are actually reached - Fix "Multiple plans eligible" → "Multiple plans ready" message text to match existing test expectations increase Robot Framework subprocess timeouts for CI resource contention Three integration tests were timing out in CI due to resource contention when pabot runs multiple test suites in parallel. All three pass locally and the timeouts were simply too tight for constrained CI environments. - tdd_session_create_di.robot: 30s → 90s (DI container init + DB setup) - database_integration.robot: 60s → 120s (Run Python Script keyword) - m3_e2e_verification.robot: 60s → 120s (correction-live-revert spawns 3 sequential CLI subprocesses with full container initialization) ISSUES CLOSED: #789	2026-03-15 20:50:02 +00:00
freemo	cb3b7aab44	test(e2e): E2E acceptance criteria for M1 (v3.0.0) — minimal plan execution flow ... Added Robot Framework E2E test suite for M1 milestone acceptance criteria. Tests the complete plan lifecycle (action create → resource add → project create → plan use → plan execute strategize → plan execute → plan diff → plan apply) with real LLM API keys and no mocking. Key implementation details: - Uses openai/gpt-4o-mini as strategy/execution actor (cost-effective) - Simple definition_of_done: "Create a file called HELLO.md" - Creates isolated temp git repo via Create Temp Git Repo keyword - Extracts plan ID via ULID regex from plain-text output - Uses expected_rc=None for LLM-dependent steps (execute, diff, apply) to handle non-deterministic LLM behavior gracefully - Flexible structural assertions: checks rc, output presence, git log - Skips gracefully when no LLM API keys (ANTHROPIC/OPENAI) are set - Tagged [E2E] so it runs only in nox -s e2e_tests session ISSUES CLOSED: #741	2026-03-15 20:50:02 +00:00
freemo	ec450e9085	fix(test): fix tolerant exit code and missing RC check in resource CLI test ... - Replace `rc == 0 or rc == 1` with strict `rc == 0` in resource type list test so failures are no longer silently accepted - Capture and assert the return code of the Suite Setup database schema creation to fail fast if the setup itself is broken apply strict RC checks to resource_cli.robot - Capture return value of Run Process in Suite Setup and assert rc==0 - Replace tolerant RC check (rc==0 or rc==1) with strict Should Be Equal As Integers check for resource type list test case broaden exception handling in resource CLI commands Add catch-all `except Exception` handler after each `except CleverAgentsError` block in all 14 resource CLI command handlers. This ensures unexpected exceptions (e.g. sqlalchemy.exc.OperationalError) are caught and displayed gracefully instead of producing raw tracebacks. re-raise typer.Abort/Exit in broad exception handlers The `except Exception` handlers added in the previous commit inadvertently caught typer.Abort and typer.Exit, which are subclasses of Exception (via click.exceptions). This turned successful CLI exits into aborts and double-handled already-caught errors, breaking integration tests that rely on normal typer exit behaviour. Add an isinstance guard to re-raise typer.Abort and typer.Exit before the catch-all handler runs. Fixes #896	2026-03-14 15:47:03 -04:00
freemo	a69af285a8	fix(test): remove retry masking and output filtering in database integration test ... - Remove silent retry logic in Run Python Script that masked intermittent failures by retrying on empty stdout + non-zero RC - Change output filter from 'in' to 'startswith' so only lines beginning with log prefixes are filtered, not lines containing them mid-string - Remove [error from the filter list so error lines are never silently discarded from test output Fixes #898	2026-03-14 03:45:18 +00:00
freemo	64e606099c	fix(test): convert soft warnings to hard failures in architecture test ... Replace Run Keyword And Return Status soft checks with direct assertions in three test cases: - Docstring check now uses Should Contain directly instead of logging WARN, so missing docstrings cause a hard test failure. - Settings file (config/settings.py) existence is now a hard File Should Exist assertion instead of a conditional that silently skips all checks when the file is absent. - Exceptions file (core/exceptions.py) existence is now a hard File Should Exist assertion instead of a conditional that silently skips the exception hierarchy checks when the file is absent.	2026-03-14 03:33:37 +00:00
freemo	586ea4557f	fix(test): add return code checks to intermediate commands in core CLI test ... Add rc == 0 assertions to ~11 intermediate Run Process calls in core_cli_commands.robot that were used as setup steps without any return code verification. If any setup command (init, tell, build, context add, plan new) fails silently, subsequent test assertions could pass for wrong reasons. All setup commands now capture their result and assert rc == 0.	2026-03-14 01:35:49 +00:00
freemo	c169cb201d	fix(test): remove skip guard and soft assertions from scientific paper E2E test ... - Remove Require OpenAI Key skip guard that silently skipped the suite - Replace Run Keyword And Return Status soft RC check with direct assertion - Convert conditional LaTeX presence check to hard assertions - Convert conditional latex_source context field check to hard assertion - Remove dead stderr checks (stderr redirected to stdout via stderr=STDOUT)	2026-03-14 00:50:29 +00:00
freemo	77a7813f0e	fix(test): tighten diagnostics check exit code assertion in CLI core test ... Replace the tolerant "rc == 0 or rc == 1" assertion with a strict "rc == 0" check for the diagnostics --check command. In a clean test environment, diagnostics should complete without finding errors, and the previous assertion could not distinguish a legitimate diagnostic failure from a command crash.	2026-03-14 00:37:34 +00:00
freemo	eb2c2362bd	fix(test): remove skip guard from scientific paper basic test ... Remove the Require OpenAI Key keyword and its call from Suite Setup. Tests now fail with a real error when OPENAI_API_KEY is absent, providing honest signal instead of silently skipping.	2026-03-13 23:55:27 +00:00
hamza.khyari	3c014a9565	feat(acms): implement UKO Layer 2 Paradigm Vocabularies (uko-oo, uko-func, uko-proc)	2026-03-13 18:21:50 +00:00
hurui200320	317d015260	test(e2e): validate M4 acceptance criteria for v3.3.0 milestone closure ... Strengthen M4 E2E CLI acceptance tests and address all review findings from the self-review on PR #681 (3 blocking, 7 non-blocking issues). Blocking fixes: - Remove tautological domain assertions from cli_plan_execute() that verified values the test itself constructed via _mock_parent_plan and could never fail. Replace with a comment documenting that subplan info cannot be verified from CLI output alone. - Fix unguarded positional arg access in plan_diff() that would raise IndexError if the production CLI passes plan_id as a keyword argument. Now uses kwargs-with-positional-fallback pattern. - Split the 1074-line helper_m4_e2e_verification.py into four focused modules under the CONTRIBUTING.md 500-line limit: helper_m4_e2e_common.py (237 lines) — constants, helpers, factories helper_m4_e2e_domain.py (495 lines) — domain model tests helper_m4_e2e_cli.py (426 lines) — CLI integration tests helper_m4_e2e_verification.py (81 lines) — thin dispatcher Non-blocking improvements: - Add ProjectLink isinstance type guard before iterating project_links. - Wrap all assert_called_once*() calls in try/except with _fail() pattern to produce standardised FAIL: diagnostics instead of raw AssertionError. - Make project_links extraction symmetric with action_name (kwargs + positional fallback at index 1). - Replace broad "execute" substring match with word-boundary regex r"\bexecute\b" to avoid false positives from incidental substrings. - Extract _make_subplan_status() factory, _assert_exit_code() and _assert_mock_called_once*() wrappers to eliminate DRY violations (SubplanStatus construction repeated 3x, exit-code checks 4x). - Replace non-deterministic datetime.now() calls with frozen constant _FROZEN_NOW = datetime(2026, 3, 1, 12, 0, 0) for deterministic fixtures. - Rename generic _DECISION_ULID_N constants to descriptive names (_DECISION_PROMPT_DEF, _DECISION_STRATEGY, _DECISION_PARALLEL_SPAWN, _DECISION_SPAWN_API, _DECISION_SPAWN_UI). Quality gates: - lint: PASS - format: PASS (1404 files unchanged) - typecheck: PASS (0 errors) - unit_tests: 10,431 scenarios, 0 failures - integration_tests: 1,452 tests, 0 failures - coverage_report: PASS (98%, threshold 97%) - security_scan: PASS - dead_code: PASS - docs: PASS - build: PASS ISSUES CLOSED: #495	2026-03-13 14:03:37 +08:00
hurui200320	eb770643c2	test(e2e): validate M5 acceptance criteria for v3.4.0 milestone closure ... Add four CLI-based integration test cases to the M5 E2E verification suite that exercise the exact commands from the v3.4.0 milestone description via real subprocess calls to `python -m cleveragents`. CLI test cases: - CLI Project Create Large Project: `agents project create local/large-project` - CLI Resource Add Git Checkout: `agents resource add git-checkout ...` - CLI Project Link Resource: `agents project link-resource ...` - CLI Project Show Displays Linked Resource: `agents project show ...` Each test creates an isolated temp directory with its own CLEVERAGENTS_HOME, initialises a workspace via `agents init`, runs the target CLI command as a subprocess with `on_timeout=kill`, asserts zero exit code and verifies expected output, then tears down the temp directory. Bug fix: ProjectResourceLinkRepository.create_link() and remove_link() only called session.flush() without session.commit(), causing linked resource data to be silently lost between sessions. Added session.commit() to both methods, plus finally: session.close() to match the session-factory lifecycle pattern used by all other mutating repository methods. Added session.refresh() and session.expunge() before return in create_link() so the returned ORM instance is fully loaded and usable in detached state after session close. Regression guard: Added two cross-session persistence Behave scenarios (project_repository.feature) that open a new session from the same engine after create_link/remove_link and verify the operation was durably committed. Test improvements from review feedback (rounds 1 and 2): - Per-test CLEVERAGENTS_HOME isolation via env: override on all Run Process calls to prevent shared state between tests. - Stronger ULID-based assertions: CLI tests capture the resource_id ULID from resource show output and verify the exact ID in project show output. - Regex-based branch assertion (branch.*main) instead of generic string match. - Test 4 differentiated from Test 3 by verifying resource show independently after linking and asserting the specific resource ULID. - Documentation noting context tier and ACMS criteria are validated at the Python API level via helper_m5_e2e_verification.py. - Updated ProjectResourceLinkRepository class docstring documenting commit and close behaviour of mutating methods. - CHANGELOG entry for both test additions and production bug fix. - Redundant Library imports removed, --format plain on project list, comments explaining fake repo directories, standardised Run Process line style. Context/ACMS CLI coverage is intentionally not added because the CLI does not yet expose dedicated context/ACMS inspection commands. These criteria are validated at the Python API level via helper_m5_e2e_verification.py. ISSUES CLOSED: #496	2026-03-13 13:32:52 +08:00
CoreRasurae	3e3e9b4b5d	feat(observability): wire AuditService.record() into domain services via EventBus auto-dispatch ... Implements AuditEventSubscriber that subscribes to all 9 security-relevant EventType members and persists redacted audit entries via AuditService.record(). Key components: - AuditEventSubscriber: bridges EventBus and AuditService (SEC7) - SECURITY_EVENT_MAP: maps EventType enum to audit type strings - Redaction via redact_dict() on event details before persistence - Graceful error handling: failures logged, never propagated Post-review fixes applied: - BUG-1: Remove dead correlation_id null-check guard (DomainEvent.correlation_id is always non-None via ULID default_factory) - SEC-2: Redact exception messages in warning logs via redact_value() to prevent potential leakage of sensitive internal state (e.g. DB connection strings) - PERF-3: Pre-generate unique DomainEvent instances in ASV benchmark setup to avoid skew from reusing a single frozen object - Wire event_bus from the DI container into CorrectionService (plan.py), ConfigService (config.py, skill.py x2, server.py), and PersistentSessionService (session.py) at their CLI construction sites. Closes #581	2026-03-12 23:32:48 +00:00
brent.edwards	717550c59e	Merge branch 'master' into bugfix/m6-e2e-mock-only-coverage	2026-03-12 22:03:28 +00:00
brent.edwards	94d953246d	fix(test): correct mock wiring in plan_correct_tree_wiring Robot helper ... Replace container.resolve.return_value with container.decision_service.return_value to match actual CLI code (plan.py uses container.decision_service()). This was the residual pattern from before the DI migration in this same PR. Fixes CI: Robot.Plan Correct Tree Wiring (2 of 3 tests failing with exit code 1)	2026-03-12 21:53:38 +00:00
brent.edwards	703511a28f	Merge branch 'master' into tdd/m6-e2e-mock-only-coverage	2026-03-12 21:43:23 +00:00
brent.edwards	76fb84c3d0	Merge remote-tracking branch 'origin/master' into bugfix/m3-session-list-error ... # Conflicts: # CHANGELOG.md	2026-03-12 21:22:20 +00:00
freemo	d4a1a0d87b	test(e2e): set up E2E test infrastructure — nox session, CI job, Robot Framework @E2E tag ... Added dedicated E2E test infrastructure completely separate from the existing integration test suite. E2E tests use zero mocking — they exercise the real CleverAgents CLI with real LLM API keys. Key changes: - New e2e_tests nox session running Robot Framework with --include E2E tag filter against robot/e2e/ directory. Uses sequential robot (not pabot) since E2E tests hit real API endpoints with rate limits. Propagates ANTHROPIC_API_KEY, OPENAI_API_KEY, and GOOGLE_API_KEY from environment. Output goes to build/reports/robot-e2e/ to avoid artifact collisions. - Existing integration_tests session now excludes E2E-tagged tests via --exclude E2E on the pabot invocation. - New robot/e2e/common_e2e.resource provides shared E2E keywords: suite setup/teardown with per-suite isolation (no mock AI), graceful skip when LLM API keys are absent, CLI runner keyword, flexible output assertions, and temporary git repo fixture creation. - Minimal smoke test (robot/e2e/smoke_test.robot) validates the harness by running agents --version and agents --help. Does not require LLM keys. - Dedicated e2e_tests CI job in .forgejo/workflows/ci.yml injects LLM API keys from Forgejo secrets. Runs independently (no needs dependencies) and does not block regular CI. - e2e_tests is deliberately NOT in the default nox sessions list since it requires real API keys not present in all environments. ISSUES CLOSED: #740	2026-03-12 21:11:37 +00:00
Brent E. Edwards	5e625b22e1	fix(test): convert M1-M6 E2E suites to real subprocess CLI invocations (closes #658) ... Replace CliRunner + unittest.mock.patch with subprocess.run for all 21 CLI-facing test functions across the M1-M6 E2E verification helpers. Application code fixes: - action.py: _get_lifecycle_service() uses container.plan_lifecycle_service() - plan.py: _get_lifecycle_service() uses container.plan_lifecycle_service() - plan.py: three container.resolve(DecisionService) → container.decision_service() Test infrastructure: - New robot/helper_e2e_common.py with shared subprocess utilities (run_cli, setup_workspace with DB migrations, cleanup_workspace) - M1-M4, M6 helpers refactored to use run_cli() with real SQLite DB - M5 unchanged (0 CLI tests, all domain-level) - TDD detection updated to recognise run_cli() as subprocess invocation - Remove @tdd_expected_fail from TDD feature + robot tags - Update 8 Behave step files that mocked container.resolve() to use container.decision_service() / container.plan_lifecycle_service()	2026-03-12 20:42:14 +00:00