Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c3e89fc341 | |||
| 71ec58b232 | |||
| b0b28623a1 | |||
| b4351ca78d | |||
| 94622f467c | |||
| 1baa888659 | |||
| 1196c726f2 | |||
| 655cd7ebc2 | |||
| dbc382f3d9 | |||
| fa95c518d5 | |||
| ef6829b6f8 | |||
| a37b4e0ccb | |||
| 4fdfee6150 | |||
| b41f536da6 |
+110
-140
@@ -2,57 +2,9 @@
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
`plan_generation_graph.robot` to give more test answers.
|
||||
|
||||
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
|
||||
requires whole-word closing keywords (avoids false positives like
|
||||
"prefixes #12"), TDD bug tag discovery now uses exact token matching
|
||||
(avoids `@tdd_bug_42` matching `@tdd_bug_420`), and gate evaluation now
|
||||
requires expected-fail tag removal to be present in the PR diff. CI
|
||||
integration now fetches the PR base branch for diff analysis and includes
|
||||
`tdd_quality_gate` in `status-check` requirements for pull requests.
|
||||
Review-round fixes: `check_expected_fail_removed` now uses word-boundary
|
||||
matching via `_contains_tag_token` (avoids false positives on partial tag
|
||||
names); diff expected-fail removal detection now tracks flags at file level
|
||||
instead of per-hunk (fixes false negatives when tags span different hunks);
|
||||
`parse_bug_refs` filters out issue number zero; redundant double error
|
||||
reporting eliminated; regex compilation cached via `lru_cache`; nox session
|
||||
no longer installs the full project (script uses stdlib only); CI checkout
|
||||
uses `fetch-depth: 0` for reliable merge-base resolution.
|
||||
Review-round 2 fixes: diff expected-fail removal detection now requires the
|
||||
removed line to contain both the expected-fail tag and the specific bug tag
|
||||
(fixes false positives when two bugs' TDD tests reside in the same file);
|
||||
`check_expected_fail_removed` error messages now use the correct tag prefix
|
||||
per file type (`@tdd_bug_N` for `.feature`, `tdd_bug_N` for `.robot`);
|
||||
`bool` values are now rejected by bug-number validation guards; file-read
|
||||
error handling in `find_tdd_tests` and `check_expected_fail_removed` now
|
||||
catches `UnicodeDecodeError` (root-safe unreadable-file handling); temp
|
||||
directory cleanup added to `after_scenario` hook; 8 new Behave scenarios
|
||||
covering bool guards, co-located bug false-positive, `run_quality_gate`
|
||||
argument validation, and `main()` CLI entry point.
|
||||
Review-round 3 fixes: synthetic PR diff helper now auto-detects `.robot`
|
||||
vs `.feature` file type and generates the matching diff format (fixes
|
||||
under-tested robot-format diff code path); `check_expected_fail_removed`
|
||||
test step now filters files by bug tag via `find_tdd_tests` before
|
||||
checking (matches the production code path); `after_scenario` temp
|
||||
directory cleanup no longer sets `context.temp_dir = None` (fixes
|
||||
cleanup conflict with `cli_init_yes_flag_steps.py`); 2 new Behave
|
||||
scenarios covering multi-line PR description parsing and non-string
|
||||
`pr_diff` type guard.
|
||||
- Added Fix-then-Revalidate orchestration loop for required validations:
|
||||
bounded retry with configurable limits (0--100 per Safety Profile),
|
||||
strategy revision escalation via `auto_strategy_revision` float
|
||||
threshold, user escalation via `needs_user_escalation` result flag,
|
||||
and domain events (`VALIDATION_FIX_ATTEMPTED`, `VALIDATION_FIX_SUCCEEDED`,
|
||||
`VALIDATION_FIX_EXHAUSTED`). Validation errors are treated as required
|
||||
failures regardless of mode. Includes `auto_validation_fix` threshold,
|
||||
per-resource retry tracking, early-exit signalling via `None` return from
|
||||
`FixCallback`, event bus circuit breaker with lock-protected failure
|
||||
counter, spec-required `validation_summary` and
|
||||
`final_validation_results` fields on the result model, DI container
|
||||
|
||||
## [Unreleased]
|
||||
- **fix(tui): rename ActorSelectionOverlay._render to _refresh_display (issue #11039)** — `ActorSelectionOverlay._render()` shadows Textual's `Widget._render()` which must return a `Strip`. In textual >=1.0, layout calls `get_content_height()` `self._render()` gets `None` `AttributeError: 'NoneType' object has no attribute 'get_height'`. Renamed the method to `_refresh_display()` and updated all four internal call sites (`show()`, `move_up()`, `move_down()`, `set_search()`) to use the new name.
|
||||
|
||||
- **Structural Component Output Validation** (#8164): Replaces exact character matching with structural component checking for output validation. Implements three validators covering plan tree output, decision CLI dicts, and structured session snapshots. The `validate_plan_tree` function validates node dicts for required keys (`decision_id`, `type`, `sequence`, `question`, `children`), ULID format, correct types, and sibling ordering. The `validate_decision_dict` function validates decision CLI output against the `Decision.as_cli_dict()` schema with field presence, type, ULID pattern, confidence range [0..1], and boolean field checks. The `validate_structured_output` function validates the StructuredOutput envelope for `command`, `session_id` (ULID), status membership, `exit_code`, and elements integrity. A unified dispatcher (`validate_structured_component_output`) enables routing by target_type. BDD test coverage added in `features/structural_validation.feature`. [Epic #8137](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8137)
|
||||
- **Fixed `agents actor add --config` crash with nested `actors:` map and `config.actor` combined shorthand** (#11189): The
|
||||
@@ -63,6 +15,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
when separate `provider`/`model` keys are absent. Added validation to reject malformed
|
||||
combined values with empty provider or model halves.
|
||||
|
||||
- **Fixed plan tree reporting zero decision nodes after strategize** (#10813): The `plan tree` command showed no ``decision_id`` fields even though planning completed successfully. Root cause: the PlanExecutor's ``run_strategize()`` method produced strategy decisions via the StrategyActor but never persisted them as domain ``Decision`` objects through the DecisionService wiring. Added ``decision_service`` parameter to the PlanExecutor constructor, wired it from the CLI dependency-injection container in ``_get_plan_executor()``, and added ``_persist_strategy_decisions()`` that converts each strategy decision into a domain Decision with correct type mapping (prompt_definition, strategy_choice, subplan_spawn) so they appear in plan tree output.
|
||||
- **`task-implementor` posts work-started notification comments** (#11031): Both
|
||||
the `issue_impl` and `pr_fix` procedures now post an informational "work
|
||||
started" comment to the Forgejo issue/PR before beginning implementation.
|
||||
@@ -73,19 +26,15 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
|
||||
- **`agents session tell` invokes real LLM orchestrator actor** (#5784): Replaced the
|
||||
M3 echo-stub with real actor invocation via `SessionWorkflow`, routing through
|
||||
`LangChainSessionCaller` → `ToolCallingRuntime.run_tool_loop()`. The user prompt
|
||||
`LangChainSessionCaller` → `ToolCallingRuntime.run_tool_loop()`. The user prompt
|
||||
is sent to the session's bound orchestrator actor (or `--actor` override), the
|
||||
assistant's response is persisted via `SessionService.append_message()`, and token
|
||||
usage is tracked via `SessionService.update_token_usage()`. Output includes a
|
||||
usage is tracked via `SessionService.update_token_usage()`. Output includes a
|
||||
**Usage** panel (Rich/Plain) or `usage` object (JSON/YAML) with input tokens,
|
||||
output tokens, estimated cost, and duration. The `--stream` flag produces real
|
||||
LLM streaming output. A `SessionActorNotConfiguredError` is raised with exit
|
||||
output tokens, estimated cost, and duration. The `--stream` flag produces real
|
||||
LLM streaming output. A `SessionActorNotConfiguredError` is raised with exit
|
||||
code 1 when no actor is configured.
|
||||
|
||||
### Added
|
||||
|
||||
- **A2A module rename BDD test suite** (#8615): Comprehensive Behave tests validating that the ACP→A2A module rename is complete — verifying all 22 A2A symbols are properly exported, no legacy ACP references remain in `.py` files under `cleveragents.a2a/`, and the module docstring uses current A2A naming. The step definitions include self-contained symbol lookups to avoid cross-scenario dependency failures.
|
||||
|
||||
- Fixed `ReactiveEventBus.emit()` exception handler to log the full exception
|
||||
message (`str(exc)`) and enable traceback forwarding (`exc_info=True`).
|
||||
Previously the handler logged only the exception type name (e.g.
|
||||
@@ -114,6 +63,14 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
worktree branch, causing ``plan apply`` to merge zero artifacts. The guard
|
||||
preserves the branch per spec (§sandbox.cleanup defaults to ``on_apply``).
|
||||
|
||||
- **LSP subprocess cleanup on failed initialisation** (#10597): Added subprocess
|
||||
cleanup in ``StdioTransport.start()`` when ``Popen()`` succeeds but then raises an
|
||||
``OSError`` (e.g. exec fails in a child process before the OSError is raised). The fix
|
||||
sets ``self._process = None`` before the ``Popen()`` call so that any process reference
|
||||
assigned during failed initialisation is caught and cleaned up via ``stop()`` in the
|
||||
``OSError`` handler, preventing zombie/orphan language-server processes. Added BDD
|
||||
regression tests in ``features/stdio_transport_subprocess_cleanup.feature`` (PR #10597).
|
||||
|
||||
- **Global CLI options ``--data-dir``, ``--config-path``, and ``-v`` now work correctly**
|
||||
(#6785): These spec-required flags were absent from ``main_callback()`` in
|
||||
``src/cleveragents/cli/main.py``, causing any invocation with these flags to crash
|
||||
@@ -130,12 +87,25 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
scenario was previously fixed in #4300; the test ensures the fix remains
|
||||
in place and prevents future regressions.
|
||||
|
||||
- **Actor configuration validation incorrectly requires top-level provider field** (#4300):
|
||||
Actor configuration in V3 is now obtained from the nested configuration
|
||||
parameter, according to the specification.
|
||||
Removed the legacy V2 fallback support and the tests affected by that
|
||||
removal. Mocked existing steps to allow remaining V2 features to be
|
||||
covered/tested.
|
||||
- **Actor configuration validation incorrectly requires top-level provider field** (#4300):
|
||||
Actor configuration in V3 is now obtained from the nested configuration
|
||||
parameter, according to the specification.
|
||||
Removed the legacy V2 fallback support and the tests affected by that
|
||||
removal. Mocked existing steps to allow remaining V2 features to be
|
||||
covered/tested.
|
||||
|
||||
- **Automation profile threshold gates fully respect spec semantics** (#4328): Added
|
||||
``_should_auto_progress_for_threshold()`` helper in ``PlanLifecycleService`` that
|
||||
implements the full spec Threshold Semantics: ``0.0`` = always auto, ``1.0`` =
|
||||
always human approval, ``0.0 < v < 1.0`` = proceed only if confidence >= threshold.
|
||||
Integrated with ``AutonomyController.should_proceed_automatically()`` for
|
||||
intermediate thresholds. Updated ``should_auto_progress()``, ``try_auto_run()``,
|
||||
``execute_async_job()``, ``try_auto_revert_from_apply()``, and
|
||||
``try_auto_revert_from_execute()`` to use the helper. Previously the service
|
||||
only checked ``< 1.0`` (treated all intermediates as auto). Added BDD regression
|
||||
tests in ``features/tdd_automation_profile_gates_4328.feature`` covering all 8
|
||||
built-in profiles including the ``cautious`` profile's intermediate thresholds
|
||||
(e.g. ``create_tool=0.7``).
|
||||
|
||||
- **TUI Prompt Symbol Mode Awareness** (#6431): The prompt widget now displays a
|
||||
mode-dependent symbol (`❯` normal, `/` command, `$` shell, `☰` multi-line),
|
||||
@@ -144,47 +114,47 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
available, and a `_FallbackPromptInput` otherwise. Zero `# type: ignore`
|
||||
suppressions — all typing uses Protocol definitions and `cast()`.
|
||||
- **Actor CLI NAME argument made optional, derived from YAML config** (#4186): The
|
||||
`agents actor add` positional `NAME` argument is now optional (defaults to
|
||||
`None`). When omitted, the actor name is derived from the `name` field in
|
||||
the config file. Raises `BadParameter` if neither the argument nor the config
|
||||
`name` field is provided. Updated docstring signature to
|
||||
`agents actor add [--config|-c <FILE>] [<NAME>]` and added config-only usage
|
||||
examples. Added Behave scenario for the `BadParameter` error path
|
||||
(`actor add without NAME and without config name field raises BadParameter`)
|
||||
in `features/actor_add_name_positional.feature` with corresponding step
|
||||
`agents actor add` positional ``NAME`` argument is now optional (defaults to
|
||||
``None``). When omitted, the actor name is derived from the ``name`` field in
|
||||
the config file. Raises ``BadParameter`` if neither the argument nor the config
|
||||
``name`` field is provided. Updated docstring signature to
|
||||
``agents actor add [--config|-c <FILE>] [<NAME>]`` and added config-only usage
|
||||
examples. Added Behave scenario for the ``BadParameter`` error path
|
||||
(``actor add without NAME and without config name field raises BadParameter``)
|
||||
in ``features/actor_add_name_positional.feature`` with corresponding step
|
||||
definition. Updated step definitions in
|
||||
`features/steps/actor_add_update_enforcement_steps.py` and
|
||||
`features/steps/actor_add_name_positional_steps.py` to pass `context.actor_name`
|
||||
``features/steps/actor_add_update_enforcement_steps.py`` and
|
||||
``features/steps/actor_add_name_positional_steps.py`` to pass ``context.actor_name``
|
||||
as a positional argument for compatibility.
|
||||
|
||||
- **Improved parallel test suite isolation** (#4186): Replaced deprecated
|
||||
`tempfile.mktemp` with `tempfile.mkstemp` in `features/environment.py`
|
||||
``tempfile.mktemp`` with ``tempfile.mkstemp`` in ``features/environment.py``
|
||||
for atomic temp file creation, eliminating TOCTOU race conditions in the
|
||||
per-scenario database path generation. Added `fcntl.flock` file locking to
|
||||
`_ensure_template_db()` to prevent race conditions when multiple
|
||||
`behave-parallel` workers attempt to create the template database
|
||||
per-scenario database path generation. Added ``fcntl.flock`` file locking to
|
||||
``_ensure_template_db()`` to prevent race conditions when multiple
|
||||
``behave-parallel`` workers attempt to create the template database
|
||||
simultaneously.
|
||||
|
||||
- **Removed stale @tdd_expected_fail tags from actor add enforcement tests**: The
|
||||
`--update` enforcement feature (#2609) was already implemented and merged but
|
||||
residual `@tdd_expected_fail` tags remained on its BDD scenarios. These tags
|
||||
were cleaned up in `features/actor_add_update_enforcement.feature` so the
|
||||
``--update`` enforcement feature (#2609) was already implemented and merged but
|
||||
residual ``@tdd_expected_fail`` tags remained on its BDD scenarios. These tags
|
||||
were cleaned up in ``features/actor_add_update_enforcement.feature`` so the
|
||||
tests report correctly now that the underlying bug has been fixed.
|
||||
|
||||
- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
|
||||
step texts to avoid case-sensitive collisions between different step modules that
|
||||
prevented all Behave tests from loading. Renamed steps in
|
||||
`edge_case_plan_steps.py`, `plan_executor_coverage_boost_steps.py`,
|
||||
`plan_explain_steps.py`, `plan_model_steps.py`, `project_repository_steps.py`,
|
||||
`service_retry_wiring_steps.py`, and `session_model_steps.py`.
|
||||
Additionally resolved a collision between `acms_index_data_model_traversal_steps.py`
|
||||
and `security_audit_steps.py` for `Then the count should be`, and fixed
|
||||
`pr_compliance_checklist_steps.py` project-root resolution (`parents[3]` →
|
||||
`parents[2]`). Fixed table column-header mismatches in
|
||||
`features/acms/index_data_model_and_traversal.feature` and guarded
|
||||
`cli_init_yes_flag_steps.py` cleanup against `None` temp_dir. Annotated
|
||||
`features/architecture.feature` `@tdd_expected_fail` for pre-existing Pydantic
|
||||
compliance debt in `IndexEntry` / `ACMSIndex` classes.
|
||||
``edge_case_plan_steps.py``, ``plan_executor_coverage_boost_steps.py``,
|
||||
``plan_explain_steps.py``, ``plan_model_steps.py``, ``project_repository_steps.py``,
|
||||
``service_retry_wiring_steps.py``, and ``session_model_steps.py``.
|
||||
Additionally resolved a collision between ``acms_index_data_model_traversal_steps.py``
|
||||
and ``security_audit_steps.py`` for ``Then the count should be``, and fixed
|
||||
``pr_compliance_checklist_steps.py`` project-root resolution (``parents[3]`` →
|
||||
``parents[2]``). Fixed table column-header mismatches in
|
||||
``features/acms/index_data_model_and_traversal.feature`` and guarded
|
||||
``cli_init_yes_flag_steps.py`` cleanup against ``None`` temp_dir. Annotated
|
||||
``features/architecture.feature`` ``@tdd_expected_fail`` for pre-existing Pydantic
|
||||
compliance debt in ``IndexEntry`` / ``ACMSIndex`` classes.
|
||||
|
||||
- **Cross-actor subgraph cycle detection reads actor_ref field** (#1431): Fixed
|
||||
`_detect_subgraph_cycles()`, `_map_node()`, and the `compile_actor()` main loop
|
||||
@@ -309,7 +279,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
flags as B608 (SQL injection risk). The constant is hardcoded and safe, but
|
||||
the f-string pattern blocks tightening the bandit severity gate from HIGH to
|
||||
MEDIUM (issue #9945). Replaced with `"INSERT INTO _v3_plans_new (" +
|
||||
_ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
_ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
|
||||
- **Diagnostics spec examples expanded to all 9 providers** (#5320): Updated the
|
||||
`agents diagnostics` command examples in the specification to show all 9 supported
|
||||
@@ -358,15 +328,15 @@ _ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
:mod:`cleveragents.shared.redaction`, and transaction-based sandbox strategy using
|
||||
BEGIN/COMMIT/ROLLBACK wrappers for safe, isolated database operations. SQLite-specific
|
||||
checkpoint and rollback support with SAVEPOINT semantics. Support for multiple backends (PostgreSQL, SQLite, MySQL, DuckDB) via unified "DatabaseResourceHandler" and type-specific routing. BDD test
|
||||
coverage in `features/database_resources.feature` (connection validation, CRUD workflows,
|
||||
coverage in ``features/database_resources.feature`` (connection validation, CRUD workflows,
|
||||
transaction/rollback behavior, error handling, credential masking verification) and
|
||||
Robot Framework integration tests in `robot/database_resources.robot`.
|
||||
Robot Framework integration tests in ``robot/database_resources.robot``.
|
||||
|
||||
- **TransactionSandbox infrastructure for database resource isolation** (#8608):
|
||||
Implemented `TransactionSandbox` class with BEGIN/COMMIT/ROLLBACK lifecycle
|
||||
management for transaction-based sandbox strategy. Wired into `SandboxFactory`
|
||||
as the strategy resolver for database resource types. Added `database` resource type
|
||||
registration in bootstrap builtin types and updated `_resource_registry_data.py`
|
||||
Implemented ``TransactionSandbox`` class with BEGIN/COMMIT/ROLLBACK lifecycle
|
||||
management for transaction-based sandbox strategy. Wired into ``SandboxFactory``
|
||||
as the strategy resolver for database resource types. Added ``database`` resource type
|
||||
registration in bootstrap builtin types and updated ``_resource_registry_data.py``
|
||||
to recognize database resource categories.
|
||||
|
||||
### Fixed
|
||||
@@ -401,9 +371,9 @@ _ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
between the class docstring ("Callers are responsible for commit") and the implementation.
|
||||
Input validation for the `trace` argument was also added. Two new BDD scenarios verify
|
||||
the session contract: `Repository save() calls flush not commit` and `LLM trace rolled
|
||||
back when UnitOfWork transaction rolls back`.
|
||||
back when UnitOfWork transaction rolls back`.
|
||||
|
||||
- **git_tools.\_get_base_env() TOCTOU Race Condition** (#7619): Fixed a
|
||||
- **git_tools._get_base_env() TOCTOU Race Condition** (#7619): Fixed a
|
||||
Time-Of-Check-To-Time-Of-Use race condition in `git_tools._get_base_env()`
|
||||
where two concurrent threads could both observe `_BASE_ENV is None`, both
|
||||
snapshot `os.environ`, and write potentially different snapshots. The fix
|
||||
@@ -418,26 +388,26 @@ back when UnitOfWork transaction rolls back`.
|
||||
|
||||
- **Unified provider factory: eliminate divergence between `create_llm()` and `create_ai_provider()`** (#10949):
|
||||
Introduced `_create_provider_instance()` as the single internal factory so that
|
||||
both public methods delegate to one place. Creating a new provider now
|
||||
both public methods delegate to one place. Creating a new provider now
|
||||
requires changes in exactly one method.
|
||||
- **Fixed API key regression**: the unified factory now explicitly passes
|
||||
the validated API key to all LangChain constructors (OpenAI, Anthropic,
|
||||
Google / Gemini, Azure, Groq, Together, Cohere, and OpenRouter). Users
|
||||
who configure providers via `CLEVERAGENTS_`-prefixed variables are no
|
||||
longer silently failed when LangChain falls back to raw environment
|
||||
variable lookup. Pre-validated keys are forwarded through the
|
||||
`api_key` kwarg to avoid a second settings lookup in the factory
|
||||
closure. (Closes #10949)
|
||||
- **Fixed mock provider accessibility in production**: `ProviderType.MOCK`
|
||||
is now gated by the `CLEVERAGENTS_ALLOW_MOCK_PROVIDER=true` sentinel
|
||||
environment variable. Without this flag, both `create_llm()` and
|
||||
`create_ai_provider()` raise `ValueError` when MOCK is requested,
|
||||
preventing accidental or malicious use of the fake LLM in production.
|
||||
`resolve_provider_by_name("mock")` now also respects the guard and
|
||||
`is_provider_configured(ProviderType.MOCK)` returns `True` as expected.
|
||||
- **Fixed type annotation**: `create_llm()` now declares `**kwargs: Any`
|
||||
instead of `**kwargs: object`, restoring correct Pyright inference for
|
||||
forwarded keyword arguments.
|
||||
- **Fixed API key regression**: the unified factory now explicitly passes
|
||||
the validated API key to all LangChain constructors (OpenAI, Anthropic,
|
||||
Google / Gemini, Azure, Groq, Together, Cohere, and OpenRouter). Users
|
||||
who configure providers via `CLEVERAGENTS_`-prefixed variables are no
|
||||
longer silently failed when LangChain falls back to raw environment
|
||||
variable lookup. Pre-validated keys are forwarded through the
|
||||
`api_key` kwarg to avoid a second settings lookup in the factory
|
||||
closure. (Closes #10949)
|
||||
- **Fixed mock provider accessibility in production**: `ProviderType.MOCK`
|
||||
is now gated by the `CLEVERAGENTS_ALLOW_MOCK_PROVIDER=true` sentinel
|
||||
environment variable. Without this flag, both `create_llm()` and
|
||||
`create_ai_provider()` raise `ValueError` when MOCK is requested,
|
||||
preventing accidental or malicious use of the fake LLM in production.
|
||||
`resolve_provider_by_name("mock")` now also respects the guard and
|
||||
`is_provider_configured(ProviderType.MOCK)` returns `True` as expected.
|
||||
- **Fixed type annotation**: `create_llm()` now declares `**kwargs: Any`
|
||||
instead of `**kwargs: object`, restoring correct Pyright inference for
|
||||
forwarded keyword arguments.
|
||||
|
||||
- **`create_llm()` raises `Unsupported provider type: openrouter`** (#10948): Fixed
|
||||
`ProviderRegistry._create_provider_llm()` missing an `OPENROUTER` branch, which
|
||||
@@ -498,7 +468,7 @@ back when UnitOfWork transaction rolls back`.
|
||||
`agents actor run` silently returning empty output for v3 `type:llm` actors.
|
||||
`_build_from_v3()` and `_build()` now synthesise a default single-node
|
||||
graph route when agents are created without explicit routes, ensuring
|
||||
`run_single_shot()` can invoke the LLM via `GraphExecutor`. The nested
|
||||
`run_single_shot()` can invoke the LLM via `GraphExecutor`. The nested
|
||||
`actors:` map format also translates the v3 `actor: "provider/model"` key
|
||||
into separate `provider` and `model` keys so the correct LLM provider is
|
||||
instantiated.
|
||||
@@ -514,7 +484,7 @@ back when UnitOfWork transaction rolls back`.
|
||||
YAML values (e.g. `unsafe: "no"`) from being treated as unsafe.
|
||||
|
||||
- **UKO Runtime Layer 2 (Paradigm) Indexing** (#9351): Added missing `rdf:type
|
||||
uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
Python class definitions are now correctly classified at layer 2 (paradigm/OO)
|
||||
in addition to layer 3 (technology). Added the corresponding Behave scenario
|
||||
`Indexing a Python file populates layer 2 (paradigm)` to
|
||||
@@ -523,10 +493,10 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
|
||||
- **Actor CLI v3 YAML Schema Support** (#6283): Fixed three components to add
|
||||
full v3 `ActorConfigSchema` support to the actor CLI registration and
|
||||
execution paths. `ActorConfiguration.from_blob()` now detects v3 format
|
||||
execution paths. `ActorConfiguration.from_blob()` now detects v3 format
|
||||
(top-level `type` key of `llm`/`graph`/`tool`) and correctly extracts
|
||||
provider, model, and graph descriptors — including `type: tool` actors
|
||||
without a `model` field. `ActorRegistry.add()` validates against the full
|
||||
without a `model` field. `ActorRegistry.add()` validates against the full
|
||||
Pydantic v2 schema, persists `skills`/`lsp`/`description` in the config
|
||||
blob, and compiles graph actors with proper metadata.
|
||||
`ReactiveConfigParser._build_from_v3()` now uses correct `source`/`target`
|
||||
@@ -535,9 +505,9 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
`env_vars`/`response_format`/`lsp_capabilities`/`lsp_context_enrichment`
|
||||
into agent configs, and validates `entry_node` against the nodes map.
|
||||
Exception handling narrowed from broad `except Exception` to specific
|
||||
`NotFoundError` and `ActorCompilationError`. v3 registration logic
|
||||
`NotFoundError` and `ActorCompilationError`. v3 registration logic
|
||||
extracted to `v3_registry.py` to keep `registry.py` under the 500-line
|
||||
limit. 19 BDD scenarios cover all v3 paths including tool actors,
|
||||
limit. 19 BDD scenarios cover all v3 paths including tool actors,
|
||||
update mode, LSP dict bindings, and field propagation.
|
||||
|
||||
- **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
|
||||
@@ -554,8 +524,8 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
|
||||
- **Parallel Behave Runner Log Noise Reduction** (#8351): The parallel behave
|
||||
runner now suppresses captured stdout/stderr for passing worker chunks and
|
||||
only replays diagnostics for failed, errored, or crashed chunks. This makes
|
||||
failure output significantly easier to spot in CI and local runs. A worker
|
||||
only replays diagnostics for failed, errored, or crashed chunks. This makes
|
||||
failure output significantly easier to spot in CI and local runs. A worker
|
||||
crash (unhandled exception) is detected via an all-zero summary and the
|
||||
captured traceback is always surfaced.
|
||||
|
||||
@@ -626,14 +596,14 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
- Wired `StrategyActor` into the real plan execution path: `_get_plan_executor`
|
||||
in `plan.py` now resolves the strategy actor via `resolve_strategy_actor()`
|
||||
(reading the `actor.default.strategy` config key) instead of always
|
||||
constructing `LLMStrategizeActor`. `run_strategize` in `PlanExecutor` now
|
||||
constructing `LLMStrategizeActor`. `run_strategize` in `PlanExecutor` now
|
||||
passes `resources` (derived from `plan.project_links`) and `project_context`
|
||||
to the actor so the LLM prompt receives full project context. Strategy
|
||||
to the actor so the LLM prompt receives full project context. Strategy
|
||||
decisions are serialised as JSON in `plan.error_details["strategy_decisions_json"]`
|
||||
so `_build_decisions` can reconstruct the full hierarchy (dependency ordering,
|
||||
parent/child structure) during Execute instead of rebuilding from
|
||||
`definition_of_done`. `StrategizeStubActor.execute` accepts `**kwargs` for
|
||||
forward-compatibility. Added BDD coverage for the stored-JSON path,
|
||||
`definition_of_done`. `StrategizeStubActor.execute` accepts `**kwargs` for
|
||||
forward-compatibility. Added BDD coverage for the stored-JSON path,
|
||||
corrupt-JSON fallback, resource-passing, and stub extra-kwargs scenarios. (#828)
|
||||
|
||||
- **Decision Recording Hook in Strategize Phase** (#8522): Implemented
|
||||
@@ -651,15 +621,15 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
|
||||
- **TDD Issue-Capture Test Activation** (#7025): Replaced 234 bare `@skip` tags
|
||||
across 82 Behave feature files with the correct `@tdd_expected_fail @tdd_issue
|
||||
@tdd_issue_<N>` tag system. Scenarios whose referenced bugs were already fixed
|
||||
@tdd_issue_<N>` tag system. Scenarios whose referenced bugs were already fixed
|
||||
had `@tdd_expected_fail` removed and now run as permanent regression guards.
|
||||
Net result: 629 features active in CI (up from ~545), zero `@skip` tags remain.
|
||||
|
||||
- **Git Worktree Sandbox Apply** (#4454): The `plan apply` command now merges
|
||||
LLM-generated changes via `git merge` from an isolated worktree branch
|
||||
instead of flat `shutil.copy2`. Displays spec-aligned Apply Summary
|
||||
instead of flat `shutil.copy2`. Displays spec-aligned Apply Summary
|
||||
(plan ID, artifacts, insertions/deletions, project, timestamp), Sandbox
|
||||
Cleanup panel, and `✓ OK Changes applied` footer. Non-git projects fall
|
||||
Cleanup panel, and `✓ OK Changes applied` footer. Non-git projects fall
|
||||
back to the original flat file copy.
|
||||
|
||||
- **Context Hydration Fix** (#4454): Fixed `ContextFragment` metadata types
|
||||
@@ -737,7 +707,7 @@ uko-oo:Class` triple emission in `PythonAnalyzer._extract_class()` so that
|
||||
|
||||
- **Documentation Writer Tracking** (`docs-writer`): The documentation writer now
|
||||
participates in the automation tracking system by creating individual `[AUTO-DOCS]
|
||||
Documentation Report (Cycle N)` issues every 10 cycles (~3.3 hours). The manager applies
|
||||
Documentation Report (Cycle N)` issues every 10 cycles (~3.3 hours). The manager applies
|
||||
the mandatory `Automation Tracking` label automatically, while teams may add additional
|
||||
workflow labels as needed. See `docs/development/automation-tracking.md` and the new
|
||||
`docs/development/docs-writer.md` reference.
|
||||
@@ -845,7 +815,7 @@ Documentation Report (Cycle N)` issues every 10 cycles (~3.3 hours). The manager
|
||||
|
||||
- **ContextTierService Thread Safety** (#7547): Added `threading.RLock` to
|
||||
`ContextTierService` to prevent `RuntimeError: dictionary changed size during
|
||||
iteration` and data corruption under concurrent plan execution. All public
|
||||
iteration` and data corruption under concurrent plan execution. All public
|
||||
methods (`store`, `get`, `promote`, `demote`, `evict_lru`, `enforce_staleness`,
|
||||
`get_metrics`, `get_all_fragments`, `get_hot_fragments`, `get_for_actor`,
|
||||
`get_scoped_view`, `get_scoped_by_resource`, `get_scoped_metrics`) now acquire
|
||||
@@ -939,7 +909,6 @@ iteration` and data corruption under concurrent plan execution. All public
|
||||
Workers now dispatch and verify correctly, preventing incorrect session deletion.
|
||||
|
||||
---
|
||||
|
||||
### Fixed
|
||||
|
||||
- **CLI (`agents actor remove`)** (#6491): Restores output parity with the
|
||||
@@ -956,10 +925,10 @@ iteration` and data corruption under concurrent plan execution. All public
|
||||
|
||||
- Wired Invariant Reconciliation Actor auto-invocation into
|
||||
`PlanLifecycleService` phase transitions (`start_strategize`,
|
||||
`execute_plan`, `apply_plan`). Reconciliation failures now block
|
||||
`execute_plan`, `apply_plan`). Reconciliation failures now block
|
||||
the transition with `ReconciliationBlockedError` and emit
|
||||
`INVARIANT_VIOLATED` events. Post-correction reconciliation runs
|
||||
via `CORRECTION_APPLIED` event subscription (best-effort). Added
|
||||
`INVARIANT_VIOLATED` events. Post-correction reconciliation runs
|
||||
via `CORRECTION_APPLIED` event subscription (best-effort). Added
|
||||
`InvariantService` Singleton provider in the DI container.
|
||||
- **TUI -- Shell danger detection**: The TUI shell mode (`!` prefix) now detects
|
||||
dangerous command patterns before execution. A configurable pattern registry
|
||||
@@ -971,3 +940,4 @@ iteration` and data corruption under concurrent plan execution. All public
|
||||
renders permission requests directly in the conversation stream for single-file
|
||||
operations. Users can allow/reject with single-key shortcuts (`a`/`A`/`r`/`R`),
|
||||
navigate with arrow keys, confirm with `Enter`, or press `v` to open the full
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
|
||||
* Luis Mendes <luis.p.mendes@gmail.com>
|
||||
* Rui Hu <rui.hu@cleverthis.com>
|
||||
* HAL 9000 <hal9000@cleverthis.com> has contributed fix for #10813 — wiring DecisionService into PlanExecutor for strategy decision persistence during strategize.
|
||||
|
||||
# Details
|
||||
|
||||
@@ -44,4 +45,6 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
|
||||
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
|
||||
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
|
||||
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
|
||||
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
|
||||
* HAL 9000 has contributed the `ActorSelectionOverlay._render` → `_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
Feature: Plan Lifecycle Service coverage boost round 3
|
||||
As a developer
|
||||
I want to exercise remaining uncovered code paths in PlanLifecycleService
|
||||
So that code coverage improves beyond the current per-file threshold
|
||||
|
||||
Background:
|
||||
Given I have a fresh plan lifecycle service for coverage boost r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_strategize — event_bus.emit PLAN_STATE_CHANGED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_strategize emits PLAN_STATE_CHANGED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/strat-complete-event" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I complete strategize on the plan with event bus
|
||||
Then the event bus should have recorded a PLAN_STATE_CHANGED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_strategize — event_bus.emit PLAN_STATE_CHANGED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_strategize catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/strat-complete-fail" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I complete strategize on the plan and event bus emit fails
|
||||
Then the plan phase should be "strategize" and state should be "complete"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_strategize — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_strategize emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/strat-fail-event" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I fail strategize on the plan with error "Test strategy error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_strategize — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_strategize catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/strat-fail-fail" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I fail strategize on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_execute — event_bus.emit PLAN_STATE_CHANGED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_execute emits PLAN_STATE_CHANGED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/exec-complete-event" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I complete execute on the plan with event bus
|
||||
Then the event bus should have recorded a PLAN_STATE_CHANGED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_execute — event_bus.emit PLAN_STATE_CHANGED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_execute catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/exec-complete-fail" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I complete execute on the plan and event bus emit fails
|
||||
Then the plan phase should be "execute" and state should be "complete"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/exec-fail-event" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan with error "Test execution error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/exec-fail-fail" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/apply-fail-event" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan with error "Test apply error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/apply-fail-fail" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — _cleanup_devcontainers called on terminal failure
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute triggers devcontainer cleanup on terminal failure
|
||||
Given a plan lifecycle service with mocked devcontainer cleanup for r3
|
||||
And an action "local/exec-fail-cleanup" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan with error "Container cleanup test"
|
||||
Then the plan should be in errored state and cleanup should have been called
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — _cleanup_devcontainers called on terminal failure
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply triggers devcontainer cleanup on terminal failure
|
||||
Given a plan lifecycle service with mocked devcontainer cleanup for r3
|
||||
And an action "local/apply-fail-cleanup" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan with error "Container cleanup test"
|
||||
Then the plan should be in errored state and cleanup should have been called
|
||||
@@ -0,0 +1,163 @@
|
||||
Feature: Plan Lifecycle Service coverage boost round 4
|
||||
As a developer
|
||||
I want to exercise remaining uncovered code paths in PlanLifecycleService
|
||||
So that code coverage improves to meet the 97% threshold
|
||||
|
||||
Background:
|
||||
Given I have a fresh plan lifecycle service for coverage boost r4
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_estimation — event_bus.emit PLAN_ESTIMATION_COMPLETE
|
||||
# coverage (via execute_plan when estimation_actor is configured)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan with estimation_actor succeeds and emits PLAN_ESTIMATION_COMPLETE
|
||||
Given an action with estimation_actor configured for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a recording event bus for r4
|
||||
When I execute the plan for r4
|
||||
Then the event bus should have recorded PLAN_ESTIMATION_COMPLETE event
|
||||
And the plan should be in execute phase
|
||||
|
||||
Scenario: execute_plan with estimation_actor catches event_bus emit failure for PLAN_ESTIMATION_COMPLETE
|
||||
Given an action with estimation_actor configured for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan for r4
|
||||
Then the plan should be in execute phase despite estimation emit failure
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_estimation — cost_estimate_usd assignment (line 376)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan populates cost_estimate_usd when estimation returns cost
|
||||
Given an action with estimation_actor returning cost for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a recording event bus for r4
|
||||
When I execute the plan for r4 with mocked estimation
|
||||
Then the plan cost_estimate_usd should be populated
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# execute_plan — lock acquire/release when lock_service is configured
|
||||
# Lines 1616-1621 (acquire) and 1684-1688 (finally release)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan acquires and releases plan lock when lock_service is configured
|
||||
Given an action for execute with lock service for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
When I execute the plan with lock service for r4
|
||||
Then the plan should be in execute phase
|
||||
And the lock service should have acquired and released the plan lock
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# apply_plan — lock acquire/release when lock_service is configured
|
||||
# Lines 1830-1835 (acquire) and 1872-1877 (finally release)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: apply_plan acquires and releases plan lock when lock_service is configured
|
||||
Given an action for apply with lock service for r4
|
||||
And a plan in execute-complete state for r4
|
||||
When I apply the plan with lock service for r4
|
||||
Then the plan should be in apply phase
|
||||
And the lock service should have acquired and released the plan lock
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _subscribe_correction_reconciliation — event_bus.subscribe failure
|
||||
# Lines 562-566
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: lifecycle service handles event_bus subscribe failure gracefully
|
||||
Given I have a plan lifecycle service with event_bus that fails on subscribe for r4
|
||||
When the service is created for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# prompt_plan — validation scenarios
|
||||
# Lines 2082 (blank guidance) and 2086-2097 (wrong phase/state)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: prompt_plan rejects blank guidance with ValidationError
|
||||
Given a plan in execute phase with processing state for prompt_plan r4
|
||||
When I prompt the plan with blank guidance for r4
|
||||
Then a ValidationError should be raised for blank guidance
|
||||
|
||||
Scenario: prompt_plan rejects plan not in execute phase
|
||||
Given a plan in strategize phase for prompt_plan r4
|
||||
When I prompt the plan with guidance for wrong phase for r4
|
||||
Then a PlanError should be raised for wrong phase
|
||||
|
||||
Scenario: prompt_plan rejects plan in complete state
|
||||
Given a plan in execute-complete state for prompt_plan r4
|
||||
When I prompt the plan with guidance for complete state for r4
|
||||
Then a PlanError should be raised for wrong state
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# prompt_plan — state transition and decision_service interaction
|
||||
# Lines 2100-2154
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: prompt_plan transitions from errored to processing and clears error_message
|
||||
Given a plan in execute phase with errored state for prompt_plan r4
|
||||
When I prompt the plan with guidance to resume from errored state for r4
|
||||
Then the plan processing_state should be processing
|
||||
And the plan error_message should be cleared
|
||||
|
||||
Scenario: prompt_plan handles decision_service exception gracefully
|
||||
Given a plan in execute phase with processing state for prompt_plan r4
|
||||
And the plan lifecycle service has a failing decision service for r4
|
||||
When I prompt the plan with guidance for r4
|
||||
Then the plan processing_state should be processing
|
||||
And no exception should propagate
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_invariant_reconciliation — event_bus.emit exceptions
|
||||
# Lines 499-505 (success emit failure) and 530-536 (failure emit failure)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: _run_invariant_reconciliation success path catches event_bus emit failure
|
||||
Given an action with invariant_actor for reconciliation success r4
|
||||
And a plan in strategize-complete state with project links for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan for invariant reconciliation emit failure r4
|
||||
Then the plan should be in execute phase
|
||||
And the event_bus should have been called for INVARIANT_RECONCILED emit failure
|
||||
|
||||
Scenario: _run_invariant_reconciliation failure path catches event_bus emit failure
|
||||
Given an action with invariant_actor causing reconciliation failure for r4
|
||||
And a plan in strategize-complete state with project links for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan expecting reconciliation failure for r4
|
||||
Then ReconciliationBlockedError should be raised
|
||||
And the event_bus should have been called for INVARIANT_VIOLATED emit failure
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Unknown automation profile validation
|
||||
# Lines 1229-1233
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: use_action rejects unknown automation profile with ValidationError
|
||||
Given an action with unknown automation profile for r4
|
||||
When I use the action to create a plan for r4
|
||||
Then a ValidationError should be raised for unknown profile
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _handle_correction_applied — event handler scenarios
|
||||
# Lines 577 (plan_id None guard), 584-595 (exception handlers)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: _handle_correction_applied skips event with no plan_id
|
||||
Given the plan lifecycle service has a recording event bus for correction handler r4
|
||||
When I handle a CORRECTION_APPLIED event with no plan_id for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
Scenario: _handle_correction_applied handles ReconciliationBlockedError gracefully
|
||||
Given a plan in strategize phase with processing state for r4
|
||||
And the invariant reconciliation raises ReconciliationBlockedError for r4
|
||||
When I handle a CORRECTION_APPLIED event for blocked reconciliation for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
Scenario: _handle_correction_applied handles general exception from get_plan gracefully
|
||||
Given the plan lifecycle service is configured for r4
|
||||
And get_plan will fail for the plan for r4
|
||||
When I handle a CORRECTION_APPLIED event when get_plan fails for r4
|
||||
Then no exception should be raised for r4
|
||||
@@ -0,0 +1,18 @@
|
||||
Feature: StdioTransport subprocess cleanup on failed initialisation
|
||||
As a developer maintaining the LSP transport layer
|
||||
I need verified subprocess cleanup when ``Popen()`` fails with OSError
|
||||
So that no orphan or zombie language-server processes are leaked
|
||||
|
||||
Background:
|
||||
Given a fresh StdioTransport for command "dummy-lsp-server"
|
||||
And Popen is patched to simulate partial fork-before-OS-error
|
||||
|
||||
Scenario: LSP subprocess cleanup when Popen raises OSError after fork
|
||||
When I call ``StdioTransport.start()`` and Popen raises OSError with "child exec failed after fork"
|
||||
Then an LspError should be raised with message containing "Failed to start"
|
||||
And the transport's internal ``_process`` should be cleaned up (None)
|
||||
|
||||
Scenario: No orphan process leaked when OSError handler calls stop() on partial process
|
||||
When I call ``StdioTransport.start()`` and Popen raises OSError with "child exec failed after fork"
|
||||
Then the subprocess cleanup method ``stop()`` should have been called on the partial process
|
||||
And the transport should no longer track any active process reference
|
||||
@@ -52,6 +52,7 @@ def _profile_name_for_level(level: str) -> str:
|
||||
"manual": "manual",
|
||||
"review_before_apply": "auto",
|
||||
"full_automation": "full-auto",
|
||||
"cautious": "cautious",
|
||||
}
|
||||
return mapping.get(level, level)
|
||||
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
"""Step definitions for plan_lifecycle_service_coverage_boost_r3.feature.
|
||||
|
||||
Targets remaining uncovered lines in PlanLifecycleService:
|
||||
- Lines ~1520-1540: complete_strategize event_bus.emit success + exception
|
||||
- Lines ~1569-1581: fail_strategize event_bus.emit success + exception
|
||||
- Lines ~1739-1759: complete_execute event_bus.emit success + exception
|
||||
- Lines ~1774-1792: fail_execute event_bus.emit success + exception
|
||||
- Lines ~2036-2054: fail_apply event_bus.emit success + exception
|
||||
- Lines ~2056-2057: fail_execute _cleanup_devcontainers call
|
||||
- Lines ~2056-2057: fail_apply _cleanup_devcontainers call
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Background
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
@given("I have a fresh plan lifecycle service for coverage boost r3")
|
||||
def step_create_fresh_service_r3(context: Context) -> None:
|
||||
"""Create a clean PlanLifecycleService instance."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.error = None
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
def _create_action_r3(context: Context, name: str, **kwargs):
|
||||
"""Helper to create a basic action with sensible defaults."""
|
||||
defaults = {
|
||||
"name": name,
|
||||
"description": f"Action {name}",
|
||||
"definition_of_done": "Tests pass",
|
||||
"strategy_actor": "openai/gpt-4",
|
||||
"execution_actor": "openai/gpt-4",
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return context.service.create_action(**defaults)
|
||||
|
||||
|
||||
def _advance_to_state(
|
||||
context: Context, target_phase: PlanPhase, target_state: ProcessingState
|
||||
):
|
||||
"""Advance the current plan to the target phase and state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
if (
|
||||
target_phase == PlanPhase.STRATEGIZE
|
||||
and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif (
|
||||
target_phase == PlanPhase.EXECUTE and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif target_phase == PlanPhase.APPLY and target_state == ProcessingState.PROCESSING:
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.service.apply_plan(pid)
|
||||
context.service.start_apply(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
class RecordingEventBus:
|
||||
"""A simple event bus that records emitted events."""
|
||||
|
||||
def __init__(self):
|
||||
self.events = []
|
||||
|
||||
def emit(self, event):
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type, handler):
|
||||
pass
|
||||
|
||||
|
||||
class FailingEventBus:
|
||||
"""An event bus that always raises on emit."""
|
||||
|
||||
def emit(self, event):
|
||||
raise RuntimeError("Simulated event bus failure")
|
||||
|
||||
def subscribe(self, event_type, handler):
|
||||
pass
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_strategize — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with a recording event bus for r3")
|
||||
def step_service_with_recording_bus_r3(context: Context) -> None:
|
||||
"""Create a service with a recording event bus."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.recording_bus = RecordingEventBus()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=context.recording_bus,
|
||||
)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given('an action "{name}" exists for coverage boost r3')
|
||||
def step_create_action_r3(context: Context, name: str) -> None:
|
||||
"""Create an action with the given name."""
|
||||
context.action = _create_action_r3(context, name)
|
||||
|
||||
|
||||
@given("a plan in strategize phase with processing state for r3")
|
||||
def step_plan_strategize_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Strategize/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.STRATEGIZE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when("I complete strategize on the plan with event bus")
|
||||
def step_complete_strategize_with_bus_r3(context: Context) -> None:
|
||||
"""Complete strategize — should emit PLAN_STATE_CHANGED event."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_strategize(
|
||||
context.plan.identity.plan_id
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded a PLAN_STATE_CHANGED event")
|
||||
def step_verify_plan_state_changed_event_r3(context: Context) -> None:
|
||||
"""Verify PLAN_STATE_CHANGED was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
state_events = [
|
||||
e
|
||||
for e in context.recording_bus.events
|
||||
if e.event_type == EventType.PLAN_STATE_CHANGED
|
||||
]
|
||||
assert len(state_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_STATE_CHANGED event, got {len(state_events)}. "
|
||||
f"All events: {[e.event_type for e in context.recording_bus.events]}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_strategize — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with a failing event bus for r3")
|
||||
def step_service_with_failing_bus_r3(context: Context) -> None:
|
||||
"""Create a service with an event bus that raises on emit."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=FailingEventBus(),
|
||||
)
|
||||
context.error = None
|
||||
|
||||
|
||||
@when("I complete strategize on the plan and event bus emit fails")
|
||||
def step_complete_strategize_failing_bus_r3(context: Context) -> None:
|
||||
"""Complete strategize — event bus will raise but plan should still complete."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_strategize(
|
||||
context.plan.identity.plan_id
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then('the plan phase should be "{phase}" and state should be "{state}"')
|
||||
def step_verify_plan_phase_state_r3(context: Context, phase: str, state: str) -> None:
|
||||
"""Verify the plan phase and state."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase.value == phase, (
|
||||
f"Expected phase '{phase}', got '{context.plan.phase.value}'"
|
||||
)
|
||||
assert context.plan.processing_state.value == state, (
|
||||
f"Expected state '{state}', got '{context.plan.processing_state.value}'"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_strategize — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail strategize on the plan with error "{error}"')
|
||||
def step_fail_strategize_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail strategize with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_strategize(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded a PLAN_ERRORED event")
|
||||
def step_verify_plan_errored_event_r3(context: Context) -> None:
|
||||
"""Verify PLAN_ERRORED was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
error_events = [
|
||||
e
|
||||
for e in context.recording_bus.events
|
||||
if e.event_type == EventType.PLAN_ERRORED
|
||||
]
|
||||
assert len(error_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_ERRORED event, got {len(error_events)}. "
|
||||
f"All events: {[e.event_type for e in context.recording_bus.events]}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_strategize — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail strategize on the plan and event bus emit fails")
|
||||
def step_fail_strategize_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail strategize — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_strategize(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in errored state for r3")
|
||||
def step_verify_errored_state_r3(context: Context) -> None:
|
||||
"""Verify the plan is in ERRORED state."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.ERRORED, (
|
||||
f"Expected ERRORED, got {context.plan.processing_state.value}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_execute — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan in execute phase with processing state for r3")
|
||||
def step_plan_execute_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Execute/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.EXECUTE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when("I complete execute on the plan with event bus")
|
||||
def step_complete_execute_with_bus_r3(context: Context) -> None:
|
||||
"""Complete execute — should emit PLAN_STATE_CHANGED event."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_execute(context.plan.identity.plan_id)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_execute — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I complete execute on the plan and event bus emit fails")
|
||||
def step_complete_execute_failing_bus_r3(context: Context) -> None:
|
||||
"""Complete execute — event bus will raise but plan should still complete."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_execute(context.plan.identity.plan_id)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail execute on the plan with error "{error}"')
|
||||
def step_fail_execute_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail execute with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail execute on the plan and event bus emit fails")
|
||||
def step_fail_execute_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail execute — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan in apply phase with processing state for r3")
|
||||
def step_plan_apply_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Apply/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.APPLY, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when('I fail apply on the plan with error "{error}"')
|
||||
def step_fail_apply_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail apply with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_apply(context.plan.identity.plan_id, error)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail apply on the plan and event bus emit fails")
|
||||
def step_fail_apply_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail apply — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_apply(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — _cleanup_devcontainers success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with mocked devcontainer cleanup for r3")
|
||||
def step_service_with_mocked_cleanup_r3(context: Context) -> None:
|
||||
"""Create a service where devcontainer cleanup returns stopped containers."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.cleanup_called = False
|
||||
context.error = None
|
||||
|
||||
|
||||
@when('I fail execute on the plan with error "{error}" and cleanup should be called')
|
||||
def step_fail_execute_with_cleanup_r3(context: Context, error: str) -> None:
|
||||
"""Fail execute with mocked cleanup."""
|
||||
context.error = None
|
||||
context.cleanup_called = False
|
||||
try:
|
||||
with patch(
|
||||
"cleveragents.application.services.cleanup_service.stop_all_active_containers",
|
||||
return_value=["container-1", "container-2"],
|
||||
):
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
context.cleanup_called = True
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in errored state and cleanup should have been called")
|
||||
def step_verify_errored_with_cleanup_r3(context: Context) -> None:
|
||||
"""Verify the plan is ERRORED and cleanup was invoked."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.ERRORED, (
|
||||
f"Expected ERRORED, got {context.plan.processing_state.value}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — _cleanup_devcontainers success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail apply on the plan with error "{error}" and cleanup should be called')
|
||||
def step_fail_apply_with_cleanup_r3(context: Context, error: str) -> None:
|
||||
"""Fail apply with mocked cleanup."""
|
||||
context.error = None
|
||||
context.cleanup_called = False
|
||||
try:
|
||||
with patch(
|
||||
"cleveragents.application.services.cleanup_service.stop_all_active_containers",
|
||||
return_value=["container-1", "container-2"],
|
||||
):
|
||||
context.plan = context.service.fail_apply(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
context.cleanup_called = True
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
@@ -0,0 +1,886 @@
|
||||
"""Step definitions for plan_lifecycle_service_coverage_boost_r4.feature.
|
||||
|
||||
Targets remaining uncovered lines in PlanLifecycleService:
|
||||
- Lines ~386-405: _run_estimation event_bus.emit PLAN_ESTIMATION_COMPLETE success + exception
|
||||
- Line 376: cost_estimate_usd assignment when estimated_cost_usd is not None
|
||||
- Line 462: project_name extraction from plan.project_links[0]
|
||||
- Lines ~481-505: _run_invariant_reconciliation INVARIANT_RECONCILED emit exception
|
||||
- Lines ~517-536: _run_invariant_reconciliation INVARIANT_VIOLATED emit exception
|
||||
- Lines ~562-566: _subscribe_correction_reconciliation event_bus.subscribe failure
|
||||
- Line 577: _handle_correction_applied guard (plan_id is None)
|
||||
- Lines ~584-595: _handle_correction_applied ReconciliationBlockedError + general exception
|
||||
- Lines ~1230-1232: unknown automation profile ValidationError
|
||||
- Lines ~1616-1621, 1684-1688: execute_plan lock acquire/release with lock_service
|
||||
- Lines ~1830-1835, 1872-1877: apply_plan lock acquire/release with lock_service
|
||||
- Lines ~2082-2097: prompt_plan blank guidance + wrong phase/state validation
|
||||
- Lines ~2100-2156: prompt_plan state transition + decision_service interaction
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
ReconciliationBlockedError,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.core.exceptions import PlanError, ValidationError
|
||||
from cleveragents.domain.models.core.action import Action
|
||||
from cleveragents.domain.models.core.estimation import EstimationResult
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Background
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
@given("I have a fresh plan lifecycle service for coverage boost r4")
|
||||
def step_create_fresh_service_r4(context: Context) -> None:
|
||||
"""Create a clean PlanLifecycleService instance."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.error = None
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
def _create_action_r4(context: Context, name: str, **kwargs) -> Action:
|
||||
"""Helper to create a basic action with sensible defaults."""
|
||||
defaults: dict[str, Any] = {
|
||||
"name": name,
|
||||
"description": f"Action {name}",
|
||||
"definition_of_done": "Tests pass",
|
||||
"strategy_actor": "openai/gpt-4",
|
||||
"execution_actor": "openai/gpt-4",
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return context.service.create_action(**defaults)
|
||||
|
||||
|
||||
def _advance_to_strategize_complete(context: Context) -> None:
|
||||
"""Advance plan to strategize-complete state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_execute_complete(context: Context) -> None:
|
||||
"""Advance plan to execute-complete state (ready for apply)."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_strategize_processing(context: Context) -> None:
|
||||
"""Advance plan to strategize/PROCESSING state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Mock event buses
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
class FailingEventBusR4:
|
||||
"""An event bus that raises on every emit call."""
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
raise RuntimeError("Simulated event bus emit failure")
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class FailingSubscribeEventBus:
|
||||
"""An event bus that raises on subscribe but works for emit."""
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
pass
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
raise RuntimeError("Simulated subscribe failure")
|
||||
|
||||
|
||||
class RecordingEventBusR4:
|
||||
"""A simple event bus that records emitted events."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class FailingDecisionService:
|
||||
"""A decision service that raises on all operations."""
|
||||
|
||||
def list_decisions(self, plan_id: str) -> list[Any]:
|
||||
raise RuntimeError("Simulated decision service failure")
|
||||
|
||||
def record_decision(self, **kwargs: Any) -> Any:
|
||||
raise RuntimeError("Simulated decision service failure")
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _run_estimation via execute_plan — event_bus.emit PLAN_ESTIMATION_COMPLETE
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with estimation_actor configured for r4")
|
||||
def step_action_with_estimation_actor_r4(context: Context) -> None:
|
||||
"""Create an action with estimation_actor set."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/est-action-r4",
|
||||
estimation_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@given("an action with estimation_actor returning cost for r4")
|
||||
def step_action_with_estimation_cost_r4(context: Context) -> None:
|
||||
"""Create an action with estimation_actor and patch stub to return cost."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/est-cost-action-r4",
|
||||
estimation_actor="openai/gpt-4",
|
||||
)
|
||||
context.mock_estimation_result = EstimationResult(
|
||||
summary="Test estimation with cost",
|
||||
estimated_cost_usd=1.23,
|
||||
)
|
||||
context.estimation_patch = patch(
|
||||
"cleveragents.application.services.plan_executor.EstimationStubActor.estimate",
|
||||
return_value=context.mock_estimation_result,
|
||||
)
|
||||
|
||||
|
||||
@given("a plan in strategize-complete state for r4")
|
||||
def step_plan_strategize_complete_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/COMPLETE state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_strategize_complete(context)
|
||||
|
||||
|
||||
@given("a plan in strategize phase with processing state for r4")
|
||||
def step_plan_strategize_processing_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/PROCESSING state."""
|
||||
if not hasattr(context, "action") or context.action is None:
|
||||
context.action = _create_action_r4(
|
||||
context, f"local/corr-handler-r4-{id(context)}"
|
||||
)
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_strategize_processing(context)
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a recording event bus for r4")
|
||||
def step_service_with_recording_bus_r4(context: Context) -> None:
|
||||
"""Set up recording event bus on existing service."""
|
||||
context.recording_bus_r4 = RecordingEventBusR4()
|
||||
context.service.event_bus = context.recording_bus_r4
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a failing event bus for r4")
|
||||
def step_service_with_failing_bus_r4(context: Context) -> None:
|
||||
"""Set up failing event bus on existing service."""
|
||||
context.service.event_bus = FailingEventBusR4()
|
||||
|
||||
|
||||
@when("I execute the plan for r4")
|
||||
def step_execute_plan_r4(context: Context) -> None:
|
||||
"""Call execute_plan on the current plan."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded PLAN_ESTIMATION_COMPLETE event")
|
||||
def step_verify_estimation_complete_event_r4(context: Context) -> None:
|
||||
"""Verify PLAN_ESTIMATION_COMPLETE was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
est_events = [
|
||||
e
|
||||
for e in context.recording_bus_r4.events
|
||||
if e.event_type == EventType.PLAN_ESTIMATION_COMPLETE
|
||||
]
|
||||
assert len(est_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_ESTIMATION_COMPLETE event, "
|
||||
f"got {len(est_events)}. Events: {[e.event_type for e in context.recording_bus_r4.events]}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan should be in execute phase despite estimation emit failure")
|
||||
def step_plan_in_executeDespiteEstimationEmitFailure_r4(context: Context) -> None:
|
||||
"""Verify plan is in EXECUTE even when estimation emit failed."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan should be in execute phase")
|
||||
def step_plan_in_execute_phase_r4(context: Context) -> None:
|
||||
"""Verify plan is in EXECUTE phase."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan cost_estimate_usd should be populated")
|
||||
def step_verify_cost_estimate_populated_r4(context: Context) -> None:
|
||||
"""Verify plan.cost_estimate_usd was set from estimation result."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.cost_estimate_usd is not None, (
|
||||
"Expected cost_estimate_usd to be set, got None"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# execute_plan with mocked EstimationStubActor for cost_estimate_usd
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I execute the plan for r4 with mocked estimation")
|
||||
def step_execute_plan_with_mocked_estimation_r4(context: Context) -> None:
|
||||
"""Execute plan with EstimationStubActor patched to return cost."""
|
||||
mock_result = context.mock_estimation_result
|
||||
with patch(
|
||||
"cleveragents.application.services.plan_executor.EstimationStubActor.estimate",
|
||||
return_value=mock_result,
|
||||
):
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# execute_plan with lock_service
|
||||
# =================================================================
|
||||
|
||||
|
||||
class MockLockService:
|
||||
"""A mock lock service that tracks acquire/release calls."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.acquired: list[tuple[str, str, str]] = []
|
||||
self.released: list[tuple[str, str, str]] = []
|
||||
|
||||
def acquire(self, owner_id: str, resource_type: str, resource_id: str) -> None:
|
||||
self.acquired.append((owner_id, resource_type, resource_id))
|
||||
|
||||
def release(self, owner_id: str, resource_type: str, resource_id: str) -> None:
|
||||
self.released.append((owner_id, resource_type, resource_id))
|
||||
|
||||
|
||||
@given("an action for execute with lock service for r4")
|
||||
def step_action_execute_lock_r4(context: Context) -> None:
|
||||
"""Create an action for testing execute_plan with lock_service."""
|
||||
context.action = _create_action_r4(context, "local/exec-lock-r4")
|
||||
|
||||
|
||||
@when("I execute the plan with lock service for r4")
|
||||
def step_execute_with_lock_r4(context: Context) -> None:
|
||||
"""Execute plan with a mocked lock service."""
|
||||
context.mock_lock_service = MockLockService()
|
||||
context.service._lock_service = context.mock_lock_service
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the lock service should have acquired and released the plan lock")
|
||||
def step_verify_lock_acquire_release_r4(context: Context) -> None:
|
||||
"""Verify lock was acquired and released for execute_plan."""
|
||||
assert hasattr(context, "mock_lock_service"), "mock_lock_service not found"
|
||||
ls = context.mock_lock_service
|
||||
assert len(ls.acquired) >= 1, (
|
||||
f"Expected at least 1 acquire call, got {len(ls.acquired)}"
|
||||
)
|
||||
assert len(ls.released) >= 1, (
|
||||
f"Expected at least 1 release call, got {len(ls.released)}"
|
||||
)
|
||||
plan_id = context.plan.identity.plan_id
|
||||
for call in ls.acquired:
|
||||
assert call[1] == "plan", f"Expected resource_type 'plan', got {call[1]}"
|
||||
assert call[2] == plan_id, f"Expected resource_id {plan_id}, got {call[2]}"
|
||||
|
||||
|
||||
# =================================================================
|
||||
# apply_plan with lock_service
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action for apply with lock service for r4")
|
||||
def step_action_apply_lock_r4(context: Context) -> None:
|
||||
"""Create an action for testing apply_plan with lock_service."""
|
||||
context.action = _create_action_r4(context, "local/apply-lock-r4")
|
||||
|
||||
|
||||
@given("a plan in execute-complete state for r4")
|
||||
def step_plan_execute_complete_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/COMPLETE state (ready for apply)."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_complete(context)
|
||||
|
||||
|
||||
@when("I apply the plan with lock service for r4")
|
||||
def step_apply_with_lock_r4(context: Context) -> None:
|
||||
"""Apply plan with a mocked lock service."""
|
||||
context.mock_lock_service = MockLockService()
|
||||
context.service._lock_service = context.mock_lock_service
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.apply_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in apply phase")
|
||||
def step_plan_in_apply_phase_r4(context: Context) -> None:
|
||||
"""Verify plan is in APPLY phase."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.APPLY, (
|
||||
f"Expected APPLY phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _subscribe_correction_reconciliation — event_bus.subscribe failure
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("I have a plan lifecycle service with event_bus that fails on subscribe for r4")
|
||||
def step_service_with_failing_subscribe_r4(context: Context) -> None:
|
||||
"""Create service where event_bus.subscribe raises."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.error = None
|
||||
try:
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=FailingSubscribeEventBus(),
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@when("the service is created for r4")
|
||||
def step_service_created_r4(context: Context) -> None:
|
||||
"""Service creation step (no-op since given already created it)."""
|
||||
pass
|
||||
|
||||
|
||||
@then("no exception should be raised for r4")
|
||||
def step_no_exception_r4(context: Context) -> None:
|
||||
"""Verify no exception was raised."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
|
||||
|
||||
# =================================================================
|
||||
# prompt_plan validation scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
def _advance_to_execute_processing(context: Context) -> None:
|
||||
"""Advance plan to execute/PROCESSING state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_execute_complete_for_prompt(context: Context) -> None:
|
||||
"""Advance plan to execute/COMPLETE state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
@given("a plan in execute phase with processing state for prompt_plan r4")
|
||||
def step_plan_execute_processing_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/PROCESSING state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-test-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_processing(context)
|
||||
|
||||
|
||||
@given("a plan in strategize phase for prompt_plan r4")
|
||||
def step_plan_strategize_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/PROCESSING state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-wrong-phase-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_state_prompt(context, PlanPhase.STRATEGIZE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@given("a plan in execute-complete state for prompt_plan r4")
|
||||
def step_plan_execute_complete_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/COMPLETE state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-complete-state-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_complete_for_prompt(context)
|
||||
|
||||
|
||||
@given("a plan in execute phase with errored state for prompt_plan r4")
|
||||
def step_plan_execute_errored_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/ERRORED state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-errored-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_processing(context)
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.fail_execute(pid, error_message="Prior test error")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_state_prompt(
|
||||
context: Context, target_phase: PlanPhase, target_state: ProcessingState
|
||||
) -> None:
|
||||
"""Advance plan to target phase/state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
if (
|
||||
target_phase == PlanPhase.STRATEGIZE
|
||||
and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif (
|
||||
target_phase == PlanPhase.EXECUTE and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
@when("I prompt the plan with blank guidance for r4")
|
||||
def step_prompt_blank_guidance_r4(context: Context) -> None:
|
||||
"""Call prompt_plan with blank guidance."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance=" ")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a ValidationError should be raised for blank guidance")
|
||||
def step_verify_validation_error_r4(context: Context) -> None:
|
||||
"""Verify ValidationError was raised for blank guidance."""
|
||||
assert isinstance(context.error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for wrong phase for r4")
|
||||
def step_prompt_wrong_phase_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when plan is not in Execute phase."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Do something")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a PlanError should be raised for wrong phase")
|
||||
def step_verify_plan_error_wrong_phase_r4(context: Context) -> None:
|
||||
"""Verify PlanError was raised for wrong phase."""
|
||||
assert isinstance(context.error, PlanError), (
|
||||
f"Expected PlanError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for complete state for r4")
|
||||
def step_prompt_wrong_state_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when plan is in COMPLETE state (not recoverable)."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Resume work")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a PlanError should be raised for wrong state")
|
||||
def step_verify_plan_error_wrong_state_r4(context: Context) -> None:
|
||||
"""Verify PlanError was raised for wrong state."""
|
||||
assert isinstance(context.error, PlanError), (
|
||||
f"Expected PlanError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance to resume from errored state for r4")
|
||||
def step_prompt_errored_resume_r4(context: Context) -> None:
|
||||
"""Call prompt_plan to resume from errored state."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Please recover")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan processing_state should be processing")
|
||||
def step_verify_processing_state_r4(context: Context) -> None:
|
||||
"""Verify plan.processing_state is PROCESSING."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING state, got {context.plan.processing_state}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan error_message should be cleared")
|
||||
def step_verify_error_cleared_r4(context: Context) -> None:
|
||||
"""Verify plan.error_message was cleared after resuming from errored."""
|
||||
assert context.plan.error_message is None, (
|
||||
f"Expected error_message to be cleared, got: {context.plan.error_message}"
|
||||
)
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a failing decision service for r4")
|
||||
def step_failing_decision_service_r4(context: Context) -> None:
|
||||
"""Replace decision_service with one that always fails."""
|
||||
context.service.decision_service = FailingDecisionService()
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for r4")
|
||||
def step_prompt_with_failing_decision_service_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when decision_service raises."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Continue anyway")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _run_invariant_reconciliation — event_bus.emit exception scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with invariant_actor for reconciliation success r4")
|
||||
def step_action_invariant_success_r4(context: Context) -> None:
|
||||
"""Create action with invariant_actor set for successful reconciliation."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/inv-success-r4",
|
||||
invariant_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@given("a plan in strategize-complete state with project links for r4")
|
||||
def step_plan_strategize_complete_with_links_r4(context: Context) -> None:
|
||||
"""Create plan with project_links for testing project_name extraction."""
|
||||
if (
|
||||
not hasattr(context.service, "invariant_service")
|
||||
or context.service.invariant_service is None
|
||||
):
|
||||
from cleveragents.application.services.invariant_service import InvariantService
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
|
||||
context.service.invariant_service = InvariantService()
|
||||
context.service.decision_service = DecisionService()
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="test-project-r4")],
|
||||
)
|
||||
_advance_to_strategize_complete(context)
|
||||
|
||||
|
||||
@when("I execute the plan for invariant reconciliation emit failure r4")
|
||||
def step_execute_invariant_emit_fail_r4(context: Context) -> None:
|
||||
"""Execute plan with mocked reconciliation actor (succeeds) and failing emit."""
|
||||
from cleveragents.actor.reconciliation import ReconciliationResult
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
from cleveragents.domain.models.core.invariant import InvariantSet
|
||||
|
||||
context.error = None
|
||||
mock_result = ReconciliationResult(
|
||||
reconciled_set=InvariantSet(invariants=[]),
|
||||
conflicts=[],
|
||||
enforced_decision_ids=[],
|
||||
)
|
||||
|
||||
class PatchedActor(InvariantReconciliationActor):
|
||||
def run(self, **kwargs: Any) -> Any:
|
||||
return mock_result
|
||||
|
||||
with patch(
|
||||
"cleveragents.actor.reconciliation.InvariantReconciliationActor",
|
||||
PatchedActor,
|
||||
):
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event_bus should have been called for INVARIANT_RECONCILED emit failure")
|
||||
def step_verify_invariant_reconciled_emit_fail_r4(context: Context) -> None:
|
||||
"""Verify plan reached EXECUTE despite emit failure in reconciliation."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@given("an action with invariant_actor causing reconciliation failure for r4")
|
||||
def step_action_invariant_fail_r4(context: Context) -> None:
|
||||
"""Create action with invariant_actor that will cause failure."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/inv-fail-r4",
|
||||
invariant_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@when("I execute the plan expecting reconciliation failure for r4")
|
||||
def step_execute_invariant_fail_r4(context: Context) -> None:
|
||||
"""Execute plan where reconciliation raises (triggers INVARIANT_VIOLATED emit fail path)."""
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
|
||||
class FailingActor(InvariantReconciliationActor):
|
||||
def run(self, **kwargs: Any) -> Any:
|
||||
raise RuntimeError("Simulated invariant violation")
|
||||
|
||||
with patch(
|
||||
"cleveragents.actor.reconciliation.InvariantReconciliationActor",
|
||||
FailingActor,
|
||||
):
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except ReconciliationBlockedError as e:
|
||||
context.error = e
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("ReconciliationBlockedError should be raised")
|
||||
def step_verify_reconciliation_blocked_error_r4(context: Context) -> None:
|
||||
"""Verify ReconciliationBlockedError was raised."""
|
||||
assert isinstance(context.error, ReconciliationBlockedError), (
|
||||
f"Expected ReconciliationBlockedError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@then("the event_bus should have been called for INVARIANT_VIOLATED emit failure")
|
||||
def step_verify_invariant_violated_emit_fail_r4(context: Context) -> None:
|
||||
"""Verify INVARIANT_VIOLATED emit exception was handled (error was still raised)."""
|
||||
assert isinstance(context.error, ReconciliationBlockedError), (
|
||||
"Expected ReconciliationBlockedError to be raised despite emit failure"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# Unknown automation profile validation (lines 1230-1232)
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with unknown automation profile for r4")
|
||||
def step_action_unknown_profile_r4(context: Context) -> None:
|
||||
"""Create action with an unknown automation profile name."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/unknown-profile-r4",
|
||||
automation_profile="nonexistent-profile-xyz",
|
||||
)
|
||||
|
||||
|
||||
@when("I use the action to create a plan for r4")
|
||||
def step_use_action_unknown_profile_r4(context: Context) -> None:
|
||||
"""Try to use_action with unknown automation profile (raises during profile resolution)."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
except ValidationError as e:
|
||||
context.error = e
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a ValidationError should be raised for unknown profile")
|
||||
def step_verify_validation_error_unknown_profile_r4(context: Context) -> None:
|
||||
"""Verify ValidationError was raised for unknown automation profile."""
|
||||
assert isinstance(context.error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _handle_correction_applied — event handler scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
class MockEventBusForHandler:
|
||||
"""Event bus that records correction applied events for handler testing."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
self._handler: Any = None
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
self._handler = handler
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event with no plan_id for r4")
|
||||
def step_handle_correction_no_plan_id_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied with an event that has no plan_id."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id=None,
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a recording event bus for correction handler r4")
|
||||
def step_service_recording_bus_for_handler_r4(context: Context) -> None:
|
||||
"""Set up service with recording event bus for _handle_correction_applied testing."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.mock_bus_handler = MockEventBusForHandler()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings, event_bus=context.mock_bus_handler
|
||||
)
|
||||
|
||||
|
||||
@given("the plan lifecycle service is configured for r4")
|
||||
def step_service_configured_r4(context: Context) -> None:
|
||||
"""Ensure service is configured (reuse existing or create new)."""
|
||||
if not hasattr(context, "service") or context.service is None:
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
|
||||
|
||||
@given("the invariant reconciliation raises ReconciliationBlockedError for r4")
|
||||
def step_invariant_raises_blocked_error_r4(context: Context) -> None:
|
||||
"""Patch _run_invariant_reconciliation to raise ReconciliationBlockedError."""
|
||||
|
||||
def _patched_run_invariant(plan: Any) -> None:
|
||||
raise ReconciliationBlockedError(
|
||||
plan_id=plan.identity.plan_id,
|
||||
phase=plan.phase,
|
||||
reason="Simulated blocked error",
|
||||
)
|
||||
|
||||
context.service._run_invariant_reconciliation = _patched_run_invariant
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event for blocked reconciliation for r4")
|
||||
def step_handle_correction_blocked_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied when reconciliation raises ReconciliationBlockedError."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id=context.plan.identity.plan_id,
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@given("get_plan will fail for the plan for r4")
|
||||
def step_get_plan_fails_r4(context: Context) -> None:
|
||||
"""Patch get_plan to raise an exception."""
|
||||
|
||||
def _failing_get_plan(plan_id: str) -> Any:
|
||||
raise RuntimeError("Simulated get_plan failure")
|
||||
|
||||
context.service.get_plan = _failing_get_plan
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event when get_plan fails for r4")
|
||||
def step_handle_correction_get_plan_fails_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied when get_plan raises."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id="nonexistent-plan-id",
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Step definitions for stdio transport subprocess cleanup BDD tests.
|
||||
|
||||
Verifies that ``StdioTransport.start()`` properly cleans up any partially-started
|
||||
subprocess when ``Popen()`` raises an ``OSError`` during failed initialisation.
|
||||
|
||||
Covers the regression guard introduced in PR #10597 (issue #11011).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.lsp.errors import LspError
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('a fresh StdioTransport for command "{cmd}"')
|
||||
def step_give_fresh_transport(context: Context, cmd: str) -> None:
|
||||
"""Create a new transport instance and reset test state."""
|
||||
context.transport = StdioTransport(command=cmd)
|
||||
context.transport_start_error = None
|
||||
context.stop_call_count = 0
|
||||
|
||||
|
||||
@given("Popen is patched to simulate partial fork-before-OS-error")
|
||||
def step_give_patched_popen(context: Context) -> None:
|
||||
"""Set up mock state for subprocess cleanup scenarios.
|
||||
|
||||
Creates a mock child process that ``stop()`` can act upon, simulating the
|
||||
case where ``subprocess.Popen()`` forks but then ``exec`` fails in the
|
||||
child before returning control.
|
||||
"""
|
||||
# Create a mock child process — as if Popen returned it before OSError
|
||||
context.mock_child = MagicMock()
|
||||
context.mock_child.pid = 9999
|
||||
context.mock_child.poll.return_value = None # still alive
|
||||
# Track whether stop() was called on it
|
||||
context.mock_child.terminate = MagicMock()
|
||||
context.mock_child.kill = MagicMock()
|
||||
context.mock_child.wait = MagicMock(return_value=None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when(
|
||||
'I call `` StdioTransport.start() `` and Popen raises OSError with "{msg}"'
|
||||
)
|
||||
def step_when_start_oserror(context: Context, msg: str) -> None:
|
||||
"""Simulate Popen assigning a child process then raising OSError.
|
||||
|
||||
This mimics the real-world scenario where ``subprocess.Popen()`` in CPython
|
||||
forks a child, and the child's ``execvp`` fails — ``Popen`` has already
|
||||
assigned ``_process`` to the child object before raising ``OSError``.
|
||||
"""
|
||||
transport = context.transport
|
||||
|
||||
def spawn_that_fails(_cmd, **_kwargs): # noqa: ANN002
|
||||
"""Mimics Popen: assign child, then raise OSError immediately."""
|
||||
transport._process = context.mock_child
|
||||
raise OSError(msg)
|
||||
|
||||
with patch("subprocess.Popen", side_effect=spawn_that_fails):
|
||||
try:
|
||||
transport.start()
|
||||
except LspError as exc:
|
||||
context.transport_start_error = exc
|
||||
|
||||
|
||||
@when("I call ``StdioTransport.start()`` and Popen raises OSError with " '"child exec failed after fork"')
|
||||
def step_when_standard_oserror(context: Context) -> None:
|
||||
"""Run the start() scenario with a standard error message."""
|
||||
msg = "child exec failed after fork"
|
||||
|
||||
transport = context.transport
|
||||
|
||||
def spawn_that_fails(_cmd, **_kwargs): # noqa: ANN002
|
||||
transport._process = context.mock_child
|
||||
raise OSError(msg)
|
||||
|
||||
patcher = patch("subprocess.Popen", side_effect=spawn_that_fails)
|
||||
patcher.start()
|
||||
context.add_cleanup(patcher.stop)
|
||||
|
||||
try:
|
||||
transport.start()
|
||||
except LspError as exc:
|
||||
context.transport_start_error = exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("an LspError should be raised with message containing 'Failed to start'")
|
||||
def step_then_lsp_error_raised(context: Context) -> None:
|
||||
"""Verify that the OSError is wrapped in an appropriate LspError."""
|
||||
assert context.transport_start_error is not None, "Expected LspError but none was raised"
|
||||
assert isinstance(context.transport_start_error, LspError), (
|
||||
f"Expected LspError but got {type(context.transport_start_error)}"
|
||||
)
|
||||
assert "Failed to start" in str(context.transport_start_error), (
|
||||
f"Expected 'Failed to start' in error message, got: {context.transport_start_error}"
|
||||
)
|
||||
|
||||
|
||||
@then("the transport's internal ``_process`` should be cleaned up (None)")
|
||||
def step_then_process_is_none(context: Context) -> None:
|
||||
"""Verify that _process was set back to None after cleanup."""
|
||||
assert context.transport._process is None, (
|
||||
f"Expected _process to be None but got {context.transport._process}"
|
||||
)
|
||||
|
||||
|
||||
@then("the subprocess cleanup method ``stop()`` should have been called on the partial process")
|
||||
def step_then_stop_was_called(context: Context) -> None:
|
||||
"""Verify that stop() was invoked on the partially-started process.
|
||||
|
||||
The OSError handler calls :meth:`~StdioTransport.stop()` to terminate and
|
||||
reap any child process that was assigned before the error occurred.
|
||||
"""
|
||||
# stop() internally calls terminate(). If it wasn't called, the cleanup
|
||||
# path in transport.py was not executed.
|
||||
if context.transport_start_error is None:
|
||||
# start() didn't raise — this scenario doesn't apply
|
||||
return
|
||||
|
||||
# Since stop() was called (indirectly) by the OSError handler, and
|
||||
# stop() calls self._process.terminate(), the mock should reflect that.
|
||||
# The mock_child was set on _process; stop() should have been invoked.
|
||||
assert context.transport._process is None, (
|
||||
"Expected _process to be cleared after stop()"
|
||||
)
|
||||
|
||||
|
||||
@then("the transport should no longer track any active process reference")
|
||||
def step_then_no_active_process(context: Context) -> None:
|
||||
"""Final assertion: transport has no lingering process reference."""
|
||||
assert context.transport._process is None, (
|
||||
"Transport still tracks a non-None _process after cleanup"
|
||||
)
|
||||
assert context.transport.is_alive is False, (
|
||||
"Transport reports itself as alive without an active process"
|
||||
)
|
||||
@@ -0,0 +1,127 @@
|
||||
@tdd_issue @tdd_issue_4328
|
||||
Feature: Automation Profile Gates - Issue #4328 Regression Tests
|
||||
Regression tests for manual automation profile being disrespected during phase
|
||||
completion. Verifies that complete_strategize() and complete_execute() respect
|
||||
automation profile gates and do NOT auto-progress when thresholds are 1.0.
|
||||
|
||||
Background:
|
||||
Given I have a plan lifecycle service with automation level support
|
||||
|
||||
# ========================================================================
|
||||
# Issue #4328: Manual profile (all thresholds = 1.0) should NOT auto-progress
|
||||
# These scenarios verify the fix - that complete_strategize/execute do NOT
|
||||
# unconditionally call auto_progress()
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Manual profile create_tool=1.0 blocks strategize to execute transition
|
||||
Given I have a plan with automation level "manual" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "strategize"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
Scenario: Manual profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "manual" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Verify auto_progress() IS called when should_auto_progress() returns true
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Full-auto profile create_tool=0.0 allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "full_automation" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Full-auto profile select_tool=0.0 allows execute to apply auto-progress
|
||||
Given I have a plan with automation level "full_automation" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "apply"
|
||||
And the automated plan processing state should be "applied"
|
||||
|
||||
# ========================================================================
|
||||
# Supervised profile: create_tool=1.0 (blocks execute), select_tool=1.0 (blocks apply)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Supervised profile create_tool=1.0 blocks strategize to execute auto-progress
|
||||
Given I have a plan with automation level "supervised" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "strategize"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
Scenario: Supervised profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "supervised" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Review-before-apply (auto) profile: create_tool=0.0, select_tool=1.0
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Auto profile create_tool=0.0 allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "review_before_apply" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Auto profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "review_before_apply" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# CI profile: all thresholds=0.0 (auto-progresses everything)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: CI profile blocks neither strategize nor execute transitions
|
||||
Given I have a plan with automation level "ci" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: CI profile execute phase auto-progresses to apply
|
||||
Given I have a plan with automation level "ci" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "apply"
|
||||
And the automated plan processing state should be "applied"
|
||||
|
||||
# ========================================================================
|
||||
# Trusted profile: create_tool=0.0, select_tool=1.0 (same as auto for strategize/execute)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Trusted profile allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "trusted" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Trusted profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "trusted" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Cautious profile: intermediate thresholds (0.0 < v < 1.0)
|
||||
# Tests the spec's Threshold Semantics for probabilistic gates:
|
||||
# "0.0 < v < 1.0 → system may proceed if confidence >= threshold"
|
||||
# With default confidence=0.5, thresholds > 0.5 should block.
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Cautious profile create_tool=0.7 blocks strategize to execute with default confidence
|
||||
Given I have a plan with automation level "cautious" in strategize phase with complete state for query
|
||||
Then should_auto_progress should return false
|
||||
|
||||
Scenario: Cautious profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "cautious" in execute phase with complete state for query
|
||||
Then should_auto_progress should return false
|
||||
@@ -4,7 +4,6 @@ Feature: TDD Issue #10470 — MCPToolAdapter.infer_resource_slots() raises TypeE
|
||||
the properties dict. When the key exists but has a null value ({"properties": None}),
|
||||
dict.get() returns None instead of the default {}, causing a TypeError when iterating.
|
||||
|
||||
@tdd_expected_fail
|
||||
Scenario: infer_resource_slots() with null properties does not raise TypeError
|
||||
Given an MCP tool input schema where properties key is null
|
||||
When I call infer_resource_slots with the null properties schema
|
||||
|
||||
@@ -192,22 +192,15 @@ def _review_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
service.start_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert p.processing_state == ProcessingState.PROCESSING
|
||||
# Strategize + execute already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0, create_tool=0.0 in review profile)
|
||||
assert p.processing_state == ProcessingState.COMPLETE
|
||||
|
||||
service.complete_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert (p.phase, p.processing_state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.processing_state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(pid)
|
||||
p = service.get_plan(pid)
|
||||
# Review: create_tool=0.0 → auto-transitioned to Execute
|
||||
assert p.phase == PlanPhase.EXECUTE, f"Expected EXECUTE phase, got {p.phase}"
|
||||
|
||||
# Review: select_tool=1.0 gates execute→apply
|
||||
|
||||
if p.phase == PlanPhase.EXECUTE and p.processing_state == ProcessingState.QUEUED:
|
||||
service.start_execute(pid)
|
||||
@@ -331,7 +324,10 @@ def _action_with_review_profile() -> None:
|
||||
ArgumentRequirement,
|
||||
ArgumentType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import PlanPhase, ProjectLink
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
|
||||
service = PlanLifecycleService(settings=Settings())
|
||||
action = service.create_action(
|
||||
@@ -390,7 +386,7 @@ def _action_with_review_profile() -> None:
|
||||
"backfill_source": "audit_log",
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
assert plan.arguments["table_name"] == "users"
|
||||
assert len(plan.invariants) >= 4
|
||||
|
||||
@@ -593,23 +589,13 @@ def _plan_lifecycle_review_profile() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Strategize
|
||||
service.start_strategize(pid)
|
||||
# Strategize + execute already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0, create_tool=0.0 in review profile)
|
||||
p = service.get_plan(pid)
|
||||
assert p.processing_state == ProcessingState.PROCESSING
|
||||
|
||||
service.complete_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert (p.phase, p.processing_state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.processing_state.value}"
|
||||
assert p.processing_state == ProcessingState.COMPLETE
|
||||
assert p.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase after auto-progress, got {p.phase}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(pid)
|
||||
p = service.get_plan(pid)
|
||||
|
||||
# Execute
|
||||
if p.phase == PlanPhase.EXECUTE and p.processing_state == ProcessingState.QUEUED:
|
||||
|
||||
@@ -182,8 +182,6 @@ def _trusted_profile_behavior() -> None:
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
AutomationProfileProvenance,
|
||||
AutomationProfileRef,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
@@ -204,32 +202,16 @@ def _trusted_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Explicitly bind the trusted profile to the plan (use_action records
|
||||
# the profile on the Action but does not yet propagate it to the Plan;
|
||||
# in production this happens via the automation-profile resolution
|
||||
# chain at plan creation time).
|
||||
plan.automation_profile = AutomationProfileRef(
|
||||
profile_name="trusted",
|
||||
provenance=AutomationProfileProvenance.ACTION,
|
||||
)
|
||||
|
||||
service.start_strategize(pid)
|
||||
plan = service.complete_strategize(pid)
|
||||
|
||||
# Trusted: create_tool=0.0 -> auto-progress fires inside
|
||||
# complete_strategize, advancing from Strategize/COMPLETE
|
||||
# to Execute/QUEUED automatically.
|
||||
# Trusted: decompose_task=0.0 → strategize auto-ran in try_auto_run
|
||||
# create_tool=0.0 → execute auto-ran in try_auto_run
|
||||
plan = service.get_plan(pid)
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected auto-progress to Execute, got {plan.phase}"
|
||||
)
|
||||
assert plan.processing_state == ProcessingState.QUEUED, (
|
||||
f"Expected QUEUED after auto-progress, got {plan.processing_state}"
|
||||
assert plan.processing_state == ProcessingState.COMPLETE, (
|
||||
f"Expected COMPLETE after auto-progress, got {plan.processing_state}"
|
||||
)
|
||||
|
||||
# Execute phase
|
||||
service.start_execute(pid)
|
||||
plan = service.complete_execute(pid)
|
||||
|
||||
# Trusted: select_tool=1.0 -> gated apply
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
assert plan.processing_state == ProcessingState.COMPLETE
|
||||
@@ -325,7 +307,7 @@ def _action_with_doc_args() -> None:
|
||||
"output_dir": "docs/generated/",
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
assert plan.arguments["doc_types"] == (
|
||||
"api-reference,architecture,module-guides,onboarding"
|
||||
)
|
||||
@@ -476,8 +458,6 @@ def _trusted_doc_lifecycle() -> None:
|
||||
ArgumentType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
AutomationProfileProvenance,
|
||||
AutomationProfileRef,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
@@ -521,26 +501,13 @@ def _trusted_doc_lifecycle() -> None:
|
||||
},
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
|
||||
# Bind trusted profile to the plan
|
||||
plan.automation_profile = AutomationProfileRef(
|
||||
profile_name="trusted",
|
||||
provenance=AutomationProfileProvenance.ACTION,
|
||||
)
|
||||
# Trusted profile auto-ran strategize (decompose_task=0.0)
|
||||
# and execute (create_tool=0.0) in try_auto_run.
|
||||
# Plan is already at Execute/COMPLETE — skip to applying.
|
||||
|
||||
# Strategize — trusted profile auto-progresses to Execute
|
||||
service.start_strategize(pid)
|
||||
plan = service.complete_strategize(pid)
|
||||
|
||||
# Auto-progress fires: plan is now in Execute/QUEUED
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected auto-progress to Execute, got {plan.phase}"
|
||||
)
|
||||
|
||||
# Execute
|
||||
service.start_execute(pid)
|
||||
plan = service.complete_execute(pid)
|
||||
# Execute is already complete
|
||||
|
||||
# Trusted gated: execute -> apply is manual
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
|
||||
@@ -688,7 +688,7 @@ def _plan_status_rendering() -> None:
|
||||
definition_of_done="Test plan status rendering",
|
||||
strategy_actor="local/s",
|
||||
execution_actor="local/e",
|
||||
automation_profile="trusted",
|
||||
automation_profile="manual",
|
||||
)
|
||||
|
||||
plan = service.use_action(
|
||||
|
||||
@@ -371,7 +371,7 @@ def cmd_plan_use() -> None:
|
||||
)
|
||||
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
assert len(plan.project_links) == 4
|
||||
assert plan.arguments == _PLAN_ARGS
|
||||
# Action invariants are inherited automatically from the action
|
||||
@@ -676,9 +676,7 @@ def cmd_full_lifecycle() -> None:
|
||||
assert len(plan.multi_project_metadata.project_scopes) == 4
|
||||
|
||||
# ── Strategize + spawn children ──
|
||||
plan_svc.start_strategize(pid)
|
||||
children = _make_child_statuses()
|
||||
plan_svc.complete_strategize(pid)
|
||||
|
||||
# ── Execute: common-lib first, then services ──
|
||||
plan_svc.execute_plan(pid)
|
||||
|
||||
+42
-40
@@ -336,8 +336,8 @@ def ci_plan_lifecycle() -> None:
|
||||
created_by="ci-pipeline",
|
||||
arguments={"pr_branch": "fix/handle-null-users", "base_branch": "main"},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.phase == PlanPhase.APPLY
|
||||
assert plan.state == ProcessingState.APPLIED
|
||||
plan_id = plan.identity.plan_id
|
||||
assert plan_id, "Plan must have a plan_id"
|
||||
# Verify arguments flowed to the plan
|
||||
@@ -372,43 +372,45 @@ def ci_plan_lifecycle() -> None:
|
||||
ci_profile = profile_svc.resolve_profile(plan_profile="ci")
|
||||
assert ci_profile.name == "ci"
|
||||
# use_action now resolves automation profile precedence and can auto-progress
|
||||
# beyond phase boundaries for non-manual profiles (e.g., ci).
|
||||
# beyond phase boundaries for non-manual profiles (e.g., ci auto-runs to Applied).
|
||||
# --- Phase-by-phase plan completion (spec Step 3) ---
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert p.state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING after start_strategize, got {p.state}"
|
||||
)
|
||||
service.complete_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(plan_id)
|
||||
# Execute phase
|
||||
service.start_execute(plan_id)
|
||||
service.complete_execute(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.EXECUTE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.APPLY, ProcessingState.QUEUED),
|
||||
(PlanPhase.APPLY, ProcessingState.APPLIED),
|
||||
}, (
|
||||
"After complete_execute expected execute/complete, apply/queued, "
|
||||
"or apply/applied, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.EXECUTE:
|
||||
service.apply_plan(plan_id)
|
||||
# Apply phase
|
||||
if p.phase == PlanPhase.APPLY and p.state == ProcessingState.QUEUED:
|
||||
service.start_apply(plan_id)
|
||||
service.complete_apply(plan_id)
|
||||
if not p.is_terminal:
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert p.state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING after start_strategize, got {p.state}"
|
||||
)
|
||||
service.complete_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(plan_id)
|
||||
# Execute phase
|
||||
service.start_execute(plan_id)
|
||||
service.complete_execute(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.EXECUTE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.APPLY, ProcessingState.QUEUED),
|
||||
(PlanPhase.APPLY, ProcessingState.APPLIED),
|
||||
}, (
|
||||
"After complete_execute expected execute/complete, apply/queued, "
|
||||
"or apply/applied, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.EXECUTE:
|
||||
service.apply_plan(plan_id)
|
||||
# Apply phase
|
||||
if p.phase == PlanPhase.APPLY and p.state == ProcessingState.QUEUED:
|
||||
service.start_apply(plan_id)
|
||||
service.complete_apply(plan_id)
|
||||
# Verify terminal state — the polling loop exits on 'applied'
|
||||
final = service.get_plan(plan_id)
|
||||
assert final.state == ProcessingState.APPLIED, (
|
||||
@@ -420,9 +422,9 @@ def ci_plan_lifecycle() -> None:
|
||||
created_by="ci-pipeline",
|
||||
arguments={"pr_branch": "fix/handle-null-users", "base_branch": "main"},
|
||||
)
|
||||
cancelled = service.cancel_plan(cancelled_plan.identity.plan_id, reason="ci cancel")
|
||||
assert cancelled.state == ProcessingState.CANCELLED
|
||||
assert cancelled.is_terminal
|
||||
assert cancelled_plan.state == ProcessingState.APPLIED, (
|
||||
"CI profile auto-runs plan to terminal APPLIED state"
|
||||
)
|
||||
print("wf07-ci-plan-lifecycle-ok")
|
||||
|
||||
|
||||
|
||||
@@ -383,9 +383,8 @@ def _supervised_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Strategize phase
|
||||
service.start_strategize(pid)
|
||||
service.complete_strategize(pid)
|
||||
# Strategize already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0 in supervised profile)
|
||||
plan = service.get_plan(pid)
|
||||
|
||||
# Supervised: create_tool=1.0 means should_auto_progress is False
|
||||
@@ -496,7 +495,7 @@ def _create_infra_optimize_action() -> None:
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
|
||||
# Verify arguments flowed to plan
|
||||
assert plan.arguments["optimization_targets"] == "compute,storage"
|
||||
|
||||
@@ -440,8 +440,8 @@ def use_shared_action() -> None:
|
||||
assert plan.phase == PlanPhase.STRATEGIZE, (
|
||||
f"Expected phase STRATEGIZE, got {plan.phase}"
|
||||
)
|
||||
assert plan.processing_state == ProcessingState.QUEUED, (
|
||||
f"Expected state QUEUED, got {plan.processing_state}"
|
||||
assert plan.processing_state == ProcessingState.COMPLETE, (
|
||||
f"Expected state COMPLETE, got {plan.processing_state}"
|
||||
)
|
||||
assert plan.action_name == "team-alpha/generate-tests", (
|
||||
f"Expected action_name 'team-alpha/generate-tests', got '{plan.action_name}'"
|
||||
@@ -510,9 +510,8 @@ def plan_namespace() -> None:
|
||||
alpha_plan = lifecycle.use_action(action_name="team-alpha/monitor-lint")
|
||||
beta_plan = lifecycle.use_action(action_name="team-beta/monitor-test")
|
||||
|
||||
# Move one plan to Execute to validate phase-aware monitoring.
|
||||
lifecycle.start_strategize(beta_plan.identity.plan_id)
|
||||
lifecycle.complete_strategize(beta_plan.identity.plan_id)
|
||||
# Both plans auto-completed strategize (decompose_task=0.0 in supervised profile)
|
||||
# Move beta to Execute — already at STRATEGIZE/COMPLETE, skip to execute_plan
|
||||
beta_plan = lifecycle.execute_plan(beta_plan.identity.plan_id)
|
||||
|
||||
# List all plans
|
||||
@@ -528,8 +527,8 @@ def plan_namespace() -> None:
|
||||
assert alpha_plans[0].phase == PlanPhase.STRATEGIZE, (
|
||||
f"Expected team-alpha phase STRATEGIZE, got {alpha_plans[0].phase}"
|
||||
)
|
||||
assert alpha_plans[0].processing_state == ProcessingState.QUEUED, (
|
||||
"Expected team-alpha processing state QUEUED, "
|
||||
assert alpha_plans[0].processing_state == ProcessingState.COMPLETE, (
|
||||
"Expected team-alpha processing state COMPLETE, "
|
||||
f"got {alpha_plans[0].processing_state}"
|
||||
)
|
||||
assert alpha_plans[0].action_name == "team-alpha/monitor-lint", (
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import json
|
||||
from pathlib import PurePath
|
||||
from typing import Any, Protocol
|
||||
|
||||
@@ -70,6 +71,72 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
return ProjectContextPolicy().resolve_view("execute")
|
||||
return policy.resolve_view("execute")
|
||||
|
||||
def _resolve_hot_max_tokens(self, project_names: list[str]) -> int:
|
||||
"""Resolve the effective ``hot_max_tokens`` budget for *project_names*.
|
||||
|
||||
Looks up each project's :attr:`ContextConfig.hot_max_tokens` from its
|
||||
stored context configuration. Projects that do not have an explicit
|
||||
setting are excluded from the aggregation so they do not affect
|
||||
the computed value.
|
||||
|
||||
Returns the **maximum** of all explicitly-set project-level values,
|
||||
falling back to :attr:`_hot_max_tokens` (the caller-passed global
|
||||
default) when no project overrides are present.
|
||||
"""
|
||||
from typing import cast
|
||||
|
||||
candidates: list[int] = []
|
||||
for namespaced_name in project_names:
|
||||
row = None
|
||||
try:
|
||||
session = self._project_repository._session() # type: ignore[attr-defined]
|
||||
from cleveragents.infrastructure.database.models import (
|
||||
NamespacedProjectModel,
|
||||
)
|
||||
|
||||
row = (
|
||||
session.query(NamespacedProjectModel)
|
||||
.filter_by(namespaced_name=namespaced_name)
|
||||
.first()
|
||||
)
|
||||
session.close()
|
||||
except AttributeError:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_session_factory_missing",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
continue
|
||||
except Exception:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_lookup_failed",
|
||||
project_name=namespaced_name,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
if row is not None and row.context_policy_json is not None:
|
||||
try:
|
||||
config_dict = json.loads(cast(str, row.context_policy_json))
|
||||
tokens = config_dict.get("hot_max_tokens")
|
||||
if tokens is not None and isinstance(tokens, int) and tokens > 0:
|
||||
candidates.append(tokens)
|
||||
except (ValueError, TypeError):
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_parse_failed",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
|
||||
if candidates:
|
||||
effective = max(candidates)
|
||||
self._logger.info(
|
||||
"hot_max_tokens_resolved_from_projects",
|
||||
project_names=project_names,
|
||||
project_values=candidates,
|
||||
effective=effective,
|
||||
)
|
||||
return effective
|
||||
# No project overrides --- use the caller-passed global default.
|
||||
return self._hot_max_tokens
|
||||
|
||||
@staticmethod
|
||||
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
|
||||
"""Return whether *path* passes include/exclude path globs.
|
||||
@@ -226,14 +293,21 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
)
|
||||
return None
|
||||
|
||||
budget = CoreContextBudget(max_tokens=self._hot_max_tokens, reserved_tokens=0)
|
||||
# Resolve effective hot_max_tokens: project-level overrides take precedence
|
||||
# over the global default. When multiple projects have explicit values,
|
||||
# use the maximum so all projects can contribute within their biggest budget.
|
||||
effective_hot_max_tokens = self._resolve_hot_max_tokens(project_names)
|
||||
|
||||
budget = CoreContextBudget(
|
||||
max_tokens=effective_hot_max_tokens, reserved_tokens=0
|
||||
)
|
||||
request = ContextRequest(
|
||||
query=(
|
||||
f"Execute-phase context for plan {plan.identity.plan_id} "
|
||||
f"({', '.join(project_names)})"
|
||||
),
|
||||
purpose="llm_execute_phase_prompt",
|
||||
max_tokens=self._hot_max_tokens,
|
||||
max_tokens=effective_hot_max_tokens,
|
||||
)
|
||||
payload = self._pipeline.assemble(
|
||||
plan_id=plan.identity.plan_id,
|
||||
|
||||
@@ -480,7 +480,15 @@ class LLMExecuteActor:
|
||||
sandbox_root: str,
|
||||
llm_output: str,
|
||||
) -> None:
|
||||
"""Write generated file contents to the sandbox directory."""
|
||||
"""Write generated file contents to the sandbox directory.
|
||||
|
||||
Uses semantic path containment via os.path.relpath instead of
|
||||
string prefix matching (str.startswith). String prefix matching
|
||||
is vulnerable to sibling-directory prefix-collision attacks where
|
||||
/tmp/sandbox would incorrectly match /tmp/sandboxmalicious/file.
|
||||
|
||||
See issue #7478 — startswith bypass in path containment checks.
|
||||
"""
|
||||
|
||||
pattern = re.compile(
|
||||
r"FILE:\s*(.+?)\s*\n```[^\n]*\n(.*?)```",
|
||||
@@ -491,8 +499,9 @@ class LLMExecuteActor:
|
||||
path = match.group(1).strip()
|
||||
content = match.group(2)
|
||||
full_path = os.path.normpath(os.path.join(sandbox_root, path))
|
||||
rel = os.path.relpath(full_path, sandbox_root)
|
||||
# Path traversal guard: reject paths escaping sandbox
|
||||
if not full_path.startswith(sandbox_root + os.sep):
|
||||
if rel.startswith(".." + os.sep) or rel == "..":
|
||||
logger.warning(
|
||||
"Rejected path traversal in LLM output",
|
||||
path=path,
|
||||
|
||||
@@ -99,6 +99,7 @@ from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.application.services.async_worker import InMemoryJobStore
|
||||
from cleveragents.application.services.autonomy_controller import AutonomyController
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
from cleveragents.application.services.error_pattern_service import (
|
||||
ErrorPatternService,
|
||||
@@ -195,6 +196,7 @@ class PlanLifecycleService:
|
||||
config_service: ConfigService | None = None,
|
||||
invariant_service: InvariantService | None = None,
|
||||
lock_service: LockService | None = None,
|
||||
autonomy_controller: AutonomyController | None = None,
|
||||
):
|
||||
"""Initialize the plan lifecycle service.
|
||||
|
||||
@@ -237,6 +239,14 @@ class PlanLifecycleService:
|
||||
block, preventing concurrent modifications to the same
|
||||
plan. When ``None``, locking is silently skipped for
|
||||
backward compatibility with existing tests.
|
||||
autonomy_controller: Optional :class:`AutonomyController` for
|
||||
evaluating intermediate automation profile thresholds
|
||||
(0.0 < v < 1.0) during phase-transition gate checks.
|
||||
When provided, the controller's confidence comparison
|
||||
is used for intermediate thresholds per the spec
|
||||
("confidence >= threshold → proceed automatically").
|
||||
When ``None``, intermediate thresholds use a default
|
||||
confidence of 0.5 (conservative).
|
||||
"""
|
||||
self.settings = settings
|
||||
self.unit_of_work = unit_of_work
|
||||
@@ -247,6 +257,7 @@ class PlanLifecycleService:
|
||||
self._config_service = config_service
|
||||
self.invariant_service = invariant_service
|
||||
self._lock_service = lock_service
|
||||
self._autonomy_controller = autonomy_controller
|
||||
self._logger = logger.bind(service="plan_lifecycle")
|
||||
self.preflight_guardrail = PlanPreflightGuardrail()
|
||||
self._subscribe_correction_reconciliation()
|
||||
@@ -1146,6 +1157,12 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# When the automation profile permits, drive the plan through its
|
||||
# lifecycle synchronously. try_auto_run() checks decompose_task,
|
||||
# create_tool, and select_tool thresholds and auto-advances the
|
||||
# plan only when each gate is below 1.0 (spec §Automation Profiles).
|
||||
plan = self.try_auto_run(plan_id)
|
||||
|
||||
return plan
|
||||
|
||||
def _resolve_plan_profile_ref(
|
||||
@@ -1533,8 +1550,9 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# Auto-progress if automation level permits
|
||||
return self.auto_progress(plan_id)
|
||||
if self.should_auto_progress(plan):
|
||||
return self.auto_progress(plan_id)
|
||||
return plan
|
||||
|
||||
def fail_strategize(self, plan_id: str, error_message: str) -> Plan:
|
||||
"""Mark Strategize phase as failed.
|
||||
@@ -1751,8 +1769,9 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# Auto-progress if automation level permits
|
||||
return self.auto_progress(plan_id)
|
||||
if self.should_auto_progress(plan):
|
||||
return self.auto_progress(plan_id)
|
||||
return plan
|
||||
|
||||
def fail_execute(self, plan_id: str, error_message: str) -> Plan:
|
||||
"""Mark Execute phase as failed."""
|
||||
@@ -2298,12 +2317,76 @@ class PlanLifecycleService:
|
||||
)
|
||||
return profile
|
||||
|
||||
def _should_auto_progress_for_threshold(
|
||||
self,
|
||||
threshold: float,
|
||||
operation_type: str,
|
||||
profile: AutomationProfile,
|
||||
) -> bool:
|
||||
"""Evaluate whether to auto-progress for a given threshold value.
|
||||
|
||||
Per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-progress (fully automatic)
|
||||
- 1.0 → never auto-progress (always requires human approval)
|
||||
- 0.0 < v < 1.0 → proceed only if confidence >= threshold
|
||||
|
||||
For intermediate thresholds, confidence is computed from default
|
||||
factors (all 0.5) since phase transitions do not have the
|
||||
contextual factors (blast_radius, file_count, test_coverage)
|
||||
that are available for tool-invocation decisions.
|
||||
|
||||
Args:
|
||||
threshold: The automation profile threshold value.
|
||||
operation_type: The threshold field name (e.g. 'create_tool').
|
||||
profile: The active AutomationProfile (needed when a
|
||||
controller is injected so it can do the comparison).
|
||||
|
||||
Returns:
|
||||
True if auto-progress is permitted, False otherwise.
|
||||
"""
|
||||
if threshold <= 0.0:
|
||||
return True
|
||||
if threshold >= 1.0:
|
||||
return False
|
||||
|
||||
if self._autonomy_controller is not None:
|
||||
from cleveragents.domain.models.core.escalation import (
|
||||
ConfidenceFactors,
|
||||
OperationContext,
|
||||
)
|
||||
|
||||
factors = ConfidenceFactors(
|
||||
past_success_rate=0.5,
|
||||
codebase_familiarity=0.5,
|
||||
risk_assessment=0.5,
|
||||
invariant_complexity=0.5,
|
||||
)
|
||||
operation = OperationContext(operation_type=operation_type)
|
||||
decision = self._autonomy_controller.should_proceed_automatically(
|
||||
operation=operation,
|
||||
factors=factors,
|
||||
profile=profile,
|
||||
)
|
||||
return decision.proceed
|
||||
|
||||
default_confidence = 0.5
|
||||
proceed = default_confidence >= threshold
|
||||
self._logger.debug(
|
||||
"intermediate_threshold_default_confidence",
|
||||
operation_type=operation_type,
|
||||
threshold=threshold,
|
||||
default_confidence=default_confidence,
|
||||
proceed=proceed,
|
||||
)
|
||||
return proceed
|
||||
|
||||
def should_auto_progress(self, plan: Plan) -> bool:
|
||||
"""Check whether the plan should automatically advance.
|
||||
|
||||
Uses AutomationProfile thresholds: a threshold of ``0.0``
|
||||
on the relevant phase means the transition is fully
|
||||
automatic. A threshold of ``1.0`` requires human approval.
|
||||
Uses AutomationProfile thresholds per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-progress (fully automatic)
|
||||
- 1.0 → never auto-progress (always requires human approval)
|
||||
- 0.0 < v < 1.0 → auto-progress only if confidence >= threshold
|
||||
|
||||
Task-type threshold fields used as phase-transition gates
|
||||
(per specification, Section "Automation Profiles"):
|
||||
@@ -2313,10 +2396,7 @@ class PlanLifecycleService:
|
||||
* ``access_network`` → auto-revert from Apply
|
||||
* ``delete_content`` → auto-revert from Execute (strategy revision)
|
||||
|
||||
Returns True when:
|
||||
|
||||
- Strategize/COMPLETE and ``create_tool < 1.0``
|
||||
- Execute/COMPLETE and ``select_tool < 1.0``
|
||||
Returns True when confidence or threshold permits auto-progress.
|
||||
|
||||
Returns False otherwise.
|
||||
"""
|
||||
@@ -2325,20 +2405,27 @@ class PlanLifecycleService:
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
# Strategize complete -> auto-execute?
|
||||
if (
|
||||
plan.phase == PlanPhase.STRATEGIZE
|
||||
and plan.processing_state == ProcessingState.COMPLETE
|
||||
and profile.create_tool < 1.0
|
||||
):
|
||||
return True
|
||||
return self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
)
|
||||
|
||||
# Execute complete -> auto-apply?
|
||||
return bool(
|
||||
if (
|
||||
plan.phase == PlanPhase.EXECUTE
|
||||
and plan.processing_state == ProcessingState.COMPLETE
|
||||
and profile.select_tool < 1.0
|
||||
)
|
||||
):
|
||||
return self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
)
|
||||
|
||||
return False
|
||||
|
||||
def _complete_apply_if_queued(self, plan_id: str) -> Plan:
|
||||
"""Drive a plan from Apply/QUEUED to terminal APPLIED.
|
||||
@@ -2439,16 +2526,16 @@ class PlanLifecycleService:
|
||||
When the plan's automation profile permits, this method advances
|
||||
the plan synchronously through Strategize → Execute → Apply:
|
||||
|
||||
* ``decompose_task < 1.0`` → start + complete Strategize
|
||||
* ``create_tool < 1.0`` → start + complete Execute
|
||||
* ``select_tool < 1.0`` → start + complete Apply
|
||||
Per the spec's Threshold Semantics, for each phase:
|
||||
- threshold = 0.0 → always auto-progress
|
||||
- threshold >= 1.0 → never auto-progress (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-progress only if confidence >= threshold
|
||||
|
||||
Each completion step calls :meth:`auto_progress` internally,
|
||||
which transitions the plan to the next phase when appropriate.
|
||||
|
||||
If any threshold is ``1.0`` (human approval required), the plan
|
||||
stops at that phase boundary. The method is idempotent: calling
|
||||
it on a plan already in a terminal state simply returns the plan.
|
||||
The method is idempotent: calling it on a plan already in a
|
||||
terminal state simply returns the plan.
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2466,46 +2553,48 @@ class PlanLifecycleService:
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
# --- Strategize phase: QUEUED → PROCESSING → COMPLETE -----------
|
||||
# (decompose_task threshold gates the Strategize phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.STRATEGIZE
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.decompose_task < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.decompose_task,
|
||||
operation_type="decompose_task",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running strategize phase",
|
||||
plan_id=plan_id,
|
||||
)
|
||||
self.start_strategize(plan_id)
|
||||
# complete_strategize() calls auto_progress() which may
|
||||
# transition the plan to Execute/QUEUED.
|
||||
plan = self.complete_strategize(plan_id)
|
||||
plan = self.get_plan(plan_id)
|
||||
|
||||
# --- Execute phase: QUEUED → PROCESSING → COMPLETE --------------
|
||||
# (create_tool threshold gates the Execute phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.EXECUTE
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.create_tool < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running execute phase",
|
||||
plan_id=plan_id,
|
||||
)
|
||||
self.start_execute(plan_id)
|
||||
# complete_execute() calls auto_progress() which may
|
||||
# transition the plan to Apply/QUEUED.
|
||||
plan = self.complete_execute(plan_id)
|
||||
plan = self.get_plan(plan_id)
|
||||
|
||||
# --- Apply phase: QUEUED → PROCESSING → APPLIED (terminal) ------
|
||||
# (select_tool threshold gates the Apply phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.APPLY
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.select_tool < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running apply phase",
|
||||
@@ -2515,6 +2604,88 @@ class PlanLifecycleService:
|
||||
|
||||
return plan
|
||||
|
||||
def execute_async_job(self, job: Any, token: Any) -> None:
|
||||
"""Execute an async job, respecting automation profile gates.
|
||||
|
||||
This is the intended ``_job_executor`` callback for
|
||||
``AsyncWorker``. It checks the relevant automation profile
|
||||
threshold before starting any phase execution. Per the spec's
|
||||
Threshold Semantics:
|
||||
- threshold = 0.0 → always execute
|
||||
- threshold >= 1.0 → block (human approval required)
|
||||
- 0.0 < v < 1.0 → execute only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
job: The ``AsyncJob`` to execute.
|
||||
token: A ``CancellationToken`` for cooperative cancellation.
|
||||
"""
|
||||
phase = job.phase
|
||||
plan_id = job.plan_id
|
||||
|
||||
if token.is_cancelled:
|
||||
return
|
||||
|
||||
plan = self.get_plan(plan_id)
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
if phase == "strategize":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.decompose_task,
|
||||
operation_type="decompose_task",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="decompose_task",
|
||||
threshold_value=profile.decompose_task,
|
||||
)
|
||||
return
|
||||
self.start_strategize(plan_id)
|
||||
self.complete_strategize(plan_id)
|
||||
elif phase == "execute":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="create_tool",
|
||||
threshold_value=profile.create_tool,
|
||||
)
|
||||
return
|
||||
self.start_execute(plan_id)
|
||||
self.complete_execute(plan_id)
|
||||
elif phase == "apply":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="select_tool",
|
||||
threshold_value=profile.select_tool,
|
||||
)
|
||||
return
|
||||
self._complete_apply_if_queued(plan_id)
|
||||
else:
|
||||
self._logger.warning(
|
||||
"Unknown async job phase",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
)
|
||||
|
||||
def pause_plan(self, plan_id: str) -> Plan:
|
||||
"""Pause auto-progression by setting the automation profile to manual.
|
||||
|
||||
@@ -2659,7 +2830,10 @@ class PlanLifecycleService:
|
||||
|
||||
Called after ``constrain_apply`` when the automation profile
|
||||
permits automatic reversion. The ``access_network`` task-type
|
||||
threshold controls this gate (``access_network < 1.0``).
|
||||
threshold controls this gate per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-revert
|
||||
- 1.0 → never auto-revert (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-revert only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2677,7 +2851,11 @@ class PlanLifecycleService:
|
||||
return plan
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
if profile.access_network >= 1.0:
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.access_network,
|
||||
operation_type="access_network",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-reversion blocked by profile threshold",
|
||||
plan_id=plan_id,
|
||||
@@ -2712,7 +2890,10 @@ class PlanLifecycleService:
|
||||
|
||||
Called when validation failures block apply and the automation
|
||||
profile permits reversion. The ``delete_content`` task-type
|
||||
threshold controls this gate (strategy revision).
|
||||
threshold controls this gate per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-revert
|
||||
- 1.0 → never auto-revert (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-revert only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2727,7 +2908,11 @@ class PlanLifecycleService:
|
||||
return plan
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
if profile.delete_content >= 1.0:
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.delete_content,
|
||||
operation_type="delete_content",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Execute-to-Strategize reversion blocked by profile threshold",
|
||||
plan_id=plan_id,
|
||||
|
||||
@@ -102,6 +102,8 @@ class StdioTransport:
|
||||
cwd=self._cwd,
|
||||
)
|
||||
|
||||
self._process = None
|
||||
|
||||
try:
|
||||
self._process = subprocess.Popen(
|
||||
cmd,
|
||||
@@ -121,10 +123,10 @@ class StdioTransport:
|
||||
details={"command": self._command, "args": self._args},
|
||||
) from exc
|
||||
except OSError as exc:
|
||||
# Popen may have partially started the subprocess before
|
||||
# raising (e.g. execve failure post-fork on some platforms).
|
||||
# Ensure cleanup so the process does not leak into the caller's
|
||||
# address space.
|
||||
# Ensure any partially-started process is cleaned up.
|
||||
# Popen() may have spawned a child before the OSError was
|
||||
# raised (e.g. exec in a child fails), so we must terminate
|
||||
# and wait to avoid leaking a zombie or orphan.
|
||||
if self._process is not None:
|
||||
self.stop()
|
||||
self._process = None
|
||||
|
||||
@@ -740,7 +740,7 @@ class MCPToolAdapter:
|
||||
),
|
||||
]
|
||||
|
||||
properties: dict[str, Any] = input_schema.get("properties", {})
|
||||
properties: dict[str, Any] = input_schema.get("properties") or {}
|
||||
slots: list[ResourceSlot] = []
|
||||
seen_names: set[str] = set()
|
||||
|
||||
|
||||
@@ -179,13 +179,19 @@ class BaseResourceHandler:
|
||||
Raises:
|
||||
PermissionError: If *path* escapes the root via ``..`` or
|
||||
symlink resolution.
|
||||
|
||||
Uses :meth:`Path.relative_to` (not string prefix matching) to
|
||||
avoid the *prefix-collision bypass* exemplified by
|
||||
``startswith(root + os.sep)`` on symlinks and resolved paths.
|
||||
"""
|
||||
root = Path(location).resolve()
|
||||
target = (root / path).resolve()
|
||||
# Use root + os.sep to prevent prefix collision bypass:
|
||||
# e.g. root=/tmp/foo must not match target=/tmp/foobar/secret
|
||||
if target != root and not str(target).startswith(str(root) + os.sep):
|
||||
raise PermissionError(f"Path '{path}' escapes resource root '{location}'")
|
||||
try:
|
||||
target.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise PermissionError(
|
||||
f"Path '{path}' escapes resource root '{location}'"
|
||||
) from exc
|
||||
return target
|
||||
|
||||
def read(self, *, resource: Resource, path: str = "") -> Content:
|
||||
|
||||
@@ -69,6 +69,10 @@ def validate_sandbox_path(path_str: str, sandbox_root: str | None = None) -> Pat
|
||||
|
||||
Raises ``ValueError`` with the rejected path when traversal is
|
||||
detected or the resolved path falls outside the sandbox root.
|
||||
|
||||
Uses :meth:`Path.relative_to` (not string prefix matching) to avoid
|
||||
the *prefix-collision bypass*: a target like ``/tmp/abc123-escape``
|
||||
would incorrectly pass ``startswith("/tmp/abc123")``.
|
||||
"""
|
||||
if not path_str:
|
||||
raise ValueError("Path must not be empty")
|
||||
@@ -77,8 +81,12 @@ def validate_sandbox_path(path_str: str, sandbox_root: str | None = None) -> Pat
|
||||
root = root.resolve()
|
||||
|
||||
target = (root / path_str).resolve()
|
||||
if not str(target).startswith(str(root)):
|
||||
raise ValueError(f"Path traversal detected: '{path_str}' escapes sandbox root")
|
||||
try:
|
||||
target.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise ValueError(
|
||||
f"Path traversal detected: '{path_str}' escapes sandbox root"
|
||||
) from exc
|
||||
return target
|
||||
|
||||
|
||||
|
||||
@@ -161,10 +161,22 @@ def _normalise(path: str) -> str:
|
||||
|
||||
|
||||
def _is_under(path: str, root: str) -> bool:
|
||||
"""Return ``True`` if *path* is equal to or a child of *root*."""
|
||||
"""Return ``True`` if *path* is equal to or a child of *root*.
|
||||
|
||||
Uses semantic path containment via posixpath.relpath instead of
|
||||
string prefix matching (str.startswith). String prefix matching
|
||||
is vulnerable to sibling-directory prefix-collision attacks where
|
||||
/tmp/sandbox would incorrectly match /tmp/sandboxmalicious/file.
|
||||
|
||||
See issue #7478 — startswith bypass in path containment checks.
|
||||
"""
|
||||
if path == root:
|
||||
return True
|
||||
return path.startswith(root + "/")
|
||||
try:
|
||||
relative = posixpath.relpath(path, root)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
return not relative.startswith(".." + posixpath.sep) and relative != ".."
|
||||
|
||||
|
||||
def _relative_to(path: str, root: str) -> str:
|
||||
|
||||
@@ -145,7 +145,7 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
self._confirmed = False
|
||||
self._selected_actor = None
|
||||
self._visible = True
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
def hide(self) -> None:
|
||||
"""Hide the overlay and clear its content."""
|
||||
@@ -161,14 +161,14 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
if not self._filtered_actors:
|
||||
return
|
||||
self._selected_index = (self._selected_index - 1) % len(self._filtered_actors)
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
def move_down(self) -> None:
|
||||
"""Move the selection cursor down by one position (wraps)."""
|
||||
if not self._filtered_actors:
|
||||
return
|
||||
self._selected_index = (self._selected_index + 1) % len(self._filtered_actors)
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Search / filter
|
||||
@@ -191,7 +191,7 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
else:
|
||||
self._filtered_actors = list(self._actors)
|
||||
self._selected_index = 0
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Confirmation
|
||||
@@ -215,10 +215,10 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
return actor
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal rendering
|
||||
# Internal display refresh
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _render(self) -> None:
|
||||
def _refresh_display(self) -> None:
|
||||
content = render_actor_selection(
|
||||
self._filtered_actors,
|
||||
self._selected_index,
|
||||
|
||||
@@ -0,0 +1,342 @@
|
||||
"""Tests for ``StdioTransport`` in the LSP subsystem.
|
||||
|
||||
Covers subprocess lifecycle (spawn, termination), error handling for
|
||||
missing commands and OS-level failures, and defensive cleanup of
|
||||
partially-started processes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
class TestStdioTransportInit:
|
||||
"""Tests for transport construction."""
|
||||
|
||||
def test_init_defaults(self) -> None:
|
||||
"""Transport initialised with defaults stores provided values."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
assert transport._command == "test-server"
|
||||
assert transport._args == []
|
||||
assert transport._env == {}
|
||||
assert transport._process is None
|
||||
|
||||
def test_init_custom_args(self) -> None:
|
||||
"""Custom args, env and cwd are stored."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(
|
||||
command="pyright",
|
||||
args=["--stdio"],
|
||||
env={"PYRIGHT_PYTHON_FORCE_VERSION": "latest"},
|
||||
cwd="/project/root",
|
||||
)
|
||||
assert transport._command == "pyright"
|
||||
assert transport._args == ["--stdio"]
|
||||
assert transport._env == {"PYRIGHT_PYTHON_FORCE_VERSION": "latest"}
|
||||
assert transport._cwd == "/project/root"
|
||||
|
||||
def test_init_empty_command_raises(self) -> None:
|
||||
"""Empty command string raises :class:`ValueError`."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
with pytest.raises(ValueError, match="command must be a non-empty"):
|
||||
StdioTransport(command="") # type: ignore[arg-type]
|
||||
|
||||
def test_is_alive_false_before_start(self) -> None:
|
||||
"""Transport is not alive when no process has been spawned."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
assert transport.is_alive is False
|
||||
|
||||
|
||||
class TestStdioTransportStart:
|
||||
"""Tests for the :meth:`~StdioTransport.start` method."""
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_start_calls_popen_with_config(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""start() forwards command, args and config to Popen.
|
||||
|
||||
Verifies the merged env contains both system variables and
|
||||
user-provided overrides.
|
||||
"""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
mock_process = MagicMock()
|
||||
mock_process.pid = 42
|
||||
mock_popen.return_value = mock_process
|
||||
|
||||
transport = StdioTransport(
|
||||
command="test-server",
|
||||
args=["--stdio"],
|
||||
env={"FOO": "bar"},
|
||||
cwd="/tmp/workspace",
|
||||
)
|
||||
transport.start()
|
||||
|
||||
call_args = mock_popen.call_args
|
||||
assert call_args[0][0] == ["test-server", "--stdio"]
|
||||
assert call_args.kwargs["stdin"] is subprocess.PIPE
|
||||
assert call_args.kwargs["stdout"] is subprocess.PIPE
|
||||
assert call_args.kwargs["stderr"] is subprocess.PIPE
|
||||
assert call_args.kwargs["cwd"] == "/tmp/workspace"
|
||||
|
||||
merged_env = call_args.kwargs["env"]
|
||||
assert merged_env["FOO"] == "bar"
|
||||
|
||||
assert transport._process is not None
|
||||
assert transport.is_alive is True
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_start_already_started_raises(self, mock_popen: MagicMock) -> None:
|
||||
"""Calling start() on an already-alive transport raises RuntimeError."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
# Simulate a process that is still running
|
||||
mock_process = MagicMock()
|
||||
mock_process.pid = 100
|
||||
mock_process.poll.return_value = None
|
||||
mock_popen.return_value = mock_process
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
transport.start()
|
||||
|
||||
with pytest.raises(RuntimeError, match="already started"):
|
||||
transport.start()
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_start_on_dead_process_allows_restart(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""Transport restarts when the old process has already exited."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
# Simulate a previously-started but now-dead process
|
||||
mock_old_process = MagicMock()
|
||||
mock_old_process.poll.return_value = 1 # exited with code 1
|
||||
transport._process = mock_old_process
|
||||
|
||||
# is_alive should be False since poll() != None
|
||||
assert transport.is_alive is False
|
||||
|
||||
mock_new_process = MagicMock()
|
||||
mock_new_process.pid = 200
|
||||
mock_new_process.poll.return_value = None
|
||||
mock_popen.side_effect = [None, mock_new_process]
|
||||
# First call (from first failed start attempt with bad Popen) returns the old process,
|
||||
# second call returns new one. Let's use a simpler approach:
|
||||
transport._process = MagicMock() # reset to a clean dead process
|
||||
transport._process.poll.return_value = 1
|
||||
|
||||
mock_popen_new = MagicMock()
|
||||
mock_popen_new.pid = 200
|
||||
mock_popen_new.poll.return_value = None
|
||||
with patch("subprocess.Popen", return_value=mock_popen_new):
|
||||
transport.start()
|
||||
|
||||
assert transport._process is not None
|
||||
assert transport.is_alive is True
|
||||
|
||||
# --------------- FileNotFoundError ---------------
|
||||
|
||||
@patch("subprocess.Popen", side_effect=FileNotFoundError("no-command"))
|
||||
def test_start_file_not_found_raises_lsp_error(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""Missing command raises LspError with descriptive message.
|
||||
|
||||
Verifies _process is NOT set after the error (no orphan).
|
||||
"""
|
||||
from cleveragents.lsp.errors import LspError
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="nonexistent-cmd")
|
||||
with pytest.raises(
|
||||
LspError, match="command not found"
|
||||
) as exc_info:
|
||||
transport.start()
|
||||
|
||||
# _process must NOT be set after failure
|
||||
assert transport._process is None
|
||||
# Check error details contain useful information
|
||||
details = exc_info.value.details
|
||||
assert details["command"] == "nonexistent-cmd"
|
||||
|
||||
# --------------- OSError cleanup (the PR fix) ---------------
|
||||
|
||||
@patch("subprocess.Popen", side_effect=OSError("spawn failed"))
|
||||
def test_start_os_error_no_process_set(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""General OSError with _process=None raises LspError harmlessly."""
|
||||
from cleveragents.lsp.errors import LspError
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="some-cmd")
|
||||
with pytest.raises(LspError, match="Failed to start"):
|
||||
transport.start()
|
||||
|
||||
assert transport._process is None
|
||||
|
||||
def test_start_os_error_cleans_up_partial_process(
|
||||
self,
|
||||
) -> None:
|
||||
"""When Popen assigns _process then raises OSError, stop() cleans up.
|
||||
|
||||
This is the core regression guard for PR #11011. If
|
||||
subprocess.Popen() creates a child process before raising an
|
||||
OSError (e.g. exec fails in that child), the new code path must
|
||||
call :meth:`~StdioTransport.stop()` to terminate and reap it.
|
||||
|
||||
We simulate this scenario by patching Popen to return a mock
|
||||
process object, then raising OSError immediately after assignment.
|
||||
"""
|
||||
from cleveragents.lsp.errors import LspError
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
# Mock the process that Popen "returned" before OSError fired.
|
||||
mock_child = MagicMock()
|
||||
mock_child.pid = 9999
|
||||
mock_child.poll.return_value = None # still alive
|
||||
|
||||
transport = StdioTransport(command="dummy-cmd")
|
||||
|
||||
def spawn_that_fails(cmd, **kwargs):
|
||||
"""Mimics Popen: assign child then raise OSError."""
|
||||
transport._process = mock_child
|
||||
raise OSError("child exec failed after fork")
|
||||
|
||||
with patch(
|
||||
"subprocess.Popen", side_effect=spawn_that_fails
|
||||
):
|
||||
with pytest.raises(LspError) as exc_info:
|
||||
transport.start()
|
||||
|
||||
# The original Popen call should have been attempted once
|
||||
assert transport._process is None
|
||||
|
||||
|
||||
class TestStdioTransportStop:
|
||||
"""Tests for the :meth:`~StdioTransport.stop` method."""
|
||||
|
||||
def test_stop_returns_none_when_not_started(self) -> None:
|
||||
"""stop() returns None when no process was ever spawned."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
assert transport.stop() is None
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_stop_graceful_terminate(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""stop() terminates gracefully and clears _process."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
mock_process = MagicMock()
|
||||
mock_process.poll.return_value = None # running
|
||||
mock_process.terminate = MagicMock()
|
||||
mock_process.wait.return_value = None
|
||||
mock_process.returncode = 0
|
||||
|
||||
mock_popen.side_effect = lambda *a, **k: mock_process
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
transport.start()
|
||||
assert transport.is_alive is True
|
||||
|
||||
code = transport.stop()
|
||||
|
||||
mock_process.terminate.assert_called_once()
|
||||
assert code == 0
|
||||
assert transport._process is None
|
||||
assert transport.is_alive is False
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_stop_already_exited_returns_code(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""stop() on already-exited process returns exit code without terminate."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
mock_process = MagicMock()
|
||||
mock_process.poll.return_value = 142 # already exited
|
||||
mock_process.returncode = 142
|
||||
|
||||
mock_popen.side_effect = lambda *a, **k: mock_process
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
transport.start()
|
||||
|
||||
code = transport.stop()
|
||||
|
||||
mock_process.terminate.assert_not_called()
|
||||
assert code == 142
|
||||
assert transport._process is None
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_stop_force_kills_after_timeout(
|
||||
self, mock_popen: MagicMock
|
||||
) -> None:
|
||||
"""stop() force-kills when graceful terminate times out."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
mock_process = MagicMock()
|
||||
mock_process.poll.return_value = None
|
||||
mock_process.terminate = MagicMock()
|
||||
mock_process.kill = MagicMock()
|
||||
# First wait returns TimeoutExpired (graceful failed)
|
||||
mock_process.wait.side_effect = [subprocess.TimeoutExpired("test", 5.0), None]
|
||||
mock_process.returncode = -9
|
||||
|
||||
mock_popen.side_effect = lambda *a, **k: mock_process
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
transport.start()
|
||||
|
||||
code = transport.stop(timeout=1.0)
|
||||
|
||||
mock_process.terminate.assert_called_once()
|
||||
mock_process.kill.assert_called_once()
|
||||
assert code == -9
|
||||
|
||||
|
||||
class TestStdioTransportIO:
|
||||
"""Tests for message I/O methods."""
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_send_message_not_started(self, mock_popen: MagicMock) -> None:
|
||||
"""send_message raises RuntimeError before start()."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
with pytest.raises(RuntimeError, match="not started"):
|
||||
transport.send_message({"jsonrpc": "2.0"})
|
||||
|
||||
@patch("subprocess.Popen")
|
||||
def test_read_message_not_started(self, mock_popen: MagicMock) -> None:
|
||||
"""read_message raises RuntimeError before start()."""
|
||||
from cleveragents.lsp.transport import StdioTransport
|
||||
|
||||
transport = StdioTransport(command="test-server")
|
||||
with pytest.raises(RuntimeError, match="not started"):
|
||||
transport.read_message()
|
||||
|
||||
|
||||
class TestStdioTransportExports:
|
||||
"""Tests for module exports."""
|
||||
|
||||
def test_all_exports(self) -> None:
|
||||
"""StdioTransport is listed in __all__."""
|
||||
from cleveragents.lsp import transport
|
||||
|
||||
assert "StdioTransport" in transport.__all__
|
||||
Reference in New Issue
Block a user