Compare commits
21 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e2167ab8e7 | |||
| b3851693c8 | |||
| 23d73e7fb2 | |||
| 86f96f299e | |||
| 5c5309f35d | |||
| f4cea72248 | |||
| 761622f746 | |||
| bf52a9c648 | |||
|
97c1007bb5
|
|||
| b0b28623a1 | |||
| b4351ca78d | |||
| 94622f467c | |||
| 1baa888659 | |||
| 1196c726f2 | |||
| 655cd7ebc2 | |||
| dbc382f3d9 | |||
| fa95c518d5 | |||
| ef6829b6f8 | |||
| a37b4e0ccb | |||
| 4fdfee6150 | |||
| b41f536da6 |
+43
-2
@@ -5,6 +5,8 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
`plan_generation_graph.robot` to give more test answers.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
|
||||
requires whole-word closing keywords (avoids false positives like
|
||||
"prefixes #12"), TDD bug tag discovery now uses exact token matching
|
||||
@@ -40,6 +42,20 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
cleanup conflict with `cli_init_yes_flag_steps.py`); 2 new Behave
|
||||
scenarios covering multi-line PR description parsing and non-string
|
||||
`pr_diff` type guard.
|
||||
|
||||
- Added `TokenAuthMiddleware` and DI wiring to emit missing auth domain
|
||||
events (`AUTH_SUCCESS`, `AUTH_FAILURE`) during token checks, including
|
||||
spec-aligned audit details (`user_identity`, `attempted_identity`,
|
||||
`ip_address`, `token_prefix`, `failure_reason`) and best-effort publish
|
||||
behavior. Auth token prefixes now persist to audit logs without
|
||||
redaction-key collisions, and short tokens are masked (`***...`) to
|
||||
prevent full token disclosure. Added Behave coverage and Robot
|
||||
audit-pipeline integration tests for auth event persistence. (#714)
|
||||
|
||||
- Added TDD bug-capture E2E tests for bug #1028 — ACMS indexing pipeline not
|
||||
wired into CLI. Four Robot Framework E2E tests prove ContextTierService starts
|
||||
empty on every CLI invocation. Tests use ``@tdd_expected_fail`` until the bug
|
||||
fix is merged. (#1029)
|
||||
- Added Fix-then-Revalidate orchestration loop for required validations:
|
||||
bounded retry with configurable limits (0--100 per Safety Profile),
|
||||
strategy revision escalation via `auto_strategy_revision` float
|
||||
@@ -52,7 +68,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
counter, spec-required `validation_summary` and
|
||||
`final_validation_results` fields on the result model, DI container
|
||||
|
||||
## [Unreleased]
|
||||
- **fix(tui): rename ActorSelectionOverlay._render to _refresh_display (issue #11039)** — `ActorSelectionOverlay._render()` shadows Textual's `Widget._render()` which must return a `Strip`. In textual >=1.0, layout calls `get_content_height()` `self._render()` gets `None` `AttributeError: 'NoneType' object has no attribute 'get_height'`. Renamed the method to `_refresh_display()` and updated all four internal call sites (`show()`, `move_up()`, `move_down()`, `set_search()`) to use the new name.
|
||||
|
||||
- **Structural Component Output Validation** (#8164): Replaces exact character matching with structural component checking for output validation. Implements three validators covering plan tree output, decision CLI dicts, and structured session snapshots. The `validate_plan_tree` function validates node dicts for required keys (`decision_id`, `type`, `sequence`, `question`, `children`), ULID format, correct types, and sibling ordering. The `validate_decision_dict` function validates decision CLI output against the `Decision.as_cli_dict()` schema with field presence, type, ULID pattern, confidence range [0..1], and boolean field checks. The `validate_structured_output` function validates the StructuredOutput envelope for `command`, `session_id` (ULID), status membership, `exit_code`, and elements integrity. A unified dispatcher (`validate_structured_component_output`) enables routing by target_type. BDD test coverage added in `features/structural_validation.feature`. [Epic #8137](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8137)
|
||||
- **Fixed `agents actor add --config` crash with nested `actors:` map and `config.actor` combined shorthand** (#11189): The
|
||||
@@ -63,6 +79,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
when separate `provider`/`model` keys are absent. Added validation to reject malformed
|
||||
combined values with empty provider or model halves.
|
||||
|
||||
- **Fixed plan tree reporting zero decision nodes after strategize** (#10813): The `plan tree` command showed no ``decision_id`` fields even though planning completed successfully. Root cause: the PlanExecutor's ``run_strategize()`` method produced strategy decisions via the StrategyActor but never persisted them as domain ``Decision`` objects through the DecisionService wiring. Added ``decision_service`` parameter to the PlanExecutor constructor, wired it from the CLI dependency-injection container in ``_get_plan_executor()``, and added ``_persist_strategy_decisions()`` that converts each strategy decision into a domain Decision with correct type mapping (prompt_definition, strategy_choice, subplan_spawn) so they appear in plan tree output.
|
||||
- **`task-implementor` posts work-started notification comments** (#11031): Both
|
||||
the `issue_impl` and `pr_fix` procedures now post an informational "work
|
||||
started" comment to the Forgejo issue/PR before beginning implementation.
|
||||
@@ -105,6 +122,16 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
- **Plan Rollback Command** (#8557): Implemented `agents plan rollback <plan-id> [<checkpoint-id>]` for checkpoint-based plan state restoration in Epic #8493. The command restores a plan's sandbox to the state captured at a given checkpoint, discarding all decisions made after that checkpoint. The checkpoint can be specified as an optional positional second argument or via the `--to-checkpoint` named option. Supports `--yes/-y` flag to skip confirmation prompts and `--format/-f` for output format selection (rich/plain/json/yaml). Included with comprehensive BDD test coverage (>= 97%) and spec-aligned output formatting showing rollback summary, changes reverted, impact analysis, and post-rollback state panels.
|
||||
|
||||
### Fixed
|
||||
- **ACMS execute-phase assembler respects project-level hot_max_tokens** (#11035): Fixed
|
||||
``_resolve_hot_max_tokens()`` to read ``hot_max_tokens`` from
|
||||
``context_policy_json["acms_config"]["hot_max_tokens"]`` — the correct sub-key written
|
||||
by ``agents project context set --hot-max-tokens``. The previous implementation read
|
||||
from the top-level key (``config_dict.get("hot_max_tokens")``), which was always
|
||||
``None``, causing the assembler to silently fall back to the global 16K default even
|
||||
when a project-level override was configured. Also adds two Behave regression scenarios
|
||||
with ``@tdd_issue @tdd_issue_11035`` tags that exercise the DB query code path and
|
||||
verify the project-level budget is applied to ``CoreContextBudget`` and
|
||||
``ContextRequest``.
|
||||
- **Guard cleanup_stale against execute/processing and execute/complete plans** (#11121):
|
||||
``_create_sandbox_for_plan()`` in ``src/cleveragents/cli/commands/plan.py`` now
|
||||
skips ``GitWorktreeSandbox.cleanup_stale()`` when the plan is in
|
||||
@@ -137,6 +164,19 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
removal. Mocked existing steps to allow remaining V2 features to be
|
||||
covered/tested.
|
||||
|
||||
- **Automation profile threshold gates fully respect spec semantics** (#4328): Added
|
||||
``_should_auto_progress_for_threshold()`` helper in ``PlanLifecycleService`` that
|
||||
implements the full spec Threshold Semantics: ``0.0`` = always auto, ``1.0`` =
|
||||
always human approval, ``0.0 < v < 1.0`` = proceed only if confidence >= threshold.
|
||||
Integrated with ``AutonomyController.should_proceed_automatically()`` for
|
||||
intermediate thresholds. Updated ``should_auto_progress()``, ``try_auto_run()``,
|
||||
``execute_async_job()``, ``try_auto_revert_from_apply()``, and
|
||||
``try_auto_revert_from_execute()`` to use the helper. Previously the service
|
||||
only checked ``< 1.0`` (treated all intermediates as auto). Added BDD regression
|
||||
tests in ``features/tdd_automation_profile_gates_4328.feature`` covering all 8
|
||||
built-in profiles including the ``cautious`` profile's intermediate thresholds
|
||||
(e.g. ``create_tool=0.7``).
|
||||
|
||||
- **TUI Prompt Symbol Mode Awareness** (#6431): The prompt widget now displays a
|
||||
mode-dependent symbol (`❯` normal, `/` command, `$` shell, `☰` multi-line),
|
||||
implemented via `_PromptSymbolMixin` and `InputMode.MULTILINE`. The widget uses
|
||||
@@ -171,7 +211,8 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
were cleaned up in `features/actor_add_update_enforcement.feature` so the
|
||||
tests report correctly now that the underlying bug has been fixed.
|
||||
|
||||
- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
|
||||
- **Fixed `merge_invariants()` missing ACTION scope — 4-tier precedence restored** (#9126): Updated ``merge_invariants()`` and ``InvariantSet.merge()`` to accept a fourth parameter ``action_invariants`` alongside plan, project, and global tiers. The module docstring, ``InvariantScope`` docstring, and ``InvariantService.get_effective_invariants()`` now all reflect the correct precedence chain: ``plan > action > project > global``. Added ``action_name`` parameter to ``get_effective_invariants()`` so action-scoped invariants are collected and passed through the merge pipeline instead of silently dropped. All docstrings across both files were corrected from ``plan > project > global`` to ``plan > action > project > global``. Comprehensive Behave scenarios added covering four-tier merge precedence, action-before-project ordering, action override of project with same text, and effective invariant computation with all four scopes.
|
||||
|
||||
step texts to avoid case-sensitive collisions between different step modules that
|
||||
prevented all Behave tests from loading. Renamed steps in
|
||||
`edge_case_plan_steps.py`, `plan_executor_coverage_boost_steps.py`,
|
||||
|
||||
+4
-1
@@ -8,13 +8,14 @@
|
||||
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
|
||||
* Luis Mendes <luis.p.mendes@gmail.com>
|
||||
* Rui Hu <rui.hu@cleverthis.com>
|
||||
* HAL 9000 <hal9000@cleverthis.com> has contributed fix for #10813 — wiring DecisionService into PlanExecutor for strategy decision persistence during strategize.
|
||||
|
||||
# Details
|
||||
|
||||
Below are some of the specific details of various contributions.
|
||||
|
||||
* Jeffrey Phillips Freeman has acted as Lead Developer, daily contributor, and Project Owner.
|
||||
* Jeffrey Phillips Freeman has contributed an implementation for the invariant propagation fix (PR #10881 / issue #9131): added `_propagate_invariant_decisions()` to `SubplanService` to propagate all `invariant_enforced` decisions from parent plans to child plan decision trees during subplan spawn, satisfying the specification requirement for invariant propagation across hierarchical plan execution.
|
||||
* Jeffrey Phillips Freeman has contributed the invariant merge precedence fix (#9126): restored the missing ACTION scope in ``merge_invariants()`` and ``InvariantSet.merge()``, corrected all module docstrings from ``plan > project > global`` to the spec-compliant ``plan > action > project > global``, and added comprehensive BDD test coverage for four-tier merge precedence.
|
||||
* Brent E. Edwards has contributed quality assurance, test coverage, and CI pipeline improvements.
|
||||
* HAL 9000 has contributed automated implementation, bug fixes, and feature development as part of the CleverAgents automation pool.
|
||||
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
|
||||
@@ -44,4 +45,6 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
|
||||
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
|
||||
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
|
||||
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
|
||||
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
|
||||
* HAL 9000 has contributed the `ActorSelectionOverlay._render` → `_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
|
||||
|
||||
@@ -74,7 +74,7 @@ class MergeSmallSuite:
|
||||
|
||||
def time_merge_small(self) -> None:
|
||||
"""Benchmark merge with ~13 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeMediumSuite:
|
||||
@@ -88,7 +88,7 @@ class MergeMediumSuite:
|
||||
|
||||
def time_merge_medium(self) -> None:
|
||||
"""Benchmark merge with ~50 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeLargeSuite:
|
||||
@@ -102,7 +102,7 @@ class MergeLargeSuite:
|
||||
|
||||
def time_merge_large(self) -> None:
|
||||
"""Benchmark merge with ~250 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeDeduplicationSuite:
|
||||
@@ -137,7 +137,7 @@ class MergeDeduplicationSuite:
|
||||
|
||||
def time_merge_dedup(self) -> None:
|
||||
"""Benchmark merge with 60 invariants, all duplicates."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class InvariantSetMergeSuite:
|
||||
@@ -151,7 +151,7 @@ class InvariantSetMergeSuite:
|
||||
|
||||
def time_invariant_set_merge(self) -> None:
|
||||
"""Benchmark InvariantSet.merge()."""
|
||||
InvariantSet.merge(self.plan, self.project, self.global_invs)
|
||||
InvariantSet.merge(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class ServiceEffectiveSuite:
|
||||
|
||||
@@ -211,3 +211,24 @@ Feature: v3 Actor YAML schema support in CLI and registry
|
||||
Given a v3 graph actor config dict with skills and lsp as dict
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then each graph node agent should have lsp dict propagated
|
||||
|
||||
# M5: options block forwarded to agent_config in _build_from_v3 (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser propagates options block to agent config
|
||||
Given a v3 LLM actor config dict with options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should contain the options block
|
||||
|
||||
# M5: actor without options block is unaffected (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser handles v3 actor without options block
|
||||
Given a v3 LLM actor config dict without options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should not contain an options key
|
||||
|
||||
# M5: empty options dict is preserved (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser preserves empty options block
|
||||
Given a v3 LLM actor config dict with empty options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should contain an empty options block
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
Feature: Auth middleware emits security events
|
||||
As a platform operator
|
||||
I want authentication outcomes emitted as domain events
|
||||
So the audit pipeline captures auth success and failure details
|
||||
|
||||
Scenario: Successful authentication emits AUTH_SUCCESS
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_secret_123" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be true
|
||||
And the auth event bus should have exactly 1 event
|
||||
And the latest auth event type should be AUTH_SUCCESS
|
||||
And the latest auth event should contain detail "user_identity" with value "alice@example.com"
|
||||
And the latest auth event should contain detail "ip_address" with value "10.0.0.5"
|
||||
And the latest auth event should contain detail "token_prefix" with value "tok_se..."
|
||||
|
||||
Scenario: Short token success masks token_prefix
|
||||
Given an auth middleware with expected token "short"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "short" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be true
|
||||
And the latest auth event type should be AUTH_SUCCESS
|
||||
And the latest auth event should contain detail "token_prefix" with value "***..."
|
||||
|
||||
Scenario: Failed authentication emits AUTH_FAILURE
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_wrong_999" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be false
|
||||
And the auth event bus should have exactly 1 event
|
||||
And the latest auth event type should be AUTH_FAILURE
|
||||
And the latest auth event should contain detail "attempted_identity" with value "alice@example.com"
|
||||
And the latest auth event should contain detail "failure_reason" with value "invalid_token"
|
||||
And the latest auth event should contain detail "token_prefix" with value "tok_wr..."
|
||||
|
||||
Scenario: Missing configured token emits AUTH_FAILURE
|
||||
Given an auth middleware with no configured token
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_any_111" identity "bob@example.com" and ip "10.0.0.9"
|
||||
Then the auth result should be false
|
||||
And the latest auth event type should be AUTH_FAILURE
|
||||
And the latest auth event should contain detail "failure_reason" with value "token_not_configured"
|
||||
|
||||
Scenario: Empty authentication token is rejected before emission
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I try to authenticate with an empty token
|
||||
Then an auth ValueError should be raised
|
||||
And the auth event bus should have exactly 0 event
|
||||
|
||||
Scenario: Auth middleware events persist through AuditEventSubscriber
|
||||
Given auth middleware is wired to AuditEventSubscriber with expected token "tok_secret_123"
|
||||
When I authenticate with token "tok_secret_123" identity "carol@example.com" and ip "10.1.2.3"
|
||||
And I authenticate with token "tok_bad_123" identity "dave@example.com" and ip "10.1.2.4"
|
||||
Then the auth result history should be "true,false"
|
||||
And the audit log should contain 1 auth_success entry from middleware
|
||||
And the latest auth_success audit entry should contain detail "ip_address" with value "10.1.2.3"
|
||||
And the latest auth_success audit entry should contain detail "token_prefix" with value "tok_se..."
|
||||
And the audit log should contain 1 auth_failure entry from middleware
|
||||
And the latest auth_failure audit entry should contain detail "ip_address" with value "10.1.2.4"
|
||||
And the latest auth_failure audit entry should contain detail "attempted_identity" with value "dave@example.com"
|
||||
And the latest auth_failure audit entry should contain detail "failure_reason" with value "invalid_token"
|
||||
And the latest auth_failure audit entry should contain detail "token_prefix" with value "tok_ba..."
|
||||
@@ -418,7 +418,7 @@ Feature: Consolidated Domain Models
|
||||
Scenario: InvariantScope has four values
|
||||
Then InvariantScope should have values "global, project, action, plan"
|
||||
|
||||
# === Merge Precedence (plan > project > global) ===
|
||||
# === Merge Precedence (plan > action > project > global) ===
|
||||
|
||||
|
||||
Scenario: Merge with no duplicates preserves all invariants
|
||||
@@ -480,6 +480,75 @@ Feature: Consolidated Domain Models
|
||||
And the merged invariant at index 0 should have text "LOG ALL CHANGES"
|
||||
|
||||
|
||||
Scenario: Merge with all four scopes preserves action tier — Issue #9126
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Action rule | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Project rule | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
| Global rule | system |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 4 invariants
|
||||
And plan invariants appear before action invariants in merge
|
||||
And action invariants appear before project invariants in merge
|
||||
And the merged invariant at index 0 should have text "Plan rule"
|
||||
And the merged invariant at index 1 should have text "Action rule"
|
||||
And the merged invariant at index 2 should have text "Project rule"
|
||||
And the merged invariant at index 3 should have text "Global rule"
|
||||
|
||||
|
||||
Scenario: Action invariant overrides project with same text
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Log all changes | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Log all changes | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "action"
|
||||
|
||||
|
||||
Scenario: Action invariant is included in de-duplication with plan override
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Log all changes | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Log all changes | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "plan"
|
||||
|
||||
|
||||
Scenario: Inactive action invariants are excluded from merge
|
||||
Given I have plan invariants with an inactive entry
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Active | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "action"
|
||||
|
||||
|
||||
Scenario: Inactive invariants are excluded from merge
|
||||
Given I have plan invariants with an inactive entry
|
||||
And I have project invariants
|
||||
@@ -492,7 +561,7 @@ Feature: Consolidated Domain Models
|
||||
# === InvariantSet merge class method ===
|
||||
|
||||
|
||||
Scenario: InvariantSet.merge produces correct result
|
||||
Scenario: InvariantSet.merge produces correct result
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
@@ -505,7 +574,25 @@ Feature: Consolidated Domain Models
|
||||
When I merge using InvariantSet
|
||||
Then the invariant set should have 3 invariants
|
||||
|
||||
# === Service: Add/List/Remove ===
|
||||
|
||||
Scenario: InvariantSet.merge with four-tier precedence — Issue #9126
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Action rule | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Project rule | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
| Global rule | system |
|
||||
When I merge using InvariantSet
|
||||
Then the invariant set should have 4 invariants
|
||||
|
||||
|
||||
# === Service: Add/List/Remove ===
|
||||
|
||||
|
||||
Scenario: Service add invariant
|
||||
@@ -574,6 +661,16 @@ Feature: Consolidated Domain Models
|
||||
And the effective set should contain global invariants last
|
||||
|
||||
|
||||
Scenario: Effective invariants include action scope — Issue #9126
|
||||
Given an invariant service with invariants at all four scopes
|
||||
When I get effective invariants for plan "plan1", action "action1", and project "proj1"
|
||||
Then the effective set should contain plan invariants first
|
||||
And the effective set should contain action invariants second
|
||||
And the effective set should contain project invariants after action
|
||||
And the effective set should contain global invariants last
|
||||
And the effective set should have 4 invariants
|
||||
|
||||
|
||||
Scenario: Effective invariants de-duplicate across scopes
|
||||
Given an invariant service with duplicate text across scopes
|
||||
When I get effective invariants for plan "plan1" and project "proj1"
|
||||
|
||||
@@ -1379,3 +1379,24 @@ Feature: Consolidated Routing
|
||||
When the stream router tool agent processes "keep_me" through operation
|
||||
Then the stream router tool agent operation result should be "keep_me"
|
||||
|
||||
# M5: SimpleLLMAgent._resolve_llm forwards options to registry.create_llm (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent forwards options block to LLM constructor
|
||||
Given a stream router llm agent with options block containing custom base url
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should have received the custom base url
|
||||
|
||||
# M5: actor without options block is unaffected (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent without options block calls LLM constructor without extra kwargs
|
||||
Given a stream router llm agent without options block
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should not have received extra kwargs
|
||||
|
||||
# M5: top-level keys take precedence over duplicates in options (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent top-level key takes precedence over options duplicate
|
||||
Given a stream router llm agent with top-level temperature and options block
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should have received the top-level temperature
|
||||
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
Feature: Database URL sanitisation — credentials must never be exposed
|
||||
|
||||
As a CleverAgents operator
|
||||
I want database URLs in CLI output to have their credentials masked
|
||||
So that running ``agents info`` never leaks passwords or API tokens
|
||||
|
||||
@tdd_bug_8395
|
||||
Scenario Outline: PostgreSQL URL with user and password — all are masked
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| postgresql://user:secret@localhost/mydb | postgresql://***:***@localhost/mydb |
|
||||
| postgresql://admin:p%40ssw0rd@db.example.com:5432/agents | postgresql://***:***@db.example.com:5432/agents |
|
||||
| postgresql://readonly:r0ad_only@pg.cluster.internal:5433/production | postgresql://***:***@pg.cluster.internal:5433/production |
|
||||
|
||||
Scenario Outline: MySQL URL with credentials — all are masked
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| mysql://app:s3cret@mysql.internal:3306/agents | mysql://***:***@mysql.internal:3306/agents |
|
||||
| mysql+pymysql://root:toor@localhost/testdb | mysql+pymysql://***:***@localhost/testdb |
|
||||
|
||||
Scenario Outline: SQLite URLs — remain unchanged (no credentials)
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| sqlite:///data/cleveragents.db | sqlite:///data/cleveragents.db |
|
||||
| sqlite:////absolute/path/to/db.sqlite | sqlite:////absolute/path/to/db.sqlite |
|
||||
| memory | memory |
|
||||
|
||||
Scenario: Username-only URL — password is still masked
|
||||
Given the database url is "postgres://deploy@db.example.com/prod"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "postgres://***:***@db.example.com/prod"
|
||||
|
||||
Scenario: SQLite URL without credentials remains unchanged
|
||||
Given the database url is "sqlite:///test.db"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "sqlite:///test.db"
|
||||
|
||||
Scenario: build_info_data returns sanitised database URL
|
||||
Given a mock settings object with database url "postgresql://admin:supersecret@host.example.com:5432/mydb"
|
||||
When I call build_info_data
|
||||
Then the database field in info data should be "postgresql://***:***@host.example.com:5432/mydb"
|
||||
@@ -235,3 +235,21 @@ Feature: Execute-phase context assembler coverage
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the assembled result should be an AssembledContext
|
||||
And epcov the assembled result should have fragments and metadata
|
||||
|
||||
# ---- _resolve_hot_max_tokens: reads context_policy_json from DB ----
|
||||
|
||||
@tdd_issue @tdd_issue_11035
|
||||
Scenario: epcov assemble uses project-level hot_max_tokens from context_policy_json
|
||||
Given epcov an assembler with context_policy_json hot_max_tokens 32000
|
||||
And epcov a plan with project links
|
||||
And epcov scoped fragments that pass all filters
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the pipeline received budget max_tokens of 32000
|
||||
|
||||
@tdd_issue @tdd_issue_11035
|
||||
Scenario: epcov assemble falls back to global hot_max_tokens when context_policy_json has no override
|
||||
Given epcov an assembler with no hot_max_tokens in context_policy_json
|
||||
And epcov a plan with project links
|
||||
And epcov scoped fragments that pass all filters
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the pipeline received budget max_tokens of 4096
|
||||
|
||||
@@ -9,6 +9,7 @@ from .fake_provider import FakeProviderInfo, FakeProviderRegistry
|
||||
from .lsp_transport_mock import MockLspTransport, parse_lsp_responses
|
||||
from .mock_ai_provider import MockAIProvider
|
||||
from .mock_mcp_transport import MockMCPTransport
|
||||
from .recording_event_bus import RecordingEventBus
|
||||
from .transient_fail_audit_service import TransientFailAuditService
|
||||
|
||||
# NOTE: tdd_test_helpers is NOT re-exported here because it imports
|
||||
@@ -22,6 +23,7 @@ __all__ = [
|
||||
"MockAIProvider",
|
||||
"MockLspTransport",
|
||||
"MockMCPTransport",
|
||||
"RecordingEventBus",
|
||||
"TransientFailAuditService",
|
||||
"parse_lsp_responses",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Minimal in-memory event bus used for Behave assertions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
class RecordingEventBus:
|
||||
"""Minimal in-memory event bus used for Behave assertions."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(
|
||||
self,
|
||||
event_type: EventType,
|
||||
handler: Callable[[DomainEvent], None],
|
||||
) -> None:
|
||||
_ = (event_type, handler)
|
||||
@@ -0,0 +1,139 @@
|
||||
Feature: Plan Lifecycle Service coverage boost round 3
|
||||
As a developer
|
||||
I want to exercise remaining uncovered code paths in PlanLifecycleService
|
||||
So that code coverage improves beyond the current per-file threshold
|
||||
|
||||
Background:
|
||||
Given I have a fresh plan lifecycle service for coverage boost r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_strategize — event_bus.emit PLAN_STATE_CHANGED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_strategize emits PLAN_STATE_CHANGED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/strat-complete-event" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I complete strategize on the plan with event bus
|
||||
Then the event bus should have recorded a PLAN_STATE_CHANGED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_strategize — event_bus.emit PLAN_STATE_CHANGED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_strategize catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/strat-complete-fail" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I complete strategize on the plan and event bus emit fails
|
||||
Then the plan phase should be "strategize" and state should be "complete"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_strategize — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_strategize emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/strat-fail-event" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I fail strategize on the plan with error "Test strategy error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_strategize — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_strategize catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/strat-fail-fail" exists for coverage boost r3
|
||||
And a plan in strategize phase with processing state for r3
|
||||
When I fail strategize on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_execute — event_bus.emit PLAN_STATE_CHANGED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_execute emits PLAN_STATE_CHANGED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/exec-complete-event" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I complete execute on the plan with event bus
|
||||
Then the event bus should have recorded a PLAN_STATE_CHANGED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# complete_execute — event_bus.emit PLAN_STATE_CHANGED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: complete_execute catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/exec-complete-fail" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I complete execute on the plan and event bus emit fails
|
||||
Then the plan phase should be "execute" and state should be "complete"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/exec-fail-event" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan with error "Test execution error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/exec-fail-fail" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — event_bus.emit PLAN_ERRORED success
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply emits PLAN_ERRORED event via event bus
|
||||
Given a plan lifecycle service with a recording event bus for r3
|
||||
And an action "local/apply-fail-event" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan with error "Test apply error"
|
||||
Then the event bus should have recorded a PLAN_ERRORED event
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — event_bus.emit PLAN_ERRORED exception
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply catches event bus emission failure
|
||||
Given a plan lifecycle service with a failing event bus for r3
|
||||
And an action "local/apply-fail-fail" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan and event bus emit fails
|
||||
Then the plan should be in errored state for r3
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_execute — _cleanup_devcontainers called on terminal failure
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_execute triggers devcontainer cleanup on terminal failure
|
||||
Given a plan lifecycle service with mocked devcontainer cleanup for r3
|
||||
And an action "local/exec-fail-cleanup" exists for coverage boost r3
|
||||
And a plan in execute phase with processing state for r3
|
||||
When I fail execute on the plan with error "Container cleanup test"
|
||||
Then the plan should be in errored state and cleanup should have been called
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# fail_apply — _cleanup_devcontainers called on terminal failure
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: fail_apply triggers devcontainer cleanup on terminal failure
|
||||
Given a plan lifecycle service with mocked devcontainer cleanup for r3
|
||||
And an action "local/apply-fail-cleanup" exists for coverage boost r3
|
||||
And a plan in apply phase with processing state for r3
|
||||
When I fail apply on the plan with error "Container cleanup test"
|
||||
Then the plan should be in errored state and cleanup should have been called
|
||||
@@ -0,0 +1,163 @@
|
||||
Feature: Plan Lifecycle Service coverage boost round 4
|
||||
As a developer
|
||||
I want to exercise remaining uncovered code paths in PlanLifecycleService
|
||||
So that code coverage improves to meet the 97% threshold
|
||||
|
||||
Background:
|
||||
Given I have a fresh plan lifecycle service for coverage boost r4
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_estimation — event_bus.emit PLAN_ESTIMATION_COMPLETE
|
||||
# coverage (via execute_plan when estimation_actor is configured)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan with estimation_actor succeeds and emits PLAN_ESTIMATION_COMPLETE
|
||||
Given an action with estimation_actor configured for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a recording event bus for r4
|
||||
When I execute the plan for r4
|
||||
Then the event bus should have recorded PLAN_ESTIMATION_COMPLETE event
|
||||
And the plan should be in execute phase
|
||||
|
||||
Scenario: execute_plan with estimation_actor catches event_bus emit failure for PLAN_ESTIMATION_COMPLETE
|
||||
Given an action with estimation_actor configured for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan for r4
|
||||
Then the plan should be in execute phase despite estimation emit failure
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_estimation — cost_estimate_usd assignment (line 376)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan populates cost_estimate_usd when estimation returns cost
|
||||
Given an action with estimation_actor returning cost for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
And the plan lifecycle service has a recording event bus for r4
|
||||
When I execute the plan for r4 with mocked estimation
|
||||
Then the plan cost_estimate_usd should be populated
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# execute_plan — lock acquire/release when lock_service is configured
|
||||
# Lines 1616-1621 (acquire) and 1684-1688 (finally release)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: execute_plan acquires and releases plan lock when lock_service is configured
|
||||
Given an action for execute with lock service for r4
|
||||
And a plan in strategize-complete state for r4
|
||||
When I execute the plan with lock service for r4
|
||||
Then the plan should be in execute phase
|
||||
And the lock service should have acquired and released the plan lock
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# apply_plan — lock acquire/release when lock_service is configured
|
||||
# Lines 1830-1835 (acquire) and 1872-1877 (finally release)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: apply_plan acquires and releases plan lock when lock_service is configured
|
||||
Given an action for apply with lock service for r4
|
||||
And a plan in execute-complete state for r4
|
||||
When I apply the plan with lock service for r4
|
||||
Then the plan should be in apply phase
|
||||
And the lock service should have acquired and released the plan lock
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _subscribe_correction_reconciliation — event_bus.subscribe failure
|
||||
# Lines 562-566
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: lifecycle service handles event_bus subscribe failure gracefully
|
||||
Given I have a plan lifecycle service with event_bus that fails on subscribe for r4
|
||||
When the service is created for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# prompt_plan — validation scenarios
|
||||
# Lines 2082 (blank guidance) and 2086-2097 (wrong phase/state)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: prompt_plan rejects blank guidance with ValidationError
|
||||
Given a plan in execute phase with processing state for prompt_plan r4
|
||||
When I prompt the plan with blank guidance for r4
|
||||
Then a ValidationError should be raised for blank guidance
|
||||
|
||||
Scenario: prompt_plan rejects plan not in execute phase
|
||||
Given a plan in strategize phase for prompt_plan r4
|
||||
When I prompt the plan with guidance for wrong phase for r4
|
||||
Then a PlanError should be raised for wrong phase
|
||||
|
||||
Scenario: prompt_plan rejects plan in complete state
|
||||
Given a plan in execute-complete state for prompt_plan r4
|
||||
When I prompt the plan with guidance for complete state for r4
|
||||
Then a PlanError should be raised for wrong state
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# prompt_plan — state transition and decision_service interaction
|
||||
# Lines 2100-2154
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: prompt_plan transitions from errored to processing and clears error_message
|
||||
Given a plan in execute phase with errored state for prompt_plan r4
|
||||
When I prompt the plan with guidance to resume from errored state for r4
|
||||
Then the plan processing_state should be processing
|
||||
And the plan error_message should be cleared
|
||||
|
||||
Scenario: prompt_plan handles decision_service exception gracefully
|
||||
Given a plan in execute phase with processing state for prompt_plan r4
|
||||
And the plan lifecycle service has a failing decision service for r4
|
||||
When I prompt the plan with guidance for r4
|
||||
Then the plan processing_state should be processing
|
||||
And no exception should propagate
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _run_invariant_reconciliation — event_bus.emit exceptions
|
||||
# Lines 499-505 (success emit failure) and 530-536 (failure emit failure)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: _run_invariant_reconciliation success path catches event_bus emit failure
|
||||
Given an action with invariant_actor for reconciliation success r4
|
||||
And a plan in strategize-complete state with project links for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan for invariant reconciliation emit failure r4
|
||||
Then the plan should be in execute phase
|
||||
And the event_bus should have been called for INVARIANT_RECONCILED emit failure
|
||||
|
||||
Scenario: _run_invariant_reconciliation failure path catches event_bus emit failure
|
||||
Given an action with invariant_actor causing reconciliation failure for r4
|
||||
And a plan in strategize-complete state with project links for r4
|
||||
And the plan lifecycle service has a failing event bus for r4
|
||||
When I execute the plan expecting reconciliation failure for r4
|
||||
Then ReconciliationBlockedError should be raised
|
||||
And the event_bus should have been called for INVARIANT_VIOLATED emit failure
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Unknown automation profile validation
|
||||
# Lines 1229-1233
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: use_action rejects unknown automation profile with ValidationError
|
||||
Given an action with unknown automation profile for r4
|
||||
When I use the action to create a plan for r4
|
||||
Then a ValidationError should be raised for unknown profile
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# _handle_correction_applied — event handler scenarios
|
||||
# Lines 577 (plan_id None guard), 584-595 (exception handlers)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: _handle_correction_applied skips event with no plan_id
|
||||
Given the plan lifecycle service has a recording event bus for correction handler r4
|
||||
When I handle a CORRECTION_APPLIED event with no plan_id for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
Scenario: _handle_correction_applied handles ReconciliationBlockedError gracefully
|
||||
Given a plan in strategize phase with processing state for r4
|
||||
And the invariant reconciliation raises ReconciliationBlockedError for r4
|
||||
When I handle a CORRECTION_APPLIED event for blocked reconciliation for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
Scenario: _handle_correction_applied handles general exception from get_plan gracefully
|
||||
Given the plan lifecycle service is configured for r4
|
||||
And get_plan will fail for the plan for r4
|
||||
When I handle a CORRECTION_APPLIED event when get_plan fails for r4
|
||||
Then no exception should be raised for r4
|
||||
@@ -14,6 +14,7 @@ from unittest.mock import patch
|
||||
|
||||
import yaml
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.actor.compiler import ActorCompilationError
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
@@ -781,3 +782,77 @@ def step_graph_nodes_have_lsp_dict(context: Any) -> None:
|
||||
assert lsp.get("auto") is True, (
|
||||
f"Node '{node_id}' expected lsp.auto=True, got {lsp}"
|
||||
)
|
||||
|
||||
|
||||
# ── M5: options block forwarding (#11223) ─────────────────────────────────
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict with options block")
|
||||
def step_v3_llm_config_with_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/my-llama-actor",
|
||||
"type": "llm",
|
||||
"description": "Local llama.cpp actor",
|
||||
"provider": "openai",
|
||||
"model": "my-local-model",
|
||||
"options": {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
"temperature": 1.0,
|
||||
},
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict without options block")
|
||||
def step_v3_llm_config_without_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/standard-actor",
|
||||
"type": "llm",
|
||||
"description": "Standard actor without options",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@then("the agent config should contain the options block")
|
||||
def step_agent_config_has_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
options = agent.config.get("options")
|
||||
assert options == {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
"temperature": 1.0,
|
||||
}, f"Expected exact options dict, got {options}"
|
||||
|
||||
|
||||
@then("the agent config should not contain an options key")
|
||||
def step_agent_config_no_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
assert "options" not in agent.config, (
|
||||
f"Expected no 'options' key in agent config, got {agent.config}"
|
||||
)
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict with empty options block")
|
||||
def step_v3_llm_config_with_empty_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/empty-options-actor",
|
||||
"type": "llm",
|
||||
"description": "Actor with empty options",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"options": {},
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@then("the agent config should contain an empty options block")
|
||||
def step_agent_config_has_empty_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
options = agent.config.get("options")
|
||||
assert options == {}, f"Expected empty options dict, got {options}"
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Step definitions for auth_middleware_events.feature."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from cleveragents.application.services.audit_event_subscriber import (
|
||||
AuditEventSubscriber,
|
||||
)
|
||||
from cleveragents.application.services.audit_service import AuditService
|
||||
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.infrastructure.database.models import Base
|
||||
from cleveragents.infrastructure.events.reactive import ReactiveEventBus
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
from features.mocks.recording_event_bus import RecordingEventBus
|
||||
|
||||
|
||||
def _fresh_audit_service() -> AuditService:
|
||||
Settings._instance = None
|
||||
settings = Settings(database_url="sqlite:///:memory:")
|
||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
Base.metadata.create_all(engine)
|
||||
session = sessionmaker(bind=engine)()
|
||||
return AuditService(settings=settings, session=session)
|
||||
|
||||
|
||||
@given('an auth middleware with expected token "{token}"')
|
||||
def step_auth_middleware_with_token(context: Context, token: str) -> None:
|
||||
context.expected_token = token
|
||||
|
||||
|
||||
@given("an auth middleware with no configured token")
|
||||
def step_auth_middleware_no_token(context: Context) -> None:
|
||||
context.expected_token = None
|
||||
|
||||
|
||||
@given("an auth recording event bus")
|
||||
def step_auth_recording_event_bus(context: Context) -> None:
|
||||
context.event_bus = RecordingEventBus()
|
||||
context.auth_middleware = TokenAuthMiddleware(
|
||||
expected_token=context.expected_token,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_result_history = []
|
||||
|
||||
|
||||
@given('auth middleware is wired to AuditEventSubscriber with expected token "{token}"')
|
||||
def step_auth_middleware_audit_pipeline(context: Context, token: str) -> None:
|
||||
context.audit_service = _fresh_audit_service()
|
||||
context.event_bus = ReactiveEventBus()
|
||||
context.audit_subscriber = AuditEventSubscriber(
|
||||
audit_service=context.audit_service,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_middleware = TokenAuthMiddleware(
|
||||
expected_token=token,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_result_history = []
|
||||
|
||||
|
||||
@when('I authenticate with token "{token}" identity "{identity}" and ip "{ip_address}"')
|
||||
def step_authenticate(
|
||||
context: Context, token: str, identity: str, ip_address: str
|
||||
) -> None:
|
||||
context.auth_result = context.auth_middleware.authenticate(
|
||||
token,
|
||||
identity=identity,
|
||||
ip_address=ip_address,
|
||||
)
|
||||
context.auth_result_history.append(context.auth_result)
|
||||
|
||||
|
||||
@when("I try to authenticate with an empty token")
|
||||
def step_try_authenticate_empty_token(context: Context) -> None:
|
||||
context.auth_error = None
|
||||
try:
|
||||
context.auth_middleware.authenticate("", identity="empty@example.com")
|
||||
except ValueError as exc:
|
||||
context.auth_error = exc
|
||||
|
||||
|
||||
@then("the auth result should be true")
|
||||
def step_auth_result_true(context: Context) -> None:
|
||||
assert context.auth_result is True
|
||||
|
||||
|
||||
@then("the auth result should be false")
|
||||
def step_auth_result_false(context: Context) -> None:
|
||||
assert context.auth_result is False
|
||||
|
||||
|
||||
@then("the auth event bus should have exactly {count:d} event")
|
||||
def step_auth_event_count(context: Context, count: int) -> None:
|
||||
assert len(context.event_bus.events) == count, (
|
||||
f"Expected {count} events, got {len(context.event_bus.events)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the latest auth event type should be {event_type_name}")
|
||||
def step_latest_auth_event_type(context: Context, event_type_name: str) -> None:
|
||||
expected = EventType[event_type_name]
|
||||
actual = context.event_bus.events[-1].event_type
|
||||
assert actual == expected, f"Expected event type {expected}, got {actual}"
|
||||
|
||||
|
||||
@then(
|
||||
'the latest auth event should contain detail "{key}" with value "{expected_value}"'
|
||||
)
|
||||
def step_latest_auth_event_detail(
|
||||
context: Context, key: str, expected_value: str
|
||||
) -> None:
|
||||
details = context.event_bus.events[-1].details
|
||||
assert details.get(key) == expected_value, (
|
||||
f"Expected {key}={expected_value!r}, got {details.get(key)!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("an auth ValueError should be raised")
|
||||
def step_auth_value_error(context: Context) -> None:
|
||||
assert context.auth_error is not None, (
|
||||
"Expected ValueError but no exception was raised"
|
||||
)
|
||||
assert isinstance(context.auth_error, ValueError), (
|
||||
f"Expected ValueError, got {type(context.auth_error).__name__}"
|
||||
)
|
||||
|
||||
|
||||
@then('the auth result history should be "{history_csv}"')
|
||||
def step_auth_result_history(context: Context, history_csv: str) -> None:
|
||||
expected = [part.strip().lower() == "true" for part in history_csv.split(",")]
|
||||
assert context.auth_result_history == expected, (
|
||||
f"Expected history {expected}, got {context.auth_result_history}"
|
||||
)
|
||||
|
||||
|
||||
@then("the audit log should contain 1 auth_success entry from middleware")
|
||||
def step_audit_has_auth_success(context: Context) -> None:
|
||||
entries = context.audit_service.list_entries(event_type="auth_success")
|
||||
assert len(entries) == 1, f"Expected 1 auth_success entry, got {len(entries)}"
|
||||
|
||||
|
||||
@then("the audit log should contain 1 auth_failure entry from middleware")
|
||||
def step_audit_has_auth_failure(context: Context) -> None:
|
||||
entries = context.audit_service.list_entries(event_type="auth_failure")
|
||||
assert len(entries) == 1, f"Expected 1 auth_failure entry, got {len(entries)}"
|
||||
|
||||
|
||||
@then(
|
||||
'the latest {audit_event_type} audit entry should contain detail "{key}" with value "{expected_value}"'
|
||||
)
|
||||
def step_latest_audit_entry_detail(
|
||||
context: Context, audit_event_type: str, key: str, expected_value: str
|
||||
) -> None:
|
||||
entries = context.audit_service.list_entries(event_type=audit_event_type)
|
||||
assert entries, f"Expected at least one {audit_event_type} audit entry"
|
||||
details = entries[0].details
|
||||
assert details.get(key) == expected_value, (
|
||||
f"Expected {audit_event_type}.{key}={expected_value!r}, "
|
||||
f"got {details.get(key)!r}"
|
||||
)
|
||||
@@ -52,6 +52,7 @@ def _profile_name_for_level(level: str) -> str:
|
||||
"manual": "manual",
|
||||
"review_before_apply": "auto",
|
||||
"full_automation": "full-auto",
|
||||
"cautious": "cautious",
|
||||
}
|
||||
return mapping.get(level, level)
|
||||
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Step definitions for db_url_sanitisation.feature.
|
||||
|
||||
Tests the ``_sanitise_db_url`` helper and its use in ``build_info_data``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from tempfile import mkdtemp
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('the database url is "{url}"')
|
||||
def step_db_url_given(context: Context, url: str) -> None:
|
||||
"""Store the raw database URL for processing."""
|
||||
context.raw_db_url = url
|
||||
|
||||
|
||||
@given('a mock settings object with database url "{db_url}"')
|
||||
def step_mock_settings_with_db_url(context: Context, db_url: str) -> None:
|
||||
"""Build a full mock Settings that ``build_info_data`` expects."""
|
||||
tmpdir = mkdtemp()
|
||||
mock_settings = MagicMock()
|
||||
mock_settings.database_url = db_url
|
||||
mock_settings.data_dir = Path(tmpdir)
|
||||
mock_settings.storage_path = Path(tmpdir)
|
||||
mock_settings.config_path = Path(tmpdir) / "config.toml"
|
||||
mock_settings.log_dir = Path(tmpdir) / "logs"
|
||||
mock_settings.default_automation_profile = "auto"
|
||||
mock_settings.has_provider_configured = MagicMock(return_value=False)
|
||||
mock_settings.configured_provider_names = MagicMock(return_value=[])
|
||||
mock_settings.debug_enabled = False
|
||||
context.mock_settings = mock_settings
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I run sanitise_db_url")
|
||||
def step_run_sanitise(context: Context) -> None:
|
||||
"""Call _sanitise_db_url with the stored raw URL."""
|
||||
from cleveragents.cli.commands.system import _sanitise_db_url
|
||||
|
||||
context.sanitised_url = _sanitise_db_url(context.raw_db_url)
|
||||
|
||||
|
||||
@when("I call build_info_data")
|
||||
def step_call_build_info(context: Context) -> None:
|
||||
"""Call ``build_info_data`` with the mock settings."""
|
||||
from cleveragents.cli.commands.system import build_info_data
|
||||
|
||||
with patch(
|
||||
"cleveragents.config.settings.get_settings",
|
||||
return_value=context.mock_settings,
|
||||
):
|
||||
context.info_data = build_info_data()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the sanitised database url should be "{expected}"')
|
||||
def step_assert_sanitised_url(context: Context, expected: str) -> None:
|
||||
"""Verify the sanitised URL matches expectation."""
|
||||
actual = context.sanitised_url
|
||||
assert actual == expected, f"Expected '{expected}', got '{actual}'"
|
||||
|
||||
|
||||
@then('the database field in info data should be "{expected}"')
|
||||
def step_assert_info_db_field(context: Context, expected: str) -> None:
|
||||
"""Verify the sanitised URL appears correctly in build_info_data output."""
|
||||
actual = context.info_data["database"]
|
||||
assert actual == expected, f"Expected database field '{expected}', got '{actual}'"
|
||||
@@ -12,6 +12,7 @@ Exercises all uncovered lines in
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
@@ -937,3 +938,112 @@ def step_epcov_assembled_has_data(context: Context) -> None:
|
||||
assert result.budget_used >= 0.0, "Expected non-negative budget_used"
|
||||
assert result.context_hash, "Expected non-empty context_hash"
|
||||
assert result.strategies_used, "Expected non-empty strategies_used"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _resolve_hot_max_tokens: reads hot_max_tokens from context_policy_json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_assembler_with_policy_json(
|
||||
policy_json: str | None,
|
||||
) -> ACMSExecutePhaseContextAssembler:
|
||||
"""Build an assembler whose DB session returns a row with *policy_json*."""
|
||||
pr = _make_pipeline_result()
|
||||
pr.fragments = ()
|
||||
pr.total_tokens = 5
|
||||
pr.budget_used = 0.1
|
||||
pr.strategies_used = ("relevance",)
|
||||
pr.context_hash = "hash-policy"
|
||||
pr.preamble = None
|
||||
pr.provenance_map = {}
|
||||
|
||||
mock_pipeline = MagicMock()
|
||||
mock_pipeline.assemble.return_value = pr
|
||||
|
||||
tier_service = MagicMock()
|
||||
# Return a minimal passing fragment so assemble() does not short-circuit
|
||||
frag = _make_tiered_fragment(
|
||||
fragment_id="frag-policy",
|
||||
content="content",
|
||||
token_count=5,
|
||||
metadata={"path": "src/good.py"},
|
||||
)
|
||||
tier_service.get_scoped_view.return_value = [frag]
|
||||
|
||||
# Build a mock row with context_policy_json
|
||||
mock_row = MagicMock()
|
||||
mock_row.context_policy_json = policy_json
|
||||
|
||||
# Mock the session so _resolve_hot_max_tokens can query it
|
||||
mock_session = MagicMock()
|
||||
mock_session.query.return_value.filter_by.return_value.first.return_value = mock_row
|
||||
|
||||
repo = MagicMock()
|
||||
repo._session.return_value = mock_session
|
||||
repo.get_context_policy.return_value = ProjectContextPolicy()
|
||||
|
||||
return ACMSExecutePhaseContextAssembler(
|
||||
context_tier_service=tier_service,
|
||||
project_repository=repo,
|
||||
acms_pipeline=mock_pipeline,
|
||||
hot_max_tokens=4096,
|
||||
)
|
||||
|
||||
|
||||
@given("epcov an assembler with context_policy_json hot_max_tokens 32000")
|
||||
def step_epcov_assembler_policy_json_32k(context: Context) -> None:
|
||||
"""Assembler backed by a DB row with hot_max_tokens=32000 in policy JSON.
|
||||
|
||||
Mirrors the real storage format written by
|
||||
``agents project context set --hot-max-tokens 32000``:
|
||||
``{"acms_config": {"hot_max_tokens": 32000, ...}, ...}``.
|
||||
"""
|
||||
policy_json = json.dumps({"acms_config": {"hot_max_tokens": 32000}})
|
||||
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
|
||||
|
||||
|
||||
@given("epcov an assembler with no hot_max_tokens in context_policy_json")
|
||||
def step_epcov_assembler_policy_json_no_override(context: Context) -> None:
|
||||
"""Assembler backed by a DB row with no hot_max_tokens — global fallback."""
|
||||
policy_json = json.dumps({"acms_config": {"other_setting": "value"}})
|
||||
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
|
||||
|
||||
|
||||
@then("epcov the pipeline received budget max_tokens of 32000")
|
||||
def step_epcov_pipeline_budget_32k(context: Context) -> None:
|
||||
"""Verify the pipeline was called with max_tokens=32000 in the budget."""
|
||||
if context.epcov_error is not None:
|
||||
raise AssertionError(f"Unexpected error: {context.epcov_error}")
|
||||
pipeline = context.epcov_assembler._pipeline
|
||||
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
|
||||
call_args = pipeline.assemble.call_args
|
||||
budget = call_args.kwargs.get("budget") or (
|
||||
call_args[1].get("budget") if call_args[1] else None
|
||||
)
|
||||
if budget is None and call_args[0]:
|
||||
budget = call_args[0][2] if len(call_args[0]) > 2 else None
|
||||
assert budget is not None, "Could not find budget in pipeline.assemble() call"
|
||||
assert budget.max_tokens == 32000, (
|
||||
f"Expected budget.max_tokens=32000 (from context_policy_json), "
|
||||
f"got {budget.max_tokens}"
|
||||
)
|
||||
|
||||
|
||||
@then("epcov the pipeline received budget max_tokens of 4096")
|
||||
def step_epcov_pipeline_budget_global(context: Context) -> None:
|
||||
"""Verify the pipeline fell back to the global hot_max_tokens=4096."""
|
||||
if context.epcov_error is not None:
|
||||
raise AssertionError(f"Unexpected error: {context.epcov_error}")
|
||||
pipeline = context.epcov_assembler._pipeline
|
||||
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
|
||||
call_args = pipeline.assemble.call_args
|
||||
budget = call_args.kwargs.get("budget") or (
|
||||
call_args[1].get("budget") if call_args[1] else None
|
||||
)
|
||||
if budget is None and call_args[0]:
|
||||
budget = call_args[0][2] if len(call_args[0]) > 2 else None
|
||||
assert budget is not None, "Could not find budget in pipeline.assemble() call"
|
||||
assert budget.max_tokens == 4096, (
|
||||
f"Expected budget.max_tokens=4096 (global fallback), got {budget.max_tokens}"
|
||||
)
|
||||
|
||||
@@ -135,6 +135,11 @@ def step_plan_invariants(context):
|
||||
context.plan_invariants = _parse_invariant_table(context, InvariantScope.PLAN)
|
||||
|
||||
|
||||
@given("I have action invariants")
|
||||
def step_action_invariants(context):
|
||||
context.action_invariants = _parse_invariant_table(context, InvariantScope.ACTION)
|
||||
|
||||
|
||||
@given("I have project invariants")
|
||||
def step_project_invariants(context):
|
||||
context.project_invariants = _parse_invariant_table(context, InvariantScope.PROJECT)
|
||||
@@ -160,6 +165,7 @@ def step_plan_invariants_inactive(context):
|
||||
def step_merge(context):
|
||||
context.merged = merge_invariants(
|
||||
getattr(context, "plan_invariants", []),
|
||||
getattr(context, "action_invariants", []),
|
||||
getattr(context, "project_invariants", []),
|
||||
getattr(context, "global_invariants", []),
|
||||
)
|
||||
@@ -180,6 +186,43 @@ def step_merged_scope(context, idx, scope):
|
||||
assert context.merged[idx].scope.value == scope
|
||||
|
||||
|
||||
@then("action invariants appear before project invariants in merge")
|
||||
def step_action_before_project(context):
|
||||
"""Verify ACTION tier comes before PROJECT tier in the merged result."""
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.merged if i.scope == InvariantScope.PROJECT]
|
||||
assert len(action_invs) > 0, "No action invariants found in merged result"
|
||||
assert len(project_invs) > 0, "No project invariants found in merged result"
|
||||
first_action_idx = context.merged.index(action_invs[0])
|
||||
first_project_idx = context.merged.index(project_invs[0])
|
||||
assert first_action_idx < first_project_idx, (
|
||||
f"Action invariant at index {first_action_idx} "
|
||||
f"should appear before project invariant at index {first_project_idx}"
|
||||
)
|
||||
|
||||
|
||||
@then("plan invariants appear before action invariants in merge")
|
||||
def step_plan_before_action(context):
|
||||
"""Verify PLAN tier comes before ACTION tier in the merged result."""
|
||||
plan_invs = [i for i in context.merged if i.scope == InvariantScope.PLAN]
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
assert len(plan_invs) > 0, "No plan invariants found in merged result"
|
||||
assert len(action_invs) > 0, "No action invariants found in merged result"
|
||||
first_plan_idx = context.merged.index(plan_invs[0])
|
||||
first_action_idx = context.merged.index(action_invs[0])
|
||||
assert first_plan_idx < first_action_idx, (
|
||||
f"Plan invariant at index {first_plan_idx} "
|
||||
f"should appear before action invariant at index {first_action_idx}"
|
||||
)
|
||||
|
||||
|
||||
@then("action invariants are preserved in merge result")
|
||||
def step_action_invariants_preserved(context):
|
||||
"""Verify that action-scoped invariants appear in the merged output."""
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
assert len(action_invs) > 0, "Expected action-scoped invariants in merge result"
|
||||
|
||||
|
||||
# ================================================================
|
||||
# InvariantSet
|
||||
# ================================================================
|
||||
@@ -189,6 +232,7 @@ def step_merged_scope(context, idx, scope):
|
||||
def step_merge_invariant_set(context):
|
||||
inv_set = InvariantSet.merge(
|
||||
getattr(context, "plan_invariants", []),
|
||||
getattr(context, "action_invariants", []),
|
||||
getattr(context, "project_invariants", []),
|
||||
getattr(context, "global_invariants", []),
|
||||
)
|
||||
@@ -389,6 +433,15 @@ def step_service_all_scopes(context):
|
||||
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
|
||||
|
||||
|
||||
@given("an invariant service with invariants at all four scopes")
|
||||
def step_service_all_four_scopes(context):
|
||||
context.service = InvariantService()
|
||||
context.service.add_invariant("Global rule", InvariantScope.GLOBAL, "system")
|
||||
context.service.add_invariant("Project rule", InvariantScope.PROJECT, "proj1")
|
||||
context.service.add_invariant("Action rule", InvariantScope.ACTION, "action1")
|
||||
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
|
||||
|
||||
|
||||
@when('I get effective invariants for plan "{plan_id}" and project "{project}"')
|
||||
def step_effective(context, plan_id, project):
|
||||
context.effective = context.service.get_effective_invariants(
|
||||
@@ -396,6 +449,15 @@ def step_effective(context, plan_id, project):
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I get effective invariants for plan "{plan_id}", action "{action}", and project "{project}"'
|
||||
)
|
||||
def step_effective_with_action(context, plan_id, action, project):
|
||||
context.effective = context.service.get_effective_invariants(
|
||||
plan_id=plan_id, action_name=action, project_name=project
|
||||
)
|
||||
|
||||
|
||||
@then("the effective set should contain plan invariants first")
|
||||
def step_effective_plan_first(context):
|
||||
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
|
||||
@@ -404,6 +466,35 @@ def step_effective_plan_first(context):
|
||||
assert first_plan_idx == 0
|
||||
|
||||
|
||||
@then("the effective set should contain action invariants second")
|
||||
def step_effective_action_second(context):
|
||||
"""Verify ACTION tier appears after PLAN and before PROJECT."""
|
||||
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
|
||||
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
assert len(plan_invs) > 0, "No plan invariants found"
|
||||
assert len(action_invs) > 0, "No action invariants found — bug #9126 not fixed!"
|
||||
assert len(project_invs) > 0, "No project invariants found"
|
||||
first_action_idx = context.effective.index(action_invs[0])
|
||||
if plan_invs:
|
||||
last_plan_idx = context.effective.index(plan_invs[-1])
|
||||
assert first_action_idx > last_plan_idx
|
||||
first_project_idx = context.effective.index(project_invs[0])
|
||||
assert first_project_idx > first_action_idx
|
||||
|
||||
|
||||
@then("the effective set should contain project invariants after action")
|
||||
def step_effective_project_after_action(context):
|
||||
"""Verify PROJECT tier appears after ACTION tier."""
|
||||
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
assert len(action_invs) > 0, "No action invariants found"
|
||||
assert len(project_invs) > 0, "No project invariants found"
|
||||
first_proj_idx = context.effective.index(project_invs[0])
|
||||
last_action_idx = context.effective.index(action_invs[-1])
|
||||
assert first_proj_idx > last_action_idx
|
||||
|
||||
|
||||
@then("the effective set should contain project invariants second")
|
||||
def step_effective_project_second(context):
|
||||
proj_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
@@ -435,6 +526,13 @@ def step_no_duplicates(context):
|
||||
assert len(texts) == len(set(texts)), f"Duplicates found: {texts}"
|
||||
|
||||
|
||||
@then("the effective set should have {count:d} invariants")
|
||||
def step_effective_count(context, count):
|
||||
assert len(context.effective) == count, (
|
||||
f"Expected {count}, got {len(context.effective)}"
|
||||
)
|
||||
|
||||
|
||||
# ================================================================
|
||||
# Enforcement
|
||||
# ================================================================
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
"""Step definitions for plan_lifecycle_service_coverage_boost_r3.feature.
|
||||
|
||||
Targets remaining uncovered lines in PlanLifecycleService:
|
||||
- Lines ~1520-1540: complete_strategize event_bus.emit success + exception
|
||||
- Lines ~1569-1581: fail_strategize event_bus.emit success + exception
|
||||
- Lines ~1739-1759: complete_execute event_bus.emit success + exception
|
||||
- Lines ~1774-1792: fail_execute event_bus.emit success + exception
|
||||
- Lines ~2036-2054: fail_apply event_bus.emit success + exception
|
||||
- Lines ~2056-2057: fail_execute _cleanup_devcontainers call
|
||||
- Lines ~2056-2057: fail_apply _cleanup_devcontainers call
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Background
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
@given("I have a fresh plan lifecycle service for coverage boost r3")
|
||||
def step_create_fresh_service_r3(context: Context) -> None:
|
||||
"""Create a clean PlanLifecycleService instance."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.error = None
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
def _create_action_r3(context: Context, name: str, **kwargs):
|
||||
"""Helper to create a basic action with sensible defaults."""
|
||||
defaults = {
|
||||
"name": name,
|
||||
"description": f"Action {name}",
|
||||
"definition_of_done": "Tests pass",
|
||||
"strategy_actor": "openai/gpt-4",
|
||||
"execution_actor": "openai/gpt-4",
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return context.service.create_action(**defaults)
|
||||
|
||||
|
||||
def _advance_to_state(
|
||||
context: Context, target_phase: PlanPhase, target_state: ProcessingState
|
||||
):
|
||||
"""Advance the current plan to the target phase and state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
if (
|
||||
target_phase == PlanPhase.STRATEGIZE
|
||||
and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif (
|
||||
target_phase == PlanPhase.EXECUTE and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif target_phase == PlanPhase.APPLY and target_state == ProcessingState.PROCESSING:
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.service.apply_plan(pid)
|
||||
context.service.start_apply(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
class RecordingEventBus:
|
||||
"""A simple event bus that records emitted events."""
|
||||
|
||||
def __init__(self):
|
||||
self.events = []
|
||||
|
||||
def emit(self, event):
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type, handler):
|
||||
pass
|
||||
|
||||
|
||||
class FailingEventBus:
|
||||
"""An event bus that always raises on emit."""
|
||||
|
||||
def emit(self, event):
|
||||
raise RuntimeError("Simulated event bus failure")
|
||||
|
||||
def subscribe(self, event_type, handler):
|
||||
pass
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_strategize — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with a recording event bus for r3")
|
||||
def step_service_with_recording_bus_r3(context: Context) -> None:
|
||||
"""Create a service with a recording event bus."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.recording_bus = RecordingEventBus()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=context.recording_bus,
|
||||
)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given('an action "{name}" exists for coverage boost r3')
|
||||
def step_create_action_r3(context: Context, name: str) -> None:
|
||||
"""Create an action with the given name."""
|
||||
context.action = _create_action_r3(context, name)
|
||||
|
||||
|
||||
@given("a plan in strategize phase with processing state for r3")
|
||||
def step_plan_strategize_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Strategize/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.STRATEGIZE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when("I complete strategize on the plan with event bus")
|
||||
def step_complete_strategize_with_bus_r3(context: Context) -> None:
|
||||
"""Complete strategize — should emit PLAN_STATE_CHANGED event."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_strategize(
|
||||
context.plan.identity.plan_id
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded a PLAN_STATE_CHANGED event")
|
||||
def step_verify_plan_state_changed_event_r3(context: Context) -> None:
|
||||
"""Verify PLAN_STATE_CHANGED was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
state_events = [
|
||||
e
|
||||
for e in context.recording_bus.events
|
||||
if e.event_type == EventType.PLAN_STATE_CHANGED
|
||||
]
|
||||
assert len(state_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_STATE_CHANGED event, got {len(state_events)}. "
|
||||
f"All events: {[e.event_type for e in context.recording_bus.events]}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_strategize — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with a failing event bus for r3")
|
||||
def step_service_with_failing_bus_r3(context: Context) -> None:
|
||||
"""Create a service with an event bus that raises on emit."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=FailingEventBus(),
|
||||
)
|
||||
context.error = None
|
||||
|
||||
|
||||
@when("I complete strategize on the plan and event bus emit fails")
|
||||
def step_complete_strategize_failing_bus_r3(context: Context) -> None:
|
||||
"""Complete strategize — event bus will raise but plan should still complete."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_strategize(
|
||||
context.plan.identity.plan_id
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then('the plan phase should be "{phase}" and state should be "{state}"')
|
||||
def step_verify_plan_phase_state_r3(context: Context, phase: str, state: str) -> None:
|
||||
"""Verify the plan phase and state."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase.value == phase, (
|
||||
f"Expected phase '{phase}', got '{context.plan.phase.value}'"
|
||||
)
|
||||
assert context.plan.processing_state.value == state, (
|
||||
f"Expected state '{state}', got '{context.plan.processing_state.value}'"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_strategize — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail strategize on the plan with error "{error}"')
|
||||
def step_fail_strategize_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail strategize with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_strategize(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded a PLAN_ERRORED event")
|
||||
def step_verify_plan_errored_event_r3(context: Context) -> None:
|
||||
"""Verify PLAN_ERRORED was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
error_events = [
|
||||
e
|
||||
for e in context.recording_bus.events
|
||||
if e.event_type == EventType.PLAN_ERRORED
|
||||
]
|
||||
assert len(error_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_ERRORED event, got {len(error_events)}. "
|
||||
f"All events: {[e.event_type for e in context.recording_bus.events]}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_strategize — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail strategize on the plan and event bus emit fails")
|
||||
def step_fail_strategize_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail strategize — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_strategize(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in errored state for r3")
|
||||
def step_verify_errored_state_r3(context: Context) -> None:
|
||||
"""Verify the plan is in ERRORED state."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.ERRORED, (
|
||||
f"Expected ERRORED, got {context.plan.processing_state.value}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_execute — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan in execute phase with processing state for r3")
|
||||
def step_plan_execute_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Execute/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.EXECUTE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when("I complete execute on the plan with event bus")
|
||||
def step_complete_execute_with_bus_r3(context: Context) -> None:
|
||||
"""Complete execute — should emit PLAN_STATE_CHANGED event."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_execute(context.plan.identity.plan_id)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# complete_execute — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I complete execute on the plan and event bus emit fails")
|
||||
def step_complete_execute_failing_bus_r3(context: Context) -> None:
|
||||
"""Complete execute — event bus will raise but plan should still complete."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.complete_execute(context.plan.identity.plan_id)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail execute on the plan with error "{error}"')
|
||||
def step_fail_execute_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail execute with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail execute on the plan and event bus emit fails")
|
||||
def step_fail_execute_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail execute — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — event_bus.emit success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan in apply phase with processing state for r3")
|
||||
def step_plan_apply_processing_r3(context: Context) -> None:
|
||||
"""Create a plan in Apply/PROCESSING state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r3")],
|
||||
)
|
||||
_advance_to_state(context, PlanPhase.APPLY, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@when('I fail apply on the plan with error "{error}"')
|
||||
def step_fail_apply_with_error_r3(context: Context, error: str) -> None:
|
||||
"""Fail apply with an error message."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_apply(context.plan.identity.plan_id, error)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — event_bus.emit exception
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I fail apply on the plan and event bus emit fails")
|
||||
def step_fail_apply_failing_bus_r3(context: Context) -> None:
|
||||
"""Fail apply — event bus will raise but plan should still be errored."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.fail_apply(
|
||||
context.plan.identity.plan_id, "Test error"
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_execute — _cleanup_devcontainers success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("a plan lifecycle service with mocked devcontainer cleanup for r3")
|
||||
def step_service_with_mocked_cleanup_r3(context: Context) -> None:
|
||||
"""Create a service where devcontainer cleanup returns stopped containers."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.cleanup_called = False
|
||||
context.error = None
|
||||
|
||||
|
||||
@when('I fail execute on the plan with error "{error}" and cleanup should be called')
|
||||
def step_fail_execute_with_cleanup_r3(context: Context, error: str) -> None:
|
||||
"""Fail execute with mocked cleanup."""
|
||||
context.error = None
|
||||
context.cleanup_called = False
|
||||
try:
|
||||
with patch(
|
||||
"cleveragents.application.services.cleanup_service.stop_all_active_containers",
|
||||
return_value=["container-1", "container-2"],
|
||||
):
|
||||
context.plan = context.service.fail_execute(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
context.cleanup_called = True
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in errored state and cleanup should have been called")
|
||||
def step_verify_errored_with_cleanup_r3(context: Context) -> None:
|
||||
"""Verify the plan is ERRORED and cleanup was invoked."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.ERRORED, (
|
||||
f"Expected ERRORED, got {context.plan.processing_state.value}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# fail_apply — _cleanup_devcontainers success
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when('I fail apply on the plan with error "{error}" and cleanup should be called')
|
||||
def step_fail_apply_with_cleanup_r3(context: Context, error: str) -> None:
|
||||
"""Fail apply with mocked cleanup."""
|
||||
context.error = None
|
||||
context.cleanup_called = False
|
||||
try:
|
||||
with patch(
|
||||
"cleveragents.application.services.cleanup_service.stop_all_active_containers",
|
||||
return_value=["container-1", "container-2"],
|
||||
):
|
||||
context.plan = context.service.fail_apply(
|
||||
context.plan.identity.plan_id, error
|
||||
)
|
||||
context.cleanup_called = True
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
@@ -0,0 +1,886 @@
|
||||
"""Step definitions for plan_lifecycle_service_coverage_boost_r4.feature.
|
||||
|
||||
Targets remaining uncovered lines in PlanLifecycleService:
|
||||
- Lines ~386-405: _run_estimation event_bus.emit PLAN_ESTIMATION_COMPLETE success + exception
|
||||
- Line 376: cost_estimate_usd assignment when estimated_cost_usd is not None
|
||||
- Line 462: project_name extraction from plan.project_links[0]
|
||||
- Lines ~481-505: _run_invariant_reconciliation INVARIANT_RECONCILED emit exception
|
||||
- Lines ~517-536: _run_invariant_reconciliation INVARIANT_VIOLATED emit exception
|
||||
- Lines ~562-566: _subscribe_correction_reconciliation event_bus.subscribe failure
|
||||
- Line 577: _handle_correction_applied guard (plan_id is None)
|
||||
- Lines ~584-595: _handle_correction_applied ReconciliationBlockedError + general exception
|
||||
- Lines ~1230-1232: unknown automation profile ValidationError
|
||||
- Lines ~1616-1621, 1684-1688: execute_plan lock acquire/release with lock_service
|
||||
- Lines ~1830-1835, 1872-1877: apply_plan lock acquire/release with lock_service
|
||||
- Lines ~2082-2097: prompt_plan blank guidance + wrong phase/state validation
|
||||
- Lines ~2100-2156: prompt_plan state transition + decision_service interaction
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
ReconciliationBlockedError,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.core.exceptions import PlanError, ValidationError
|
||||
from cleveragents.domain.models.core.action import Action
|
||||
from cleveragents.domain.models.core.estimation import EstimationResult
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Background
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
@given("I have a fresh plan lifecycle service for coverage boost r4")
|
||||
def step_create_fresh_service_r4(context: Context) -> None:
|
||||
"""Create a clean PlanLifecycleService instance."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
context.error = None
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
def _create_action_r4(context: Context, name: str, **kwargs) -> Action:
|
||||
"""Helper to create a basic action with sensible defaults."""
|
||||
defaults: dict[str, Any] = {
|
||||
"name": name,
|
||||
"description": f"Action {name}",
|
||||
"definition_of_done": "Tests pass",
|
||||
"strategy_actor": "openai/gpt-4",
|
||||
"execution_actor": "openai/gpt-4",
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return context.service.create_action(**defaults)
|
||||
|
||||
|
||||
def _advance_to_strategize_complete(context: Context) -> None:
|
||||
"""Advance plan to strategize-complete state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_execute_complete(context: Context) -> None:
|
||||
"""Advance plan to execute-complete state (ready for apply)."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_strategize_processing(context: Context) -> None:
|
||||
"""Advance plan to strategize/PROCESSING state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Mock event buses
|
||||
# -----------------------------------------------------------------
|
||||
|
||||
|
||||
class FailingEventBusR4:
|
||||
"""An event bus that raises on every emit call."""
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
raise RuntimeError("Simulated event bus emit failure")
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class FailingSubscribeEventBus:
|
||||
"""An event bus that raises on subscribe but works for emit."""
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
pass
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
raise RuntimeError("Simulated subscribe failure")
|
||||
|
||||
|
||||
class RecordingEventBusR4:
|
||||
"""A simple event bus that records emitted events."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class FailingDecisionService:
|
||||
"""A decision service that raises on all operations."""
|
||||
|
||||
def list_decisions(self, plan_id: str) -> list[Any]:
|
||||
raise RuntimeError("Simulated decision service failure")
|
||||
|
||||
def record_decision(self, **kwargs: Any) -> Any:
|
||||
raise RuntimeError("Simulated decision service failure")
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _run_estimation via execute_plan — event_bus.emit PLAN_ESTIMATION_COMPLETE
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with estimation_actor configured for r4")
|
||||
def step_action_with_estimation_actor_r4(context: Context) -> None:
|
||||
"""Create an action with estimation_actor set."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/est-action-r4",
|
||||
estimation_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@given("an action with estimation_actor returning cost for r4")
|
||||
def step_action_with_estimation_cost_r4(context: Context) -> None:
|
||||
"""Create an action with estimation_actor and patch stub to return cost."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/est-cost-action-r4",
|
||||
estimation_actor="openai/gpt-4",
|
||||
)
|
||||
context.mock_estimation_result = EstimationResult(
|
||||
summary="Test estimation with cost",
|
||||
estimated_cost_usd=1.23,
|
||||
)
|
||||
context.estimation_patch = patch(
|
||||
"cleveragents.application.services.plan_executor.EstimationStubActor.estimate",
|
||||
return_value=context.mock_estimation_result,
|
||||
)
|
||||
|
||||
|
||||
@given("a plan in strategize-complete state for r4")
|
||||
def step_plan_strategize_complete_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/COMPLETE state."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_strategize_complete(context)
|
||||
|
||||
|
||||
@given("a plan in strategize phase with processing state for r4")
|
||||
def step_plan_strategize_processing_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/PROCESSING state."""
|
||||
if not hasattr(context, "action") or context.action is None:
|
||||
context.action = _create_action_r4(
|
||||
context, f"local/corr-handler-r4-{id(context)}"
|
||||
)
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_strategize_processing(context)
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a recording event bus for r4")
|
||||
def step_service_with_recording_bus_r4(context: Context) -> None:
|
||||
"""Set up recording event bus on existing service."""
|
||||
context.recording_bus_r4 = RecordingEventBusR4()
|
||||
context.service.event_bus = context.recording_bus_r4
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a failing event bus for r4")
|
||||
def step_service_with_failing_bus_r4(context: Context) -> None:
|
||||
"""Set up failing event bus on existing service."""
|
||||
context.service.event_bus = FailingEventBusR4()
|
||||
|
||||
|
||||
@when("I execute the plan for r4")
|
||||
def step_execute_plan_r4(context: Context) -> None:
|
||||
"""Call execute_plan on the current plan."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event bus should have recorded PLAN_ESTIMATION_COMPLETE event")
|
||||
def step_verify_estimation_complete_event_r4(context: Context) -> None:
|
||||
"""Verify PLAN_ESTIMATION_COMPLETE was emitted."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
est_events = [
|
||||
e
|
||||
for e in context.recording_bus_r4.events
|
||||
if e.event_type == EventType.PLAN_ESTIMATION_COMPLETE
|
||||
]
|
||||
assert len(est_events) >= 1, (
|
||||
f"Expected at least 1 PLAN_ESTIMATION_COMPLETE event, "
|
||||
f"got {len(est_events)}. Events: {[e.event_type for e in context.recording_bus_r4.events]}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan should be in execute phase despite estimation emit failure")
|
||||
def step_plan_in_executeDespiteEstimationEmitFailure_r4(context: Context) -> None:
|
||||
"""Verify plan is in EXECUTE even when estimation emit failed."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan should be in execute phase")
|
||||
def step_plan_in_execute_phase_r4(context: Context) -> None:
|
||||
"""Verify plan is in EXECUTE phase."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan cost_estimate_usd should be populated")
|
||||
def step_verify_cost_estimate_populated_r4(context: Context) -> None:
|
||||
"""Verify plan.cost_estimate_usd was set from estimation result."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.cost_estimate_usd is not None, (
|
||||
"Expected cost_estimate_usd to be set, got None"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# execute_plan with mocked EstimationStubActor for cost_estimate_usd
|
||||
# =================================================================
|
||||
|
||||
|
||||
@when("I execute the plan for r4 with mocked estimation")
|
||||
def step_execute_plan_with_mocked_estimation_r4(context: Context) -> None:
|
||||
"""Execute plan with EstimationStubActor patched to return cost."""
|
||||
mock_result = context.mock_estimation_result
|
||||
with patch(
|
||||
"cleveragents.application.services.plan_executor.EstimationStubActor.estimate",
|
||||
return_value=mock_result,
|
||||
):
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# execute_plan with lock_service
|
||||
# =================================================================
|
||||
|
||||
|
||||
class MockLockService:
|
||||
"""A mock lock service that tracks acquire/release calls."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.acquired: list[tuple[str, str, str]] = []
|
||||
self.released: list[tuple[str, str, str]] = []
|
||||
|
||||
def acquire(self, owner_id: str, resource_type: str, resource_id: str) -> None:
|
||||
self.acquired.append((owner_id, resource_type, resource_id))
|
||||
|
||||
def release(self, owner_id: str, resource_type: str, resource_id: str) -> None:
|
||||
self.released.append((owner_id, resource_type, resource_id))
|
||||
|
||||
|
||||
@given("an action for execute with lock service for r4")
|
||||
def step_action_execute_lock_r4(context: Context) -> None:
|
||||
"""Create an action for testing execute_plan with lock_service."""
|
||||
context.action = _create_action_r4(context, "local/exec-lock-r4")
|
||||
|
||||
|
||||
@when("I execute the plan with lock service for r4")
|
||||
def step_execute_with_lock_r4(context: Context) -> None:
|
||||
"""Execute plan with a mocked lock service."""
|
||||
context.mock_lock_service = MockLockService()
|
||||
context.service._lock_service = context.mock_lock_service
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the lock service should have acquired and released the plan lock")
|
||||
def step_verify_lock_acquire_release_r4(context: Context) -> None:
|
||||
"""Verify lock was acquired and released for execute_plan."""
|
||||
assert hasattr(context, "mock_lock_service"), "mock_lock_service not found"
|
||||
ls = context.mock_lock_service
|
||||
assert len(ls.acquired) >= 1, (
|
||||
f"Expected at least 1 acquire call, got {len(ls.acquired)}"
|
||||
)
|
||||
assert len(ls.released) >= 1, (
|
||||
f"Expected at least 1 release call, got {len(ls.released)}"
|
||||
)
|
||||
plan_id = context.plan.identity.plan_id
|
||||
for call in ls.acquired:
|
||||
assert call[1] == "plan", f"Expected resource_type 'plan', got {call[1]}"
|
||||
assert call[2] == plan_id, f"Expected resource_id {plan_id}, got {call[2]}"
|
||||
|
||||
|
||||
# =================================================================
|
||||
# apply_plan with lock_service
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action for apply with lock service for r4")
|
||||
def step_action_apply_lock_r4(context: Context) -> None:
|
||||
"""Create an action for testing apply_plan with lock_service."""
|
||||
context.action = _create_action_r4(context, "local/apply-lock-r4")
|
||||
|
||||
|
||||
@given("a plan in execute-complete state for r4")
|
||||
def step_plan_execute_complete_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/COMPLETE state (ready for apply)."""
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_complete(context)
|
||||
|
||||
|
||||
@when("I apply the plan with lock service for r4")
|
||||
def step_apply_with_lock_r4(context: Context) -> None:
|
||||
"""Apply plan with a mocked lock service."""
|
||||
context.mock_lock_service = MockLockService()
|
||||
context.service._lock_service = context.mock_lock_service
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.apply_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan should be in apply phase")
|
||||
def step_plan_in_apply_phase_r4(context: Context) -> None:
|
||||
"""Verify plan is in APPLY phase."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.APPLY, (
|
||||
f"Expected APPLY phase, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _subscribe_correction_reconciliation — event_bus.subscribe failure
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("I have a plan lifecycle service with event_bus that fails on subscribe for r4")
|
||||
def step_service_with_failing_subscribe_r4(context: Context) -> None:
|
||||
"""Create service where event_bus.subscribe raises."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.error = None
|
||||
try:
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
event_bus=FailingSubscribeEventBus(),
|
||||
)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@when("the service is created for r4")
|
||||
def step_service_created_r4(context: Context) -> None:
|
||||
"""Service creation step (no-op since given already created it)."""
|
||||
pass
|
||||
|
||||
|
||||
@then("no exception should be raised for r4")
|
||||
def step_no_exception_r4(context: Context) -> None:
|
||||
"""Verify no exception was raised."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
|
||||
|
||||
# =================================================================
|
||||
# prompt_plan validation scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
def _advance_to_execute_processing(context: Context) -> None:
|
||||
"""Advance plan to execute/PROCESSING state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_execute_complete_for_prompt(context: Context) -> None:
|
||||
"""Advance plan to execute/COMPLETE state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.service.complete_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
@given("a plan in execute phase with processing state for prompt_plan r4")
|
||||
def step_plan_execute_processing_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/PROCESSING state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-test-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_processing(context)
|
||||
|
||||
|
||||
@given("a plan in strategize phase for prompt_plan r4")
|
||||
def step_plan_strategize_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in strategize/PROCESSING state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-wrong-phase-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_state_prompt(context, PlanPhase.STRATEGIZE, ProcessingState.PROCESSING)
|
||||
|
||||
|
||||
@given("a plan in execute-complete state for prompt_plan r4")
|
||||
def step_plan_execute_complete_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/COMPLETE state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-complete-state-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_complete_for_prompt(context)
|
||||
|
||||
|
||||
@given("a plan in execute phase with errored state for prompt_plan r4")
|
||||
def step_plan_execute_errored_for_prompt_r4(context: Context) -> None:
|
||||
"""Create a plan in execute/ERRORED state for prompt_plan testing."""
|
||||
context.action = _create_action_r4(context, "local/prompt-errored-r4")
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
_advance_to_execute_processing(context)
|
||||
pid = context.plan.identity.plan_id
|
||||
context.service.fail_execute(pid, error_message="Prior test error")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
def _advance_to_state_prompt(
|
||||
context: Context, target_phase: PlanPhase, target_state: ProcessingState
|
||||
) -> None:
|
||||
"""Advance plan to target phase/state."""
|
||||
pid = context.plan.identity.plan_id
|
||||
if (
|
||||
target_phase == PlanPhase.STRATEGIZE
|
||||
and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
elif (
|
||||
target_phase == PlanPhase.EXECUTE and target_state == ProcessingState.PROCESSING
|
||||
):
|
||||
context.service.start_strategize(pid)
|
||||
context.service.complete_strategize(pid)
|
||||
context.service.execute_plan(pid)
|
||||
context.service.start_execute(pid)
|
||||
context.plan = context.service.get_plan(pid)
|
||||
|
||||
|
||||
@when("I prompt the plan with blank guidance for r4")
|
||||
def step_prompt_blank_guidance_r4(context: Context) -> None:
|
||||
"""Call prompt_plan with blank guidance."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance=" ")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a ValidationError should be raised for blank guidance")
|
||||
def step_verify_validation_error_r4(context: Context) -> None:
|
||||
"""Verify ValidationError was raised for blank guidance."""
|
||||
assert isinstance(context.error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for wrong phase for r4")
|
||||
def step_prompt_wrong_phase_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when plan is not in Execute phase."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Do something")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a PlanError should be raised for wrong phase")
|
||||
def step_verify_plan_error_wrong_phase_r4(context: Context) -> None:
|
||||
"""Verify PlanError was raised for wrong phase."""
|
||||
assert isinstance(context.error, PlanError), (
|
||||
f"Expected PlanError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for complete state for r4")
|
||||
def step_prompt_wrong_state_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when plan is in COMPLETE state (not recoverable)."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Resume work")
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a PlanError should be raised for wrong state")
|
||||
def step_verify_plan_error_wrong_state_r4(context: Context) -> None:
|
||||
"""Verify PlanError was raised for wrong state."""
|
||||
assert isinstance(context.error, PlanError), (
|
||||
f"Expected PlanError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance to resume from errored state for r4")
|
||||
def step_prompt_errored_resume_r4(context: Context) -> None:
|
||||
"""Call prompt_plan to resume from errored state."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Please recover")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the plan processing_state should be processing")
|
||||
def step_verify_processing_state_r4(context: Context) -> None:
|
||||
"""Verify plan.processing_state is PROCESSING."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.processing_state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING state, got {context.plan.processing_state}"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan error_message should be cleared")
|
||||
def step_verify_error_cleared_r4(context: Context) -> None:
|
||||
"""Verify plan.error_message was cleared after resuming from errored."""
|
||||
assert context.plan.error_message is None, (
|
||||
f"Expected error_message to be cleared, got: {context.plan.error_message}"
|
||||
)
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a failing decision service for r4")
|
||||
def step_failing_decision_service_r4(context: Context) -> None:
|
||||
"""Replace decision_service with one that always fails."""
|
||||
context.service.decision_service = FailingDecisionService()
|
||||
|
||||
|
||||
@when("I prompt the plan with guidance for r4")
|
||||
def step_prompt_with_failing_decision_service_r4(context: Context) -> None:
|
||||
"""Call prompt_plan when decision_service raises."""
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.result = context.service.prompt_plan(pid, guidance="Continue anyway")
|
||||
context.plan = context.service.get_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _run_invariant_reconciliation — event_bus.emit exception scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with invariant_actor for reconciliation success r4")
|
||||
def step_action_invariant_success_r4(context: Context) -> None:
|
||||
"""Create action with invariant_actor set for successful reconciliation."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/inv-success-r4",
|
||||
invariant_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@given("a plan in strategize-complete state with project links for r4")
|
||||
def step_plan_strategize_complete_with_links_r4(context: Context) -> None:
|
||||
"""Create plan with project_links for testing project_name extraction."""
|
||||
if (
|
||||
not hasattr(context.service, "invariant_service")
|
||||
or context.service.invariant_service is None
|
||||
):
|
||||
from cleveragents.application.services.invariant_service import InvariantService
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
|
||||
context.service.invariant_service = InvariantService()
|
||||
context.service.decision_service = DecisionService()
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="test-project-r4")],
|
||||
)
|
||||
_advance_to_strategize_complete(context)
|
||||
|
||||
|
||||
@when("I execute the plan for invariant reconciliation emit failure r4")
|
||||
def step_execute_invariant_emit_fail_r4(context: Context) -> None:
|
||||
"""Execute plan with mocked reconciliation actor (succeeds) and failing emit."""
|
||||
from cleveragents.actor.reconciliation import ReconciliationResult
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
from cleveragents.domain.models.core.invariant import InvariantSet
|
||||
|
||||
context.error = None
|
||||
mock_result = ReconciliationResult(
|
||||
reconciled_set=InvariantSet(invariants=[]),
|
||||
conflicts=[],
|
||||
enforced_decision_ids=[],
|
||||
)
|
||||
|
||||
class PatchedActor(InvariantReconciliationActor):
|
||||
def run(self, **kwargs: Any) -> Any:
|
||||
return mock_result
|
||||
|
||||
with patch(
|
||||
"cleveragents.actor.reconciliation.InvariantReconciliationActor",
|
||||
PatchedActor,
|
||||
):
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("the event_bus should have been called for INVARIANT_RECONCILED emit failure")
|
||||
def step_verify_invariant_reconciled_emit_fail_r4(context: Context) -> None:
|
||||
"""Verify plan reached EXECUTE despite emit failure in reconciliation."""
|
||||
assert context.error is None, f"Expected no error, got: {context.error}"
|
||||
assert context.plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE, got {context.plan.phase}"
|
||||
)
|
||||
|
||||
|
||||
@given("an action with invariant_actor causing reconciliation failure for r4")
|
||||
def step_action_invariant_fail_r4(context: Context) -> None:
|
||||
"""Create action with invariant_actor that will cause failure."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/inv-fail-r4",
|
||||
invariant_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@when("I execute the plan expecting reconciliation failure for r4")
|
||||
def step_execute_invariant_fail_r4(context: Context) -> None:
|
||||
"""Execute plan where reconciliation raises (triggers INVARIANT_VIOLATED emit fail path)."""
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
|
||||
class FailingActor(InvariantReconciliationActor):
|
||||
def run(self, **kwargs: Any) -> Any:
|
||||
raise RuntimeError("Simulated invariant violation")
|
||||
|
||||
with patch(
|
||||
"cleveragents.actor.reconciliation.InvariantReconciliationActor",
|
||||
FailingActor,
|
||||
):
|
||||
context.error = None
|
||||
try:
|
||||
pid = context.plan.identity.plan_id
|
||||
context.plan = context.service.execute_plan(pid)
|
||||
except ReconciliationBlockedError as e:
|
||||
context.error = e
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("ReconciliationBlockedError should be raised")
|
||||
def step_verify_reconciliation_blocked_error_r4(context: Context) -> None:
|
||||
"""Verify ReconciliationBlockedError was raised."""
|
||||
assert isinstance(context.error, ReconciliationBlockedError), (
|
||||
f"Expected ReconciliationBlockedError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
@then("the event_bus should have been called for INVARIANT_VIOLATED emit failure")
|
||||
def step_verify_invariant_violated_emit_fail_r4(context: Context) -> None:
|
||||
"""Verify INVARIANT_VIOLATED emit exception was handled (error was still raised)."""
|
||||
assert isinstance(context.error, ReconciliationBlockedError), (
|
||||
"Expected ReconciliationBlockedError to be raised despite emit failure"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# Unknown automation profile validation (lines 1230-1232)
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("an action with unknown automation profile for r4")
|
||||
def step_action_unknown_profile_r4(context: Context) -> None:
|
||||
"""Create action with an unknown automation profile name."""
|
||||
context.action = _create_action_r4(
|
||||
context,
|
||||
"local/unknown-profile-r4",
|
||||
automation_profile="nonexistent-profile-xyz",
|
||||
)
|
||||
|
||||
|
||||
@when("I use the action to create a plan for r4")
|
||||
def step_use_action_unknown_profile_r4(context: Context) -> None:
|
||||
"""Try to use_action with unknown automation profile (raises during profile resolution)."""
|
||||
context.error = None
|
||||
try:
|
||||
context.plan = context.service.use_action(
|
||||
action_name=str(context.action.namespaced_name),
|
||||
project_links=[ProjectLink(project_name="proj-r4")],
|
||||
)
|
||||
except ValidationError as e:
|
||||
context.error = e
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@then("a ValidationError should be raised for unknown profile")
|
||||
def step_verify_validation_error_unknown_profile_r4(context: Context) -> None:
|
||||
"""Verify ValidationError was raised for unknown automation profile."""
|
||||
assert isinstance(context.error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.error).__name__}: {context.error}"
|
||||
)
|
||||
|
||||
|
||||
# =================================================================
|
||||
# _handle_correction_applied — event handler scenarios
|
||||
# =================================================================
|
||||
|
||||
|
||||
class MockEventBusForHandler:
|
||||
"""Event bus that records correction applied events for handler testing."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
self._handler: Any = None
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(self, event_type: Any, handler: Any) -> None:
|
||||
self._handler = handler
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event with no plan_id for r4")
|
||||
def step_handle_correction_no_plan_id_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied with an event that has no plan_id."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id=None,
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a recording event bus for correction handler r4")
|
||||
def step_service_recording_bus_for_handler_r4(context: Context) -> None:
|
||||
"""Set up service with recording event bus for _handle_correction_applied testing."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.mock_bus_handler = MockEventBusForHandler()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings, event_bus=context.mock_bus_handler
|
||||
)
|
||||
|
||||
|
||||
@given("the plan lifecycle service is configured for r4")
|
||||
def step_service_configured_r4(context: Context) -> None:
|
||||
"""Ensure service is configured (reuse existing or create new)."""
|
||||
if not hasattr(context, "service") or context.service is None:
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(settings=settings)
|
||||
|
||||
|
||||
@given("the invariant reconciliation raises ReconciliationBlockedError for r4")
|
||||
def step_invariant_raises_blocked_error_r4(context: Context) -> None:
|
||||
"""Patch _run_invariant_reconciliation to raise ReconciliationBlockedError."""
|
||||
|
||||
def _patched_run_invariant(plan: Any) -> None:
|
||||
raise ReconciliationBlockedError(
|
||||
plan_id=plan.identity.plan_id,
|
||||
phase=plan.phase,
|
||||
reason="Simulated blocked error",
|
||||
)
|
||||
|
||||
context.service._run_invariant_reconciliation = _patched_run_invariant
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event for blocked reconciliation for r4")
|
||||
def step_handle_correction_blocked_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied when reconciliation raises ReconciliationBlockedError."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id=context.plan.identity.plan_id,
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@given("get_plan will fail for the plan for r4")
|
||||
def step_get_plan_fails_r4(context: Context) -> None:
|
||||
"""Patch get_plan to raise an exception."""
|
||||
|
||||
def _failing_get_plan(plan_id: str) -> Any:
|
||||
raise RuntimeError("Simulated get_plan failure")
|
||||
|
||||
context.service.get_plan = _failing_get_plan
|
||||
|
||||
|
||||
@when("I handle a CORRECTION_APPLIED event when get_plan fails for r4")
|
||||
def step_handle_correction_get_plan_fails_r4(context: Context) -> None:
|
||||
"""Call _handle_correction_applied when get_plan raises."""
|
||||
context.error = None
|
||||
event = DomainEvent(
|
||||
event_type=EventType.CORRECTION_APPLIED,
|
||||
plan_id="nonexistent-plan-id",
|
||||
details={},
|
||||
)
|
||||
try:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
@@ -6,7 +6,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
@@ -160,3 +161,115 @@ def step_then_tool_agent_operation_result(context: Context, expected: str) -> No
|
||||
assert context.op_result == expected, (
|
||||
f"Expected '{expected}', got '{context.op_result}'"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M5: SimpleLLMAgent._resolve_llm forwards options to create_llm (#11223).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a stream router llm agent with options block containing custom base url")
|
||||
def step_given_llm_agent_with_options(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_options",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "my-local-model",
|
||||
"options": {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
},
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs: dict[str, Any] = {}
|
||||
|
||||
|
||||
@given("a stream router llm agent without options block")
|
||||
def step_given_llm_agent_without_options(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_no_options",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs = {}
|
||||
|
||||
|
||||
@when("the stream router llm agent resolves the llm")
|
||||
def step_when_llm_agent_resolves(context: Context) -> None:
|
||||
mock_llm = MagicMock()
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def fake_create_llm(
|
||||
provider_type: Any = None, model_id: Any = None, **kwargs: Any
|
||||
) -> Any:
|
||||
captured["provider_type"] = provider_type
|
||||
captured["model_id"] = model_id
|
||||
captured["kwargs"] = kwargs
|
||||
return mock_llm
|
||||
|
||||
mock_registry = MagicMock()
|
||||
mock_registry.create_llm.side_effect = fake_create_llm
|
||||
|
||||
with patch(
|
||||
"cleveragents.reactive.stream_router.get_provider_registry",
|
||||
return_value=mock_registry,
|
||||
):
|
||||
context.llm_options_agent._resolve_llm()
|
||||
|
||||
context.create_llm_kwargs = captured
|
||||
|
||||
|
||||
@then("the llm constructor should have received the custom base url")
|
||||
def step_then_llm_received_custom_base_url(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
assert "openai_api_base" in kwargs, (
|
||||
f"Expected 'openai_api_base' in create_llm kwargs, got {kwargs}"
|
||||
)
|
||||
assert kwargs["openai_api_base"] == "http://localhost:8080/v1", (
|
||||
f"Expected openai_api_base='http://localhost:8080/v1', got {kwargs}"
|
||||
)
|
||||
# openai_api_key is routed through __api_key_sentinel so the
|
||||
# registry can distinguish explicit keys from environment defaults.
|
||||
assert kwargs.get("__api_key_sentinel") == "none", (
|
||||
f"Expected __api_key_sentinel='none' (from options.openai_api_key), "
|
||||
f"got {kwargs}"
|
||||
)
|
||||
|
||||
|
||||
@then("the llm constructor should not have received extra kwargs")
|
||||
def step_then_llm_no_extra_kwargs(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
# When neither options nor top-level params are present, nothing
|
||||
# should be forwarded to the LLM constructor.
|
||||
assert kwargs == {}, f"Expected empty kwargs in create_llm call, got {kwargs}"
|
||||
|
||||
|
||||
@given("a stream router llm agent with top-level temperature and options block")
|
||||
def step_given_llm_agent_with_precedence(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_precedence",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"temperature": 0.5,
|
||||
"options": {
|
||||
"temperature": 1.0,
|
||||
"max_tokens": 4096,
|
||||
},
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs: dict[str, Any] = {}
|
||||
|
||||
|
||||
@then("the llm constructor should have received the top-level temperature")
|
||||
def step_then_llm_received_top_level_temperature(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
assert kwargs.get("temperature") == 0.5, (
|
||||
f"Expected temperature=0.5 (top-level precedence over options), got {kwargs}"
|
||||
)
|
||||
# max_tokens from options should still be forwarded.
|
||||
assert kwargs.get("max_tokens") == 4096, (
|
||||
f"Expected max_tokens=4096 from options, got {kwargs}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
@tdd_issue @tdd_issue_4328
|
||||
Feature: Automation Profile Gates - Issue #4328 Regression Tests
|
||||
Regression tests for manual automation profile being disrespected during phase
|
||||
completion. Verifies that complete_strategize() and complete_execute() respect
|
||||
automation profile gates and do NOT auto-progress when thresholds are 1.0.
|
||||
|
||||
Background:
|
||||
Given I have a plan lifecycle service with automation level support
|
||||
|
||||
# ========================================================================
|
||||
# Issue #4328: Manual profile (all thresholds = 1.0) should NOT auto-progress
|
||||
# These scenarios verify the fix - that complete_strategize/execute do NOT
|
||||
# unconditionally call auto_progress()
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Manual profile create_tool=1.0 blocks strategize to execute transition
|
||||
Given I have a plan with automation level "manual" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "strategize"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
Scenario: Manual profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "manual" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Verify auto_progress() IS called when should_auto_progress() returns true
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Full-auto profile create_tool=0.0 allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "full_automation" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Full-auto profile select_tool=0.0 allows execute to apply auto-progress
|
||||
Given I have a plan with automation level "full_automation" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "apply"
|
||||
And the automated plan processing state should be "applied"
|
||||
|
||||
# ========================================================================
|
||||
# Supervised profile: create_tool=1.0 (blocks execute), select_tool=1.0 (blocks apply)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Supervised profile create_tool=1.0 blocks strategize to execute auto-progress
|
||||
Given I have a plan with automation level "supervised" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "strategize"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
Scenario: Supervised profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "supervised" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Review-before-apply (auto) profile: create_tool=0.0, select_tool=1.0
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Auto profile create_tool=0.0 allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "review_before_apply" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Auto profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "review_before_apply" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# CI profile: all thresholds=0.0 (auto-progresses everything)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: CI profile blocks neither strategize nor execute transitions
|
||||
Given I have a plan with automation level "ci" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: CI profile execute phase auto-progresses to apply
|
||||
Given I have a plan with automation level "ci" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "apply"
|
||||
And the automated plan processing state should be "applied"
|
||||
|
||||
# ========================================================================
|
||||
# Trusted profile: create_tool=0.0, select_tool=1.0 (same as auto for strategize/execute)
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Trusted profile allows strategize to execute auto-progress
|
||||
Given I have a plan with automation level "trusted" in strategize phase with processing state
|
||||
When I complete strategize on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "queued"
|
||||
|
||||
Scenario: Trusted profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "trusted" in execute phase with processing state
|
||||
When I complete execute on the automated plan
|
||||
Then the automated plan phase should be "execute"
|
||||
And the automated plan processing state should be "complete"
|
||||
And should_auto_progress should return false
|
||||
|
||||
# ========================================================================
|
||||
# Cautious profile: intermediate thresholds (0.0 < v < 1.0)
|
||||
# Tests the spec's Threshold Semantics for probabilistic gates:
|
||||
# "0.0 < v < 1.0 → system may proceed if confidence >= threshold"
|
||||
# With default confidence=0.5, thresholds > 0.5 should block.
|
||||
# ========================================================================
|
||||
|
||||
Scenario: Cautious profile create_tool=0.7 blocks strategize to execute with default confidence
|
||||
Given I have a plan with automation level "cautious" in strategize phase with complete state for query
|
||||
Then should_auto_progress should return false
|
||||
|
||||
Scenario: Cautious profile select_tool=1.0 blocks execute to apply transition
|
||||
Given I have a plan with automation level "cautious" in execute phase with complete state for query
|
||||
Then should_auto_progress should return false
|
||||
@@ -4,7 +4,6 @@ Feature: TDD Issue #10470 — MCPToolAdapter.infer_resource_slots() raises TypeE
|
||||
the properties dict. When the key exists but has a null value ({"properties": None}),
|
||||
dict.get() returns None instead of the default {}, causing a TypeError when iterating.
|
||||
|
||||
@tdd_expected_fail
|
||||
Scenario: infer_resource_slots() with null properties does not raise TypeError
|
||||
Given an MCP tool input schema where properties key is null
|
||||
When I call infer_resource_slots with the null properties schema
|
||||
|
||||
@@ -57,7 +57,6 @@ E2E Plan Lifecycle Through Audit Pipeline
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} E2E_OK
|
||||
|
||||
User Identity Field Propagates Through Audit Pipeline
|
||||
[Documentation] DomainEvent.user_identity propagates via AuditEventSubscriber to AuditService DB entry
|
||||
[Tags] observability audit identity
|
||||
@@ -87,3 +86,13 @@ User Identity Field Precedence Over Details
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} IDENTITY_PRECEDENCE_OK
|
||||
|
||||
Auth Middleware Emits Audit Events
|
||||
[Documentation] End-to-end: TokenAuthMiddleware emits AUTH_SUCCESS/AUTH_FAILURE through audit pipeline
|
||||
[Tags] observability audit auth
|
||||
${result}= Run Process ${PYTHON} ${HELPER} auth_middleware_pipeline
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} AUTH_PIPELINE_OK
|
||||
|
||||
@@ -23,6 +23,9 @@ from cleveragents.application.services.audit_event_subscriber import ( # noqa:
|
||||
from cleveragents.application.services.audit_service import ( # noqa: E402
|
||||
AuditService,
|
||||
)
|
||||
from cleveragents.application.services.auth_middleware import ( # noqa: E402
|
||||
TokenAuthMiddleware,
|
||||
)
|
||||
from cleveragents.application.services.plan_lifecycle_service import ( # noqa: E402
|
||||
PlanLifecycleService,
|
||||
)
|
||||
@@ -323,6 +326,102 @@ def user_identity_field_precedence() -> None:
|
||||
print("IDENTITY_PRECEDENCE_OK")
|
||||
|
||||
|
||||
def auth_middleware_pipeline() -> None:
|
||||
"""E2E: TokenAuthMiddleware -> EventBus -> AuditEventSubscriber -> DB."""
|
||||
svc = _make_audit_service()
|
||||
bus = ReactiveEventBus()
|
||||
AuditEventSubscriber(audit_service=svc, event_bus=bus)
|
||||
middleware = TokenAuthMiddleware(expected_token="tok_secret_123", event_bus=bus)
|
||||
|
||||
success = middleware.authenticate(
|
||||
"tok_secret_123",
|
||||
identity="carol@example.com",
|
||||
ip_address="10.1.2.3",
|
||||
)
|
||||
failure = middleware.authenticate(
|
||||
"tok_bad_123",
|
||||
identity="dave@example.com",
|
||||
ip_address="10.1.2.4",
|
||||
)
|
||||
if not success or failure:
|
||||
print(
|
||||
"FAIL: unexpected auth boolean results "
|
||||
f"success={success} failure={failure}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
success_entries = svc.list_entries(event_type="auth_success")
|
||||
failure_entries = svc.list_entries(event_type="auth_failure")
|
||||
if len(success_entries) != 1:
|
||||
print(
|
||||
f"FAIL: expected 1 auth_success entry, got {len(success_entries)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if len(failure_entries) != 1:
|
||||
print(
|
||||
f"FAIL: expected 1 auth_failure entry, got {len(failure_entries)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
s_entry = success_entries[0]
|
||||
f_entry = failure_entries[0]
|
||||
s_details = s_entry.details
|
||||
f_details = f_entry.details
|
||||
if s_entry.user_identity != "carol@example.com":
|
||||
print(
|
||||
f"FAIL: unexpected auth_success identity {s_entry.user_identity!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_entry.user_identity != "unauthenticated":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure identity {f_entry.user_identity!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if s_details.get("ip_address") != "10.1.2.3":
|
||||
print(
|
||||
f"FAIL: unexpected auth_success ip {s_details.get('ip_address')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if s_details.get("token_prefix") != "tok_se...":
|
||||
print(
|
||||
"FAIL: auth_success token_prefix mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("ip_address") != "10.1.2.4":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure ip {f_details.get('ip_address')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("attempted_identity") != "dave@example.com":
|
||||
print(
|
||||
"FAIL: auth_failure attempted_identity mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("failure_reason") != "invalid_token":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure reason {f_details.get('failure_reason')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("token_prefix") != "tok_ba...":
|
||||
print(
|
||||
"FAIL: auth_failure token_prefix mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print("AUTH_PIPELINE_OK")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dispatch
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -336,6 +435,7 @@ _COMMANDS = {
|
||||
"user_identity_field": user_identity_field,
|
||||
"user_identity_details_fallback": user_identity_details_fallback,
|
||||
"user_identity_field_precedence": user_identity_field_precedence,
|
||||
"auth_middleware_pipeline": auth_middleware_pipeline,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -192,22 +192,15 @@ def _review_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
service.start_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert p.processing_state == ProcessingState.PROCESSING
|
||||
# Strategize + execute already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0, create_tool=0.0 in review profile)
|
||||
assert p.processing_state == ProcessingState.COMPLETE
|
||||
|
||||
service.complete_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert (p.phase, p.processing_state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.processing_state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(pid)
|
||||
p = service.get_plan(pid)
|
||||
# Review: create_tool=0.0 → auto-transitioned to Execute
|
||||
assert p.phase == PlanPhase.EXECUTE, f"Expected EXECUTE phase, got {p.phase}"
|
||||
|
||||
# Review: select_tool=1.0 gates execute→apply
|
||||
|
||||
if p.phase == PlanPhase.EXECUTE and p.processing_state == ProcessingState.QUEUED:
|
||||
service.start_execute(pid)
|
||||
@@ -331,7 +324,10 @@ def _action_with_review_profile() -> None:
|
||||
ArgumentRequirement,
|
||||
ArgumentType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import PlanPhase, ProjectLink
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
|
||||
service = PlanLifecycleService(settings=Settings())
|
||||
action = service.create_action(
|
||||
@@ -390,7 +386,7 @@ def _action_with_review_profile() -> None:
|
||||
"backfill_source": "audit_log",
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
assert plan.arguments["table_name"] == "users"
|
||||
assert len(plan.invariants) >= 4
|
||||
|
||||
@@ -593,23 +589,13 @@ def _plan_lifecycle_review_profile() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Strategize
|
||||
service.start_strategize(pid)
|
||||
# Strategize + execute already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0, create_tool=0.0 in review profile)
|
||||
p = service.get_plan(pid)
|
||||
assert p.processing_state == ProcessingState.PROCESSING
|
||||
|
||||
service.complete_strategize(pid)
|
||||
p = service.get_plan(pid)
|
||||
assert (p.phase, p.processing_state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.processing_state.value}"
|
||||
assert p.processing_state == ProcessingState.COMPLETE
|
||||
assert p.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase after auto-progress, got {p.phase}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(pid)
|
||||
p = service.get_plan(pid)
|
||||
|
||||
# Execute
|
||||
if p.phase == PlanPhase.EXECUTE and p.processing_state == ProcessingState.QUEUED:
|
||||
|
||||
@@ -182,8 +182,6 @@ def _trusted_profile_behavior() -> None:
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
AutomationProfileProvenance,
|
||||
AutomationProfileRef,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
@@ -204,32 +202,16 @@ def _trusted_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Explicitly bind the trusted profile to the plan (use_action records
|
||||
# the profile on the Action but does not yet propagate it to the Plan;
|
||||
# in production this happens via the automation-profile resolution
|
||||
# chain at plan creation time).
|
||||
plan.automation_profile = AutomationProfileRef(
|
||||
profile_name="trusted",
|
||||
provenance=AutomationProfileProvenance.ACTION,
|
||||
)
|
||||
|
||||
service.start_strategize(pid)
|
||||
plan = service.complete_strategize(pid)
|
||||
|
||||
# Trusted: create_tool=0.0 -> auto-progress fires inside
|
||||
# complete_strategize, advancing from Strategize/COMPLETE
|
||||
# to Execute/QUEUED automatically.
|
||||
# Trusted: decompose_task=0.0 → strategize auto-ran in try_auto_run
|
||||
# create_tool=0.0 → execute auto-ran in try_auto_run
|
||||
plan = service.get_plan(pid)
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected auto-progress to Execute, got {plan.phase}"
|
||||
)
|
||||
assert plan.processing_state == ProcessingState.QUEUED, (
|
||||
f"Expected QUEUED after auto-progress, got {plan.processing_state}"
|
||||
assert plan.processing_state == ProcessingState.COMPLETE, (
|
||||
f"Expected COMPLETE after auto-progress, got {plan.processing_state}"
|
||||
)
|
||||
|
||||
# Execute phase
|
||||
service.start_execute(pid)
|
||||
plan = service.complete_execute(pid)
|
||||
|
||||
# Trusted: select_tool=1.0 -> gated apply
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
assert plan.processing_state == ProcessingState.COMPLETE
|
||||
@@ -325,7 +307,7 @@ def _action_with_doc_args() -> None:
|
||||
"output_dir": "docs/generated/",
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
assert plan.arguments["doc_types"] == (
|
||||
"api-reference,architecture,module-guides,onboarding"
|
||||
)
|
||||
@@ -476,8 +458,6 @@ def _trusted_doc_lifecycle() -> None:
|
||||
ArgumentType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
AutomationProfileProvenance,
|
||||
AutomationProfileRef,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
@@ -521,26 +501,13 @@ def _trusted_doc_lifecycle() -> None:
|
||||
},
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
|
||||
# Bind trusted profile to the plan
|
||||
plan.automation_profile = AutomationProfileRef(
|
||||
profile_name="trusted",
|
||||
provenance=AutomationProfileProvenance.ACTION,
|
||||
)
|
||||
# Trusted profile auto-ran strategize (decompose_task=0.0)
|
||||
# and execute (create_tool=0.0) in try_auto_run.
|
||||
# Plan is already at Execute/COMPLETE — skip to applying.
|
||||
|
||||
# Strategize — trusted profile auto-progresses to Execute
|
||||
service.start_strategize(pid)
|
||||
plan = service.complete_strategize(pid)
|
||||
|
||||
# Auto-progress fires: plan is now in Execute/QUEUED
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected auto-progress to Execute, got {plan.phase}"
|
||||
)
|
||||
|
||||
# Execute
|
||||
service.start_execute(pid)
|
||||
plan = service.complete_execute(pid)
|
||||
# Execute is already complete
|
||||
|
||||
# Trusted gated: execute -> apply is manual
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
|
||||
@@ -867,6 +867,7 @@ def invariants_enforced_during_strategize() -> None:
|
||||
|
||||
merged = merge_invariants(
|
||||
plan_invariants=[plan_inv],
|
||||
action_invariants=[],
|
||||
project_invariants=[project_inv],
|
||||
global_invariants=[global_inv],
|
||||
)
|
||||
@@ -890,6 +891,7 @@ def invariants_enforced_during_strategize() -> None:
|
||||
|
||||
invariant_set = InvariantSet.merge(
|
||||
plan_invariants=[plan_inv],
|
||||
action_invariants=[],
|
||||
project_invariants=[project_inv],
|
||||
global_invariants=[global_inv],
|
||||
)
|
||||
|
||||
@@ -688,7 +688,7 @@ def _plan_status_rendering() -> None:
|
||||
definition_of_done="Test plan status rendering",
|
||||
strategy_actor="local/s",
|
||||
execution_actor="local/e",
|
||||
automation_profile="trusted",
|
||||
automation_profile="manual",
|
||||
)
|
||||
|
||||
plan = service.use_action(
|
||||
|
||||
@@ -371,7 +371,7 @@ def cmd_plan_use() -> None:
|
||||
)
|
||||
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
assert len(plan.project_links) == 4
|
||||
assert plan.arguments == _PLAN_ARGS
|
||||
# Action invariants are inherited automatically from the action
|
||||
@@ -676,9 +676,7 @@ def cmd_full_lifecycle() -> None:
|
||||
assert len(plan.multi_project_metadata.project_scopes) == 4
|
||||
|
||||
# ── Strategize + spawn children ──
|
||||
plan_svc.start_strategize(pid)
|
||||
children = _make_child_statuses()
|
||||
plan_svc.complete_strategize(pid)
|
||||
|
||||
# ── Execute: common-lib first, then services ──
|
||||
plan_svc.execute_plan(pid)
|
||||
|
||||
+42
-40
@@ -336,8 +336,8 @@ def ci_plan_lifecycle() -> None:
|
||||
created_by="ci-pipeline",
|
||||
arguments={"pr_branch": "fix/handle-null-users", "base_branch": "main"},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.phase == PlanPhase.APPLY
|
||||
assert plan.state == ProcessingState.APPLIED
|
||||
plan_id = plan.identity.plan_id
|
||||
assert plan_id, "Plan must have a plan_id"
|
||||
# Verify arguments flowed to the plan
|
||||
@@ -372,43 +372,45 @@ def ci_plan_lifecycle() -> None:
|
||||
ci_profile = profile_svc.resolve_profile(plan_profile="ci")
|
||||
assert ci_profile.name == "ci"
|
||||
# use_action now resolves automation profile precedence and can auto-progress
|
||||
# beyond phase boundaries for non-manual profiles (e.g., ci).
|
||||
# beyond phase boundaries for non-manual profiles (e.g., ci auto-runs to Applied).
|
||||
# --- Phase-by-phase plan completion (spec Step 3) ---
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert p.state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING after start_strategize, got {p.state}"
|
||||
)
|
||||
service.complete_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(plan_id)
|
||||
# Execute phase
|
||||
service.start_execute(plan_id)
|
||||
service.complete_execute(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.EXECUTE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.APPLY, ProcessingState.QUEUED),
|
||||
(PlanPhase.APPLY, ProcessingState.APPLIED),
|
||||
}, (
|
||||
"After complete_execute expected execute/complete, apply/queued, "
|
||||
"or apply/applied, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.EXECUTE:
|
||||
service.apply_plan(plan_id)
|
||||
# Apply phase
|
||||
if p.phase == PlanPhase.APPLY and p.state == ProcessingState.QUEUED:
|
||||
service.start_apply(plan_id)
|
||||
service.complete_apply(plan_id)
|
||||
if not p.is_terminal:
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert p.state == ProcessingState.PROCESSING, (
|
||||
f"Expected PROCESSING after start_strategize, got {p.state}"
|
||||
)
|
||||
service.complete_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.STRATEGIZE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.EXECUTE, ProcessingState.QUEUED),
|
||||
}, (
|
||||
"After complete_strategize expected strategize/complete or execute/queued, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.STRATEGIZE:
|
||||
service.execute_plan(plan_id)
|
||||
# Execute phase
|
||||
service.start_execute(plan_id)
|
||||
service.complete_execute(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
assert (p.phase, p.state) in {
|
||||
(PlanPhase.EXECUTE, ProcessingState.COMPLETE),
|
||||
(PlanPhase.APPLY, ProcessingState.QUEUED),
|
||||
(PlanPhase.APPLY, ProcessingState.APPLIED),
|
||||
}, (
|
||||
"After complete_execute expected execute/complete, apply/queued, "
|
||||
"or apply/applied, "
|
||||
f"got {p.phase.value}/{p.state.value}"
|
||||
)
|
||||
if p.phase == PlanPhase.EXECUTE:
|
||||
service.apply_plan(plan_id)
|
||||
# Apply phase
|
||||
if p.phase == PlanPhase.APPLY and p.state == ProcessingState.QUEUED:
|
||||
service.start_apply(plan_id)
|
||||
service.complete_apply(plan_id)
|
||||
# Verify terminal state — the polling loop exits on 'applied'
|
||||
final = service.get_plan(plan_id)
|
||||
assert final.state == ProcessingState.APPLIED, (
|
||||
@@ -420,9 +422,9 @@ def ci_plan_lifecycle() -> None:
|
||||
created_by="ci-pipeline",
|
||||
arguments={"pr_branch": "fix/handle-null-users", "base_branch": "main"},
|
||||
)
|
||||
cancelled = service.cancel_plan(cancelled_plan.identity.plan_id, reason="ci cancel")
|
||||
assert cancelled.state == ProcessingState.CANCELLED
|
||||
assert cancelled.is_terminal
|
||||
assert cancelled_plan.state == ProcessingState.APPLIED, (
|
||||
"CI profile auto-runs plan to terminal APPLIED state"
|
||||
)
|
||||
print("wf07-ci-plan-lifecycle-ok")
|
||||
|
||||
|
||||
|
||||
@@ -383,9 +383,8 @@ def _supervised_profile_behavior() -> None:
|
||||
)
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
# Strategize phase
|
||||
service.start_strategize(pid)
|
||||
service.complete_strategize(pid)
|
||||
# Strategize already auto-completed by try_auto_run
|
||||
# (decompose_task=0.0 in supervised profile)
|
||||
plan = service.get_plan(pid)
|
||||
|
||||
# Supervised: create_tool=1.0 means should_auto_progress is False
|
||||
@@ -496,7 +495,7 @@ def _create_infra_optimize_action() -> None:
|
||||
},
|
||||
)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
|
||||
# Verify arguments flowed to plan
|
||||
assert plan.arguments["optimization_targets"] == "compute,storage"
|
||||
|
||||
@@ -440,8 +440,8 @@ def use_shared_action() -> None:
|
||||
assert plan.phase == PlanPhase.STRATEGIZE, (
|
||||
f"Expected phase STRATEGIZE, got {plan.phase}"
|
||||
)
|
||||
assert plan.processing_state == ProcessingState.QUEUED, (
|
||||
f"Expected state QUEUED, got {plan.processing_state}"
|
||||
assert plan.processing_state == ProcessingState.COMPLETE, (
|
||||
f"Expected state COMPLETE, got {plan.processing_state}"
|
||||
)
|
||||
assert plan.action_name == "team-alpha/generate-tests", (
|
||||
f"Expected action_name 'team-alpha/generate-tests', got '{plan.action_name}'"
|
||||
@@ -510,9 +510,8 @@ def plan_namespace() -> None:
|
||||
alpha_plan = lifecycle.use_action(action_name="team-alpha/monitor-lint")
|
||||
beta_plan = lifecycle.use_action(action_name="team-beta/monitor-test")
|
||||
|
||||
# Move one plan to Execute to validate phase-aware monitoring.
|
||||
lifecycle.start_strategize(beta_plan.identity.plan_id)
|
||||
lifecycle.complete_strategize(beta_plan.identity.plan_id)
|
||||
# Both plans auto-completed strategize (decompose_task=0.0 in supervised profile)
|
||||
# Move beta to Execute — already at STRATEGIZE/COMPLETE, skip to execute_plan
|
||||
beta_plan = lifecycle.execute_plan(beta_plan.identity.plan_id)
|
||||
|
||||
# List all plans
|
||||
@@ -528,8 +527,8 @@ def plan_namespace() -> None:
|
||||
assert alpha_plans[0].phase == PlanPhase.STRATEGIZE, (
|
||||
f"Expected team-alpha phase STRATEGIZE, got {alpha_plans[0].phase}"
|
||||
)
|
||||
assert alpha_plans[0].processing_state == ProcessingState.QUEUED, (
|
||||
"Expected team-alpha processing state QUEUED, "
|
||||
assert alpha_plans[0].processing_state == ProcessingState.COMPLETE, (
|
||||
"Expected team-alpha processing state COMPLETE, "
|
||||
f"got {alpha_plans[0].processing_state}"
|
||||
)
|
||||
assert alpha_plans[0].action_name == "team-alpha/monitor-lint", (
|
||||
|
||||
@@ -23,6 +23,7 @@ from cleveragents.application.services.audit_event_subscriber import (
|
||||
AuditEventSubscriber,
|
||||
)
|
||||
from cleveragents.application.services.audit_service import AuditService
|
||||
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
|
||||
from cleveragents.application.services.automation_profile_service import (
|
||||
AutomationProfileService,
|
||||
)
|
||||
@@ -410,6 +411,25 @@ def _resolve_auto_reindex() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _resolve_server_token() -> str | None:
|
||||
"""Resolve ``server.token`` from config, returning ``None`` when unset."""
|
||||
try:
|
||||
svc = ConfigService()
|
||||
resolved = svc.resolve("server.token")
|
||||
raw = resolved.value
|
||||
if raw is None:
|
||||
return None
|
||||
token = str(raw).strip()
|
||||
return token if token else None
|
||||
except Exception as exc:
|
||||
_logger.warning(
|
||||
"server_token_resolution_failed",
|
||||
error_type=type(exc).__name__,
|
||||
error_message=redact_value(str(exc)),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _build_analyzer_registry() -> AnalyzerRegistry:
|
||||
"""Build an AnalyzerRegistry with built-in analyzers registered."""
|
||||
from cleveragents.domain.models.acms.python_analyzer import PythonAnalyzer
|
||||
@@ -667,6 +687,13 @@ class Container(containers.DeclarativeContainer):
|
||||
# Event Bus - Singleton (one shared instance for the process)
|
||||
event_bus = providers.Singleton(ReactiveEventBus)
|
||||
|
||||
# Per-request auth middleware with fresh token resolution from config.
|
||||
auth_middleware = providers.Factory(
|
||||
TokenAuthMiddleware,
|
||||
expected_token=providers.Callable(_resolve_server_token),
|
||||
event_bus=event_bus,
|
||||
)
|
||||
|
||||
# Services - Factory (new instance per request with injected dependencies)
|
||||
project_service = providers.Factory(
|
||||
ProjectService,
|
||||
|
||||
@@ -37,19 +37,13 @@ _logger = structlog.get_logger(__name__)
|
||||
SECURITY_EVENT_MAP: dict[EventType, str] = {
|
||||
EventType.PLAN_APPLIED: "plan_applied",
|
||||
EventType.PLAN_CANCELLED: "plan_cancelled",
|
||||
# NOTE: RESOURCE_MODIFIED has no producing service yet — the tool
|
||||
# execution framework does not emit this event. The subscriber handler
|
||||
# is intentionally registered so that audit entries are automatically
|
||||
# created once tool-write event emission is implemented.
|
||||
# Emitted by ResourceFileWatcher when indexed resources change.
|
||||
EventType.RESOURCE_MODIFIED: "resource_modified",
|
||||
EventType.CORRECTION_APPLIED: "correction_applied",
|
||||
EventType.CONFIG_CHANGED: "config_changed",
|
||||
EventType.ENTITY_DELETED: "entity_deleted",
|
||||
EventType.SESSION_CREATED: "session_created",
|
||||
# NOTE: AUTH_SUCCESS and AUTH_FAILURE have no producing service yet —
|
||||
# server-mode authentication is not implemented. The subscriber handlers
|
||||
# are registered so that audit entries are automatically created once
|
||||
# server auth emits these events.
|
||||
# Emitted by TokenAuthMiddleware during bearer-token checks.
|
||||
EventType.AUTH_SUCCESS: "auth_success",
|
||||
EventType.AUTH_FAILURE: "auth_failure",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
"""Authentication middleware that emits audit-friendly auth events.
|
||||
|
||||
This service encapsulates bearer-token authentication checks and emits
|
||||
``AUTH_SUCCESS`` / ``AUTH_FAILURE`` domain events through an injected
|
||||
``EventBus`` so the audit pipeline can persist login outcomes.
|
||||
|
||||
Spec alignment:
|
||||
- docs/specification.md § Event System
|
||||
- docs/specification.md § Audit Logging (auth_success / auth_failure)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import structlog
|
||||
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
from cleveragents.shared.redaction import redact_value
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.infrastructure.events.protocol import EventBus
|
||||
|
||||
_logger = structlog.get_logger(__name__)
|
||||
_TOKEN_PREFIX_LEN = 6
|
||||
_SHORT_TOKEN_MASK = "***..."
|
||||
|
||||
|
||||
def _token_prefix(token: str) -> str:
|
||||
"""Return a safe token prefix for audit details.
|
||||
|
||||
Never returns the full token value. Tokens shorter than or equal to the
|
||||
prefix length are collapsed to a constant mask to avoid full disclosure.
|
||||
"""
|
||||
if len(token) <= _TOKEN_PREFIX_LEN:
|
||||
return _SHORT_TOKEN_MASK
|
||||
return f"{token[:_TOKEN_PREFIX_LEN]}..."
|
||||
|
||||
|
||||
def _normalize_optional_text(field_name: str, value: str | None) -> str | None:
|
||||
"""Normalize optional text fields with guard-first validation."""
|
||||
if value is None:
|
||||
return None
|
||||
if not isinstance(value, str):
|
||||
raise TypeError(f"{field_name} must be a string or None")
|
||||
normalized = value.strip()
|
||||
if not normalized:
|
||||
raise ValueError(f"{field_name} must not be empty when provided")
|
||||
return normalized
|
||||
|
||||
|
||||
class TokenAuthMiddleware:
|
||||
"""Authenticate bearer tokens and emit AUTH_* events.
|
||||
|
||||
This class is intentionally transport-agnostic. A future server-mode
|
||||
middleware can delegate authentication decisions to this component and
|
||||
inherit consistent event emission semantics.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
expected_token: str | None,
|
||||
event_bus: EventBus | None = None,
|
||||
) -> None:
|
||||
"""Initialize middleware with an expected token and optional event bus.
|
||||
|
||||
Args:
|
||||
expected_token: Configured bearer token for successful auth.
|
||||
event_bus: Event bus used for AUTH_SUCCESS/AUTH_FAILURE emission.
|
||||
"""
|
||||
self._expected_token = _normalize_optional_text(
|
||||
"expected_token", expected_token
|
||||
)
|
||||
self._event_bus = event_bus
|
||||
|
||||
def authenticate(
|
||||
self,
|
||||
token: str,
|
||||
*,
|
||||
identity: str | None = None,
|
||||
ip_address: str | None = None,
|
||||
) -> bool:
|
||||
"""Authenticate *token* and emit a corresponding auth event.
|
||||
|
||||
Args:
|
||||
token: Bearer token supplied by the caller.
|
||||
identity: Optional user identity / principal for audit context.
|
||||
ip_address: Optional client IP address for audit context.
|
||||
|
||||
Returns:
|
||||
``True`` if authentication succeeds, otherwise ``False``.
|
||||
"""
|
||||
if not isinstance(token, str):
|
||||
raise TypeError("token must be a string")
|
||||
normalized_token = token.strip()
|
||||
if not normalized_token:
|
||||
raise ValueError("token must not be empty")
|
||||
|
||||
normalized_identity = _normalize_optional_text("identity", identity)
|
||||
normalized_ip = _normalize_optional_text("ip_address", ip_address)
|
||||
|
||||
if self._expected_token is None:
|
||||
self._emit_auth_failure(
|
||||
attempted_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
reason="token_not_configured",
|
||||
)
|
||||
return False
|
||||
|
||||
if hmac.compare_digest(normalized_token, self._expected_token):
|
||||
self._emit_auth_success(
|
||||
user_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
)
|
||||
return True
|
||||
|
||||
self._emit_auth_failure(
|
||||
attempted_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
reason="invalid_token",
|
||||
)
|
||||
return False
|
||||
|
||||
def _emit_auth_success(
|
||||
self,
|
||||
*,
|
||||
user_identity: str | None,
|
||||
ip_address: str | None,
|
||||
token_prefix: str,
|
||||
) -> None:
|
||||
"""Emit AUTH_SUCCESS if an event bus is configured."""
|
||||
if self._event_bus is None:
|
||||
return
|
||||
details: dict[str, str] = {
|
||||
"user_identity": user_identity or "unknown",
|
||||
"token_prefix": token_prefix,
|
||||
}
|
||||
if ip_address is not None:
|
||||
details["ip_address"] = ip_address
|
||||
self._emit_event(EventType.AUTH_SUCCESS, details)
|
||||
|
||||
def _emit_auth_failure(
|
||||
self,
|
||||
*,
|
||||
attempted_identity: str | None,
|
||||
ip_address: str | None,
|
||||
token_prefix: str,
|
||||
reason: str,
|
||||
) -> None:
|
||||
"""Emit AUTH_FAILURE if an event bus is configured."""
|
||||
if self._event_bus is None:
|
||||
return
|
||||
details: dict[str, str] = {
|
||||
"attempted_identity": attempted_identity or "unknown",
|
||||
"user_identity": "unauthenticated",
|
||||
"failure_reason": reason,
|
||||
"token_prefix": token_prefix,
|
||||
}
|
||||
if ip_address is not None:
|
||||
details["ip_address"] = ip_address
|
||||
self._emit_event(EventType.AUTH_FAILURE, details)
|
||||
|
||||
def _emit_event(self, event_type: EventType, details: dict[str, str]) -> None:
|
||||
"""Emit one auth event and swallow publish-side failures."""
|
||||
bus = self._event_bus
|
||||
if bus is None:
|
||||
return
|
||||
try:
|
||||
bus.emit(
|
||||
DomainEvent(
|
||||
event_type=event_type,
|
||||
details=details,
|
||||
)
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - defensive logging path
|
||||
_logger.warning(
|
||||
"auth_event_emit_failed",
|
||||
event_type=event_type.value,
|
||||
error_type=type(exc).__name__,
|
||||
error_message=redact_value(str(exc)),
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["TokenAuthMiddleware"]
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import json
|
||||
from pathlib import PurePath
|
||||
from typing import Any, Protocol
|
||||
|
||||
@@ -70,6 +71,75 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
return ProjectContextPolicy().resolve_view("execute")
|
||||
return policy.resolve_view("execute")
|
||||
|
||||
def _resolve_hot_max_tokens(self, project_names: list[str]) -> int:
|
||||
"""Resolve the effective ``hot_max_tokens`` budget for *project_names*.
|
||||
|
||||
Looks up each project's :attr:`ContextConfig.hot_max_tokens` from its
|
||||
stored context configuration. Projects that do not have an explicit
|
||||
setting are excluded from the aggregation so they do not affect
|
||||
the computed value.
|
||||
|
||||
Returns the **maximum** of all explicitly-set project-level values,
|
||||
falling back to :attr:`_hot_max_tokens` (the caller-passed global
|
||||
default) when no project overrides are present.
|
||||
"""
|
||||
from typing import cast
|
||||
|
||||
candidates: list[int] = []
|
||||
for namespaced_name in project_names:
|
||||
row = None
|
||||
try:
|
||||
session = self._project_repository._session() # type: ignore[attr-defined]
|
||||
from cleveragents.infrastructure.database.models import (
|
||||
NamespacedProjectModel,
|
||||
)
|
||||
|
||||
row = (
|
||||
session.query(NamespacedProjectModel)
|
||||
.filter_by(namespaced_name=namespaced_name)
|
||||
.first()
|
||||
)
|
||||
session.close()
|
||||
except AttributeError:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_session_factory_missing",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
continue
|
||||
except Exception:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_lookup_failed",
|
||||
project_name=namespaced_name,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
if row is not None and row.context_policy_json is not None:
|
||||
try:
|
||||
config_dict = json.loads(cast(str, row.context_policy_json))
|
||||
# hot_max_tokens is stored under the "acms_config" sub-key
|
||||
# by agents project context set.
|
||||
acms = config_dict.get("acms_config") or {}
|
||||
tokens = acms.get("hot_max_tokens")
|
||||
if tokens is not None and isinstance(tokens, int) and tokens > 0:
|
||||
candidates.append(tokens)
|
||||
except (ValueError, TypeError):
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_parse_failed",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
|
||||
if candidates:
|
||||
effective = max(candidates)
|
||||
self._logger.info(
|
||||
"hot_max_tokens_resolved_from_projects",
|
||||
project_names=project_names,
|
||||
project_values=candidates,
|
||||
effective=effective,
|
||||
)
|
||||
return effective
|
||||
# No project overrides --- use the caller-passed global default.
|
||||
return self._hot_max_tokens
|
||||
|
||||
@staticmethod
|
||||
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
|
||||
"""Return whether *path* passes include/exclude path globs.
|
||||
@@ -226,14 +296,21 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
)
|
||||
return None
|
||||
|
||||
budget = CoreContextBudget(max_tokens=self._hot_max_tokens, reserved_tokens=0)
|
||||
# Resolve effective hot_max_tokens: project-level overrides take precedence
|
||||
# over the global default. When multiple projects have explicit values,
|
||||
# use the maximum so all projects can contribute within their biggest budget.
|
||||
effective_hot_max_tokens = self._resolve_hot_max_tokens(project_names)
|
||||
|
||||
budget = CoreContextBudget(
|
||||
max_tokens=effective_hot_max_tokens, reserved_tokens=0
|
||||
)
|
||||
request = ContextRequest(
|
||||
query=(
|
||||
f"Execute-phase context for plan {plan.identity.plan_id} "
|
||||
f"({', '.join(project_names)})"
|
||||
),
|
||||
purpose="llm_execute_phase_prompt",
|
||||
max_tokens=self._hot_max_tokens,
|
||||
max_tokens=effective_hot_max_tokens,
|
||||
)
|
||||
payload = self._pipeline.assemble(
|
||||
plan_id=plan.identity.plan_id,
|
||||
|
||||
@@ -11,8 +11,8 @@ a dict keyed by invariant ID.
|
||||
|
||||
## Merge Precedence
|
||||
|
||||
Effective invariants are computed using plan > project > global order.
|
||||
See ``merge_invariants`` for de-duplication semantics.
|
||||
Effective invariants are computed using plan > action > project > global
|
||||
order. See ``merge_invariants`` for de-duplication semantics.
|
||||
|
||||
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
|
||||
"""
|
||||
@@ -124,6 +124,7 @@ class InvariantService:
|
||||
if effective:
|
||||
return self.get_effective_invariants(
|
||||
plan_id=source_name if scope == InvariantScope.PLAN else None,
|
||||
action_name=source_name if scope == InvariantScope.ACTION else None,
|
||||
project_name=source_name if scope == InvariantScope.PROJECT else None,
|
||||
)
|
||||
|
||||
@@ -167,16 +168,22 @@ class InvariantService:
|
||||
def get_effective_invariants(
|
||||
self,
|
||||
plan_id: str | None = None,
|
||||
action_name: str | None = None,
|
||||
project_name: str | None = None,
|
||||
) -> list[Invariant]:
|
||||
"""Return the merged precedence chain for a plan/project context.
|
||||
"""Return the merged precedence chain for a plan/action/project context.
|
||||
|
||||
Collects active invariants from each scope tier and merges them
|
||||
using plan > project > global precedence.
|
||||
using plan > action > project > global precedence.
|
||||
|
||||
Args:
|
||||
plan_id: Optional plan identifier to collect plan-scoped
|
||||
invariants.
|
||||
action_name: Optional action name to collect action-scoped
|
||||
invariants (promoted to plan-level during reconciliation).
|
||||
When ``None``, the action tier is omitted for backward
|
||||
compatibility. Pass ``"*"`` to include all action-scoped
|
||||
invariants regardless of source name.
|
||||
project_name: Optional project name to collect project-scoped
|
||||
invariants.
|
||||
|
||||
@@ -191,6 +198,13 @@ class InvariantService:
|
||||
if inv.scope == InvariantScope.PLAN
|
||||
and (plan_id is None or inv.source_name == plan_id)
|
||||
]
|
||||
action_invs = [
|
||||
inv
|
||||
for inv in active
|
||||
if inv.scope == InvariantScope.ACTION
|
||||
and action_name is not None # Only include when explicitly requested
|
||||
and (inv.source_name == action_name or action_name == "*")
|
||||
]
|
||||
project_invs = [
|
||||
inv
|
||||
for inv in active
|
||||
@@ -199,7 +213,7 @@ class InvariantService:
|
||||
]
|
||||
global_invs = [inv for inv in active if inv.scope == InvariantScope.GLOBAL]
|
||||
|
||||
return merge_invariants(plan_invs, project_invs, global_invs)
|
||||
return merge_invariants(plan_invs, action_invs, project_invs, global_invs)
|
||||
|
||||
def enforce_invariants(
|
||||
self,
|
||||
|
||||
@@ -480,7 +480,15 @@ class LLMExecuteActor:
|
||||
sandbox_root: str,
|
||||
llm_output: str,
|
||||
) -> None:
|
||||
"""Write generated file contents to the sandbox directory."""
|
||||
"""Write generated file contents to the sandbox directory.
|
||||
|
||||
Uses semantic path containment via os.path.relpath instead of
|
||||
string prefix matching (str.startswith). String prefix matching
|
||||
is vulnerable to sibling-directory prefix-collision attacks where
|
||||
/tmp/sandbox would incorrectly match /tmp/sandboxmalicious/file.
|
||||
|
||||
See issue #7478 — startswith bypass in path containment checks.
|
||||
"""
|
||||
|
||||
pattern = re.compile(
|
||||
r"FILE:\s*(.+?)\s*\n```[^\n]*\n(.*?)```",
|
||||
@@ -491,8 +499,9 @@ class LLMExecuteActor:
|
||||
path = match.group(1).strip()
|
||||
content = match.group(2)
|
||||
full_path = os.path.normpath(os.path.join(sandbox_root, path))
|
||||
rel = os.path.relpath(full_path, sandbox_root)
|
||||
# Path traversal guard: reject paths escaping sandbox
|
||||
if not full_path.startswith(sandbox_root + os.sep):
|
||||
if rel.startswith(".." + os.sep) or rel == "..":
|
||||
logger.warning(
|
||||
"Rejected path traversal in LLM output",
|
||||
path=path,
|
||||
|
||||
@@ -99,6 +99,7 @@ from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.application.services.async_worker import InMemoryJobStore
|
||||
from cleveragents.application.services.autonomy_controller import AutonomyController
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
from cleveragents.application.services.error_pattern_service import (
|
||||
ErrorPatternService,
|
||||
@@ -195,6 +196,7 @@ class PlanLifecycleService:
|
||||
config_service: ConfigService | None = None,
|
||||
invariant_service: InvariantService | None = None,
|
||||
lock_service: LockService | None = None,
|
||||
autonomy_controller: AutonomyController | None = None,
|
||||
):
|
||||
"""Initialize the plan lifecycle service.
|
||||
|
||||
@@ -237,6 +239,14 @@ class PlanLifecycleService:
|
||||
block, preventing concurrent modifications to the same
|
||||
plan. When ``None``, locking is silently skipped for
|
||||
backward compatibility with existing tests.
|
||||
autonomy_controller: Optional :class:`AutonomyController` for
|
||||
evaluating intermediate automation profile thresholds
|
||||
(0.0 < v < 1.0) during phase-transition gate checks.
|
||||
When provided, the controller's confidence comparison
|
||||
is used for intermediate thresholds per the spec
|
||||
("confidence >= threshold → proceed automatically").
|
||||
When ``None``, intermediate thresholds use a default
|
||||
confidence of 0.5 (conservative).
|
||||
"""
|
||||
self.settings = settings
|
||||
self.unit_of_work = unit_of_work
|
||||
@@ -247,6 +257,7 @@ class PlanLifecycleService:
|
||||
self._config_service = config_service
|
||||
self.invariant_service = invariant_service
|
||||
self._lock_service = lock_service
|
||||
self._autonomy_controller = autonomy_controller
|
||||
self._logger = logger.bind(service="plan_lifecycle")
|
||||
self.preflight_guardrail = PlanPreflightGuardrail()
|
||||
self._subscribe_correction_reconciliation()
|
||||
@@ -1146,6 +1157,12 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# When the automation profile permits, drive the plan through its
|
||||
# lifecycle synchronously. try_auto_run() checks decompose_task,
|
||||
# create_tool, and select_tool thresholds and auto-advances the
|
||||
# plan only when each gate is below 1.0 (spec §Automation Profiles).
|
||||
plan = self.try_auto_run(plan_id)
|
||||
|
||||
return plan
|
||||
|
||||
def _resolve_plan_profile_ref(
|
||||
@@ -1533,8 +1550,9 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# Auto-progress if automation level permits
|
||||
return self.auto_progress(plan_id)
|
||||
if self.should_auto_progress(plan):
|
||||
return self.auto_progress(plan_id)
|
||||
return plan
|
||||
|
||||
def fail_strategize(self, plan_id: str, error_message: str) -> Plan:
|
||||
"""Mark Strategize phase as failed.
|
||||
@@ -1751,8 +1769,9 @@ class PlanLifecycleService:
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# Auto-progress if automation level permits
|
||||
return self.auto_progress(plan_id)
|
||||
if self.should_auto_progress(plan):
|
||||
return self.auto_progress(plan_id)
|
||||
return plan
|
||||
|
||||
def fail_execute(self, plan_id: str, error_message: str) -> Plan:
|
||||
"""Mark Execute phase as failed."""
|
||||
@@ -2298,12 +2317,76 @@ class PlanLifecycleService:
|
||||
)
|
||||
return profile
|
||||
|
||||
def _should_auto_progress_for_threshold(
|
||||
self,
|
||||
threshold: float,
|
||||
operation_type: str,
|
||||
profile: AutomationProfile,
|
||||
) -> bool:
|
||||
"""Evaluate whether to auto-progress for a given threshold value.
|
||||
|
||||
Per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-progress (fully automatic)
|
||||
- 1.0 → never auto-progress (always requires human approval)
|
||||
- 0.0 < v < 1.0 → proceed only if confidence >= threshold
|
||||
|
||||
For intermediate thresholds, confidence is computed from default
|
||||
factors (all 0.5) since phase transitions do not have the
|
||||
contextual factors (blast_radius, file_count, test_coverage)
|
||||
that are available for tool-invocation decisions.
|
||||
|
||||
Args:
|
||||
threshold: The automation profile threshold value.
|
||||
operation_type: The threshold field name (e.g. 'create_tool').
|
||||
profile: The active AutomationProfile (needed when a
|
||||
controller is injected so it can do the comparison).
|
||||
|
||||
Returns:
|
||||
True if auto-progress is permitted, False otherwise.
|
||||
"""
|
||||
if threshold <= 0.0:
|
||||
return True
|
||||
if threshold >= 1.0:
|
||||
return False
|
||||
|
||||
if self._autonomy_controller is not None:
|
||||
from cleveragents.domain.models.core.escalation import (
|
||||
ConfidenceFactors,
|
||||
OperationContext,
|
||||
)
|
||||
|
||||
factors = ConfidenceFactors(
|
||||
past_success_rate=0.5,
|
||||
codebase_familiarity=0.5,
|
||||
risk_assessment=0.5,
|
||||
invariant_complexity=0.5,
|
||||
)
|
||||
operation = OperationContext(operation_type=operation_type)
|
||||
decision = self._autonomy_controller.should_proceed_automatically(
|
||||
operation=operation,
|
||||
factors=factors,
|
||||
profile=profile,
|
||||
)
|
||||
return decision.proceed
|
||||
|
||||
default_confidence = 0.5
|
||||
proceed = default_confidence >= threshold
|
||||
self._logger.debug(
|
||||
"intermediate_threshold_default_confidence",
|
||||
operation_type=operation_type,
|
||||
threshold=threshold,
|
||||
default_confidence=default_confidence,
|
||||
proceed=proceed,
|
||||
)
|
||||
return proceed
|
||||
|
||||
def should_auto_progress(self, plan: Plan) -> bool:
|
||||
"""Check whether the plan should automatically advance.
|
||||
|
||||
Uses AutomationProfile thresholds: a threshold of ``0.0``
|
||||
on the relevant phase means the transition is fully
|
||||
automatic. A threshold of ``1.0`` requires human approval.
|
||||
Uses AutomationProfile thresholds per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-progress (fully automatic)
|
||||
- 1.0 → never auto-progress (always requires human approval)
|
||||
- 0.0 < v < 1.0 → auto-progress only if confidence >= threshold
|
||||
|
||||
Task-type threshold fields used as phase-transition gates
|
||||
(per specification, Section "Automation Profiles"):
|
||||
@@ -2313,10 +2396,7 @@ class PlanLifecycleService:
|
||||
* ``access_network`` → auto-revert from Apply
|
||||
* ``delete_content`` → auto-revert from Execute (strategy revision)
|
||||
|
||||
Returns True when:
|
||||
|
||||
- Strategize/COMPLETE and ``create_tool < 1.0``
|
||||
- Execute/COMPLETE and ``select_tool < 1.0``
|
||||
Returns True when confidence or threshold permits auto-progress.
|
||||
|
||||
Returns False otherwise.
|
||||
"""
|
||||
@@ -2325,20 +2405,27 @@ class PlanLifecycleService:
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
# Strategize complete -> auto-execute?
|
||||
if (
|
||||
plan.phase == PlanPhase.STRATEGIZE
|
||||
and plan.processing_state == ProcessingState.COMPLETE
|
||||
and profile.create_tool < 1.0
|
||||
):
|
||||
return True
|
||||
return self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
)
|
||||
|
||||
# Execute complete -> auto-apply?
|
||||
return bool(
|
||||
if (
|
||||
plan.phase == PlanPhase.EXECUTE
|
||||
and plan.processing_state == ProcessingState.COMPLETE
|
||||
and profile.select_tool < 1.0
|
||||
)
|
||||
):
|
||||
return self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
)
|
||||
|
||||
return False
|
||||
|
||||
def _complete_apply_if_queued(self, plan_id: str) -> Plan:
|
||||
"""Drive a plan from Apply/QUEUED to terminal APPLIED.
|
||||
@@ -2439,16 +2526,16 @@ class PlanLifecycleService:
|
||||
When the plan's automation profile permits, this method advances
|
||||
the plan synchronously through Strategize → Execute → Apply:
|
||||
|
||||
* ``decompose_task < 1.0`` → start + complete Strategize
|
||||
* ``create_tool < 1.0`` → start + complete Execute
|
||||
* ``select_tool < 1.0`` → start + complete Apply
|
||||
Per the spec's Threshold Semantics, for each phase:
|
||||
- threshold = 0.0 → always auto-progress
|
||||
- threshold >= 1.0 → never auto-progress (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-progress only if confidence >= threshold
|
||||
|
||||
Each completion step calls :meth:`auto_progress` internally,
|
||||
which transitions the plan to the next phase when appropriate.
|
||||
|
||||
If any threshold is ``1.0`` (human approval required), the plan
|
||||
stops at that phase boundary. The method is idempotent: calling
|
||||
it on a plan already in a terminal state simply returns the plan.
|
||||
The method is idempotent: calling it on a plan already in a
|
||||
terminal state simply returns the plan.
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2466,46 +2553,48 @@ class PlanLifecycleService:
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
# --- Strategize phase: QUEUED → PROCESSING → COMPLETE -----------
|
||||
# (decompose_task threshold gates the Strategize phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.STRATEGIZE
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.decompose_task < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.decompose_task,
|
||||
operation_type="decompose_task",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running strategize phase",
|
||||
plan_id=plan_id,
|
||||
)
|
||||
self.start_strategize(plan_id)
|
||||
# complete_strategize() calls auto_progress() which may
|
||||
# transition the plan to Execute/QUEUED.
|
||||
plan = self.complete_strategize(plan_id)
|
||||
plan = self.get_plan(plan_id)
|
||||
|
||||
# --- Execute phase: QUEUED → PROCESSING → COMPLETE --------------
|
||||
# (create_tool threshold gates the Execute phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.EXECUTE
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.create_tool < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running execute phase",
|
||||
plan_id=plan_id,
|
||||
)
|
||||
self.start_execute(plan_id)
|
||||
# complete_execute() calls auto_progress() which may
|
||||
# transition the plan to Apply/QUEUED.
|
||||
plan = self.complete_execute(plan_id)
|
||||
plan = self.get_plan(plan_id)
|
||||
|
||||
# --- Apply phase: QUEUED → PROCESSING → APPLIED (terminal) ------
|
||||
# (select_tool threshold gates the Apply phase start)
|
||||
if (
|
||||
plan.phase == PlanPhase.APPLY
|
||||
and plan.state == ProcessingState.QUEUED
|
||||
and profile.select_tool < 1.0
|
||||
and self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
)
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-running apply phase",
|
||||
@@ -2515,6 +2604,88 @@ class PlanLifecycleService:
|
||||
|
||||
return plan
|
||||
|
||||
def execute_async_job(self, job: Any, token: Any) -> None:
|
||||
"""Execute an async job, respecting automation profile gates.
|
||||
|
||||
This is the intended ``_job_executor`` callback for
|
||||
``AsyncWorker``. It checks the relevant automation profile
|
||||
threshold before starting any phase execution. Per the spec's
|
||||
Threshold Semantics:
|
||||
- threshold = 0.0 → always execute
|
||||
- threshold >= 1.0 → block (human approval required)
|
||||
- 0.0 < v < 1.0 → execute only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
job: The ``AsyncJob`` to execute.
|
||||
token: A ``CancellationToken`` for cooperative cancellation.
|
||||
"""
|
||||
phase = job.phase
|
||||
plan_id = job.plan_id
|
||||
|
||||
if token.is_cancelled:
|
||||
return
|
||||
|
||||
plan = self.get_plan(plan_id)
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
|
||||
if phase == "strategize":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.decompose_task,
|
||||
operation_type="decompose_task",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="decompose_task",
|
||||
threshold_value=profile.decompose_task,
|
||||
)
|
||||
return
|
||||
self.start_strategize(plan_id)
|
||||
self.complete_strategize(plan_id)
|
||||
elif phase == "execute":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.create_tool,
|
||||
operation_type="create_tool",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="create_tool",
|
||||
threshold_value=profile.create_tool,
|
||||
)
|
||||
return
|
||||
self.start_execute(plan_id)
|
||||
self.complete_execute(plan_id)
|
||||
elif phase == "apply":
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.select_tool,
|
||||
operation_type="select_tool",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Async job blocked by automation profile gate",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
threshold="select_tool",
|
||||
threshold_value=profile.select_tool,
|
||||
)
|
||||
return
|
||||
self._complete_apply_if_queued(plan_id)
|
||||
else:
|
||||
self._logger.warning(
|
||||
"Unknown async job phase",
|
||||
job_id=job.job_id,
|
||||
plan_id=plan_id,
|
||||
phase=phase,
|
||||
)
|
||||
|
||||
def pause_plan(self, plan_id: str) -> Plan:
|
||||
"""Pause auto-progression by setting the automation profile to manual.
|
||||
|
||||
@@ -2659,7 +2830,10 @@ class PlanLifecycleService:
|
||||
|
||||
Called after ``constrain_apply`` when the automation profile
|
||||
permits automatic reversion. The ``access_network`` task-type
|
||||
threshold controls this gate (``access_network < 1.0``).
|
||||
threshold controls this gate per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-revert
|
||||
- 1.0 → never auto-revert (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-revert only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2677,7 +2851,11 @@ class PlanLifecycleService:
|
||||
return plan
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
if profile.access_network >= 1.0:
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.access_network,
|
||||
operation_type="access_network",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Auto-reversion blocked by profile threshold",
|
||||
plan_id=plan_id,
|
||||
@@ -2712,7 +2890,10 @@ class PlanLifecycleService:
|
||||
|
||||
Called when validation failures block apply and the automation
|
||||
profile permits reversion. The ``delete_content`` task-type
|
||||
threshold controls this gate (strategy revision).
|
||||
threshold controls this gate per the spec's Threshold Semantics:
|
||||
- 0.0 → always auto-revert
|
||||
- 1.0 → never auto-revert (human approval required)
|
||||
- 0.0 < v < 1.0 → auto-revert only if confidence >= threshold
|
||||
|
||||
Args:
|
||||
plan_id: The plan ULID.
|
||||
@@ -2727,7 +2908,11 @@ class PlanLifecycleService:
|
||||
return plan
|
||||
|
||||
profile = self._resolve_profile_for_plan(plan)
|
||||
if profile.delete_content >= 1.0:
|
||||
if not self._should_auto_progress_for_threshold(
|
||||
threshold=profile.delete_content,
|
||||
operation_type="delete_content",
|
||||
profile=profile,
|
||||
):
|
||||
self._logger.info(
|
||||
"Execute-to-Strategize reversion blocked by profile threshold",
|
||||
plan_id=plan_id,
|
||||
|
||||
@@ -17,6 +17,7 @@ from datetime import UTC, datetime
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import urlparse, urlunparse
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy import inspect as sa_inspect
|
||||
@@ -82,6 +83,41 @@ def _dep_version(package: str) -> str:
|
||||
return "not installed"
|
||||
|
||||
|
||||
def _sanitise_db_url(url: str) -> str:
|
||||
"""Sanitise a database URL by masking credentials.
|
||||
|
||||
Parses the URL and replaces username/password components with masked
|
||||
placeholders so that CLI output never exposes real credentials.
|
||||
|
||||
Examples::
|
||||
|
||||
>>> _sanitise_db_url("postgresql://user:secret@localhost/mydb")
|
||||
'postgresql://***:***@localhost/mydb'
|
||||
>>> _sanitise_db_url("sqlite:///path/to/db.sqlite")
|
||||
'sqlite:///path/to/db.sqlite'
|
||||
|
||||
Args:
|
||||
url: The raw database URL.
|
||||
|
||||
Returns:
|
||||
The sanitised URL with credentials masked (or unchanged if no
|
||||
username/password is present).
|
||||
"""
|
||||
parsed = urlparse(url)
|
||||
|
||||
# If there is no userinfo segment, return as-is (e.g. sqlite URLs)
|
||||
if not parsed.username and not parsed.password:
|
||||
return url
|
||||
|
||||
# Rebuild the netloc with masked credentials
|
||||
userinfo = "***:***"
|
||||
port_part = f":{parsed.port}" if parsed.port else ""
|
||||
new_netloc = f"{userinfo}@{parsed.hostname}{port_part}"
|
||||
|
||||
sanitized = parsed._replace(netloc=new_netloc)
|
||||
return urlunparse(sanitized)
|
||||
|
||||
|
||||
def build_version_data() -> dict[str, Any]:
|
||||
"""Assemble structured data for the ``version`` command."""
|
||||
return {
|
||||
@@ -145,7 +181,7 @@ def build_info_data() -> dict[str, Any]:
|
||||
"version": __version__,
|
||||
"data_dir": str(data_dir),
|
||||
"config_path": str(config_path),
|
||||
"database": db_url,
|
||||
"database": _sanitise_db_url(db_url),
|
||||
"server_mode": server_mode,
|
||||
"platform": f"{platform.system()} {platform.release()} ({platform.machine()})",
|
||||
"automation": settings.default_automation_profile,
|
||||
|
||||
@@ -2,9 +2,10 @@
|
||||
|
||||
Invariants are natural-language constraints on plan execution, scoped at
|
||||
global, project, action, or plan level. When an action is used, its
|
||||
invariants are promoted to plan-level. The runtime precedence chain is:
|
||||
invariants are promoted to plan-level and participate in the merge.
|
||||
The runtime precedence chain is:
|
||||
|
||||
plan > project > global
|
||||
plan > action > project > global
|
||||
|
||||
They are reconciled by the Invariant Reconciliation Actor at the start
|
||||
of Strategize and recorded as ``invariant_enforced`` decisions.
|
||||
@@ -21,8 +22,8 @@ of Strategize and recorded as ``invariant_enforced`` decisions.
|
||||
## Merge Precedence
|
||||
|
||||
When computing the effective set of invariants for a plan, the merge
|
||||
order is **plan > project > global**. Duplicate texts (case-insensitive)
|
||||
are de-duplicated, keeping the highest-precedence copy.
|
||||
order is **plan > action > project > global**. Duplicate texts
|
||||
(case-insensitive) are de-duplicated, keeping the highest-precedence copy.
|
||||
|
||||
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
|
||||
"""
|
||||
@@ -39,8 +40,10 @@ from ulid import ULID
|
||||
class InvariantScope(StrEnum):
|
||||
"""Scope at which an invariant applies.
|
||||
|
||||
Precedence (highest to lowest): PLAN > PROJECT > GLOBAL.
|
||||
ACTION invariants are promoted to PLAN scope at ``plan use`` time.
|
||||
Precedence (highest to lowest): PLAN > ACTION > PROJECT > GLOBAL.
|
||||
ACTION invariants are promoted to plan-level and participate in the
|
||||
merge during reconciliation, sitting between PLAN and PROJECT in the
|
||||
precedence chain.
|
||||
"""
|
||||
|
||||
GLOBAL = "global"
|
||||
@@ -137,10 +140,11 @@ class InvariantSet(BaseModel):
|
||||
def merge(
|
||||
cls,
|
||||
plan_invariants: list[Invariant],
|
||||
project_invariants: list[Invariant],
|
||||
global_invariants: list[Invariant],
|
||||
action_invariants: list[Invariant] | None = None,
|
||||
project_invariants: list[Invariant] | None = None,
|
||||
global_invariants: list[Invariant] | None = None,
|
||||
) -> InvariantSet:
|
||||
"""Merge invariants respecting plan > project > global precedence.
|
||||
"""Merge invariants respecting plan > action > project > global precedence.
|
||||
|
||||
De-duplicates by text (case-insensitive), keeping the copy from
|
||||
the highest-precedence tier. Within each tier, source ordering
|
||||
@@ -148,6 +152,9 @@ class InvariantSet(BaseModel):
|
||||
|
||||
Args:
|
||||
plan_invariants: Plan-level invariants (highest precedence).
|
||||
action_invariants: Action-scoped invariants (second-highest
|
||||
precedence; promoted to plan-level when an
|
||||
action is used).
|
||||
project_invariants: Project-level invariants.
|
||||
global_invariants: Global-level invariants (lowest precedence).
|
||||
|
||||
@@ -156,7 +163,12 @@ class InvariantSet(BaseModel):
|
||||
"""
|
||||
return cls(
|
||||
invariants=tuple(
|
||||
merge_invariants(plan_invariants, project_invariants, global_invariants)
|
||||
merge_invariants(
|
||||
plan_invariants,
|
||||
action_invariants or [],
|
||||
project_invariants or [],
|
||||
global_invariants or [],
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -165,10 +177,11 @@ class InvariantSet(BaseModel):
|
||||
|
||||
def merge_invariants(
|
||||
plan_invariants: list[Invariant],
|
||||
project_invariants: list[Invariant],
|
||||
global_invariants: list[Invariant],
|
||||
action_invariants: list[Invariant] | None = None,
|
||||
project_invariants: list[Invariant] | None = None,
|
||||
global_invariants: list[Invariant] | None = None,
|
||||
) -> list[Invariant]:
|
||||
"""Merge invariants implementing plan > project > global precedence.
|
||||
"""Merge invariants implementing plan > action > project > global precedence.
|
||||
|
||||
De-duplicates by text (case-insensitive). The first occurrence
|
||||
(from the highest-precedence tier) wins. Within each tier, the
|
||||
@@ -176,8 +189,13 @@ def merge_invariants(
|
||||
|
||||
Args:
|
||||
plan_invariants: Plan-level invariants (highest precedence).
|
||||
project_invariants: Project-level invariants.
|
||||
action_invariants: Action-scoped invariants (second-highest
|
||||
precedence; promoted to plan level when an action is used).
|
||||
Defaults to empty list when ``None``.
|
||||
project_invariants: Project-level invariants. Defaults to empty
|
||||
list when ``None``.
|
||||
global_invariants: Global-level invariants (lowest precedence).
|
||||
Defaults to empty list when ``None``.
|
||||
|
||||
Returns:
|
||||
A de-duplicated list of invariants in precedence order.
|
||||
@@ -185,7 +203,12 @@ def merge_invariants(
|
||||
seen: set[str] = set()
|
||||
result: list[Invariant] = []
|
||||
|
||||
for inv_list in (plan_invariants, project_invariants, global_invariants):
|
||||
for inv_list in (
|
||||
plan_invariants,
|
||||
action_invariants or [],
|
||||
project_invariants or [],
|
||||
global_invariants or [],
|
||||
):
|
||||
for inv in inv_list:
|
||||
if not inv.active:
|
||||
continue
|
||||
|
||||
@@ -740,7 +740,7 @@ class MCPToolAdapter:
|
||||
),
|
||||
]
|
||||
|
||||
properties: dict[str, Any] = input_schema.get("properties", {})
|
||||
properties: dict[str, Any] = input_schema.get("properties") or {}
|
||||
slots: list[ResourceSlot] = []
|
||||
seen_names: set[str] = set()
|
||||
|
||||
|
||||
@@ -286,9 +286,9 @@ class ReactiveConfigParser:
|
||||
are propagated via the agent config so the runtime can attach them.
|
||||
|
||||
All v3 fields — ``context_view``, ``memory``, ``context``,
|
||||
``env_vars``, ``response_format``, ``lsp_capabilities``, and
|
||||
``lsp_context_enrichment`` — are propagated into the agent config
|
||||
so the runtime can consume them.
|
||||
``env_vars``, ``response_format``, ``lsp_capabilities``,
|
||||
``lsp_context_enrichment``, and ``options`` — are propagated
|
||||
into the agent config so the runtime can consume them.
|
||||
"""
|
||||
# m4: guard against None-to-string coercion for all three fields.
|
||||
actor_type = str(data.get("type") or "llm").lower()
|
||||
@@ -375,6 +375,11 @@ class ReactiveConfigParser:
|
||||
if isinstance(response_format, dict):
|
||||
agent_config["response_format"] = response_format
|
||||
|
||||
# M5: propagate actor options block for custom backend support (#11223).
|
||||
options_raw = data.get("options")
|
||||
if isinstance(options_raw, dict):
|
||||
agent_config["options"] = dict(options_raw)
|
||||
|
||||
if actor_type in ("llm", "tool"):
|
||||
# Single-agent synthesis.
|
||||
rc.agents[actor_name] = AgentConfig(
|
||||
@@ -424,6 +429,9 @@ class ReactiveConfigParser:
|
||||
else dict(lsp_bindings)
|
||||
)
|
||||
node_config.setdefault("lsp", lsp_copy)
|
||||
# M5: propagate actor-level options to graph nodes (#11223).
|
||||
if isinstance(options_raw, dict) and options_raw:
|
||||
node_config.setdefault("options", dict(options_raw))
|
||||
nodes_map[node_id] = node_config
|
||||
|
||||
# Create a reactive agent for each graph node.
|
||||
|
||||
@@ -226,6 +226,55 @@ class SimpleLLMAgent:
|
||||
llm_kwargs["max_tokens"] = max_tokens
|
||||
if max_retries is not None:
|
||||
llm_kwargs["max_retries"] = max_retries
|
||||
# M5: merge options block so custom LLM backend kwargs
|
||||
# (e.g. openai_api_base, openai_api_key) are forwarded to the
|
||||
# LLM constructor. Options are applied after the fixed keys so
|
||||
# that explicit top-level keys (temperature, max_tokens, etc.)
|
||||
# take precedence over duplicates in options.
|
||||
#
|
||||
# openai_api_key is handled specially: the registry uses a
|
||||
# __api_key_sentinel mechanism to distinguish explicitly
|
||||
# provided keys from environment-sourced ones. If the user
|
||||
# supplies openai_api_key in options, we extract it and inject
|
||||
# it via the sentinel so it overrides the registry's default.
|
||||
options = dict(self.config.get("options") or {})
|
||||
if "openai_api_key" in options:
|
||||
llm_kwargs["__api_key_sentinel"] = options.pop("openai_api_key")
|
||||
# Ensure sensitive/reserved ChatOpenAI constructor params cannot
|
||||
# be injected via a crafted actor YAML.
|
||||
_ALLOWED_OPTIONS: frozenset[str] = frozenset(
|
||||
{
|
||||
"openai_api_base",
|
||||
"temperature",
|
||||
"max_tokens",
|
||||
"timeout",
|
||||
"top_p",
|
||||
"frequency_penalty",
|
||||
"presence_penalty",
|
||||
}
|
||||
)
|
||||
_RESERVED: frozenset[str] = frozenset({"provider_type", "model_id"})
|
||||
for key, value in options.items():
|
||||
if key in _RESERVED:
|
||||
logger_sr.warning(
|
||||
"Actor '%s' options block contains reserved key '%s' "
|
||||
"that conflicts with positional arguments; ignoring.",
|
||||
self.name,
|
||||
key,
|
||||
)
|
||||
continue
|
||||
if key not in llm_kwargs:
|
||||
if key in _ALLOWED_OPTIONS:
|
||||
llm_kwargs[key] = value
|
||||
else:
|
||||
logger_sr.warning(
|
||||
"Actor '%s' options block contains unrecognized "
|
||||
"key '%s'; ignoring for security. Allowed keys: "
|
||||
"%s.",
|
||||
self.name,
|
||||
key,
|
||||
sorted(_ALLOWED_OPTIONS),
|
||||
)
|
||||
registry = get_provider_registry()
|
||||
self._llm = registry.create_llm(
|
||||
provider_type=provider, model_id=model, **llm_kwargs
|
||||
|
||||
@@ -179,13 +179,19 @@ class BaseResourceHandler:
|
||||
Raises:
|
||||
PermissionError: If *path* escapes the root via ``..`` or
|
||||
symlink resolution.
|
||||
|
||||
Uses :meth:`Path.relative_to` (not string prefix matching) to
|
||||
avoid the *prefix-collision bypass* exemplified by
|
||||
``startswith(root + os.sep)`` on symlinks and resolved paths.
|
||||
"""
|
||||
root = Path(location).resolve()
|
||||
target = (root / path).resolve()
|
||||
# Use root + os.sep to prevent prefix collision bypass:
|
||||
# e.g. root=/tmp/foo must not match target=/tmp/foobar/secret
|
||||
if target != root and not str(target).startswith(str(root) + os.sep):
|
||||
raise PermissionError(f"Path '{path}' escapes resource root '{location}'")
|
||||
try:
|
||||
target.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise PermissionError(
|
||||
f"Path '{path}' escapes resource root '{location}'"
|
||||
) from exc
|
||||
return target
|
||||
|
||||
def read(self, *, resource: Resource, path: str = "") -> Content:
|
||||
|
||||
@@ -42,6 +42,7 @@ _SENSITIVE_SUBSTRINGS: set[str] = {
|
||||
# though they contain a sensitive substring (e.g. "token_count").
|
||||
_FALSE_POSITIVE_KEYS: set[str] = {
|
||||
"token_count",
|
||||
"token_prefix",
|
||||
"token_limit",
|
||||
"token_usage",
|
||||
"input_tokens",
|
||||
|
||||
@@ -69,6 +69,10 @@ def validate_sandbox_path(path_str: str, sandbox_root: str | None = None) -> Pat
|
||||
|
||||
Raises ``ValueError`` with the rejected path when traversal is
|
||||
detected or the resolved path falls outside the sandbox root.
|
||||
|
||||
Uses :meth:`Path.relative_to` (not string prefix matching) to avoid
|
||||
the *prefix-collision bypass*: a target like ``/tmp/abc123-escape``
|
||||
would incorrectly pass ``startswith("/tmp/abc123")``.
|
||||
"""
|
||||
if not path_str:
|
||||
raise ValueError("Path must not be empty")
|
||||
@@ -77,8 +81,12 @@ def validate_sandbox_path(path_str: str, sandbox_root: str | None = None) -> Pat
|
||||
root = root.resolve()
|
||||
|
||||
target = (root / path_str).resolve()
|
||||
if not str(target).startswith(str(root)):
|
||||
raise ValueError(f"Path traversal detected: '{path_str}' escapes sandbox root")
|
||||
try:
|
||||
target.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise ValueError(
|
||||
f"Path traversal detected: '{path_str}' escapes sandbox root"
|
||||
) from exc
|
||||
return target
|
||||
|
||||
|
||||
|
||||
@@ -161,10 +161,22 @@ def _normalise(path: str) -> str:
|
||||
|
||||
|
||||
def _is_under(path: str, root: str) -> bool:
|
||||
"""Return ``True`` if *path* is equal to or a child of *root*."""
|
||||
"""Return ``True`` if *path* is equal to or a child of *root*.
|
||||
|
||||
Uses semantic path containment via posixpath.relpath instead of
|
||||
string prefix matching (str.startswith). String prefix matching
|
||||
is vulnerable to sibling-directory prefix-collision attacks where
|
||||
/tmp/sandbox would incorrectly match /tmp/sandboxmalicious/file.
|
||||
|
||||
See issue #7478 — startswith bypass in path containment checks.
|
||||
"""
|
||||
if path == root:
|
||||
return True
|
||||
return path.startswith(root + "/")
|
||||
try:
|
||||
relative = posixpath.relpath(path, root)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
return not relative.startswith(".." + posixpath.sep) and relative != ".."
|
||||
|
||||
|
||||
def _relative_to(path: str, root: str) -> str:
|
||||
|
||||
@@ -145,7 +145,7 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
self._confirmed = False
|
||||
self._selected_actor = None
|
||||
self._visible = True
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
def hide(self) -> None:
|
||||
"""Hide the overlay and clear its content."""
|
||||
@@ -161,14 +161,14 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
if not self._filtered_actors:
|
||||
return
|
||||
self._selected_index = (self._selected_index - 1) % len(self._filtered_actors)
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
def move_down(self) -> None:
|
||||
"""Move the selection cursor down by one position (wraps)."""
|
||||
if not self._filtered_actors:
|
||||
return
|
||||
self._selected_index = (self._selected_index + 1) % len(self._filtered_actors)
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Search / filter
|
||||
@@ -191,7 +191,7 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
else:
|
||||
self._filtered_actors = list(self._actors)
|
||||
self._selected_index = 0
|
||||
self._render()
|
||||
self._refresh_display()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Confirmation
|
||||
@@ -215,10 +215,10 @@ class ActorSelectionOverlay(_StaticBase):
|
||||
return actor
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal rendering
|
||||
# Internal display refresh
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _render(self) -> None:
|
||||
def _refresh_display(self) -> None:
|
||||
content = render_actor_selection(
|
||||
self._filtered_actors,
|
||||
self._selected_index,
|
||||
|
||||
Reference in New Issue
Block a user