Compare commits
36 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 291f6c286f | |||
| a70a6cf835 | |||
| 1b2b225dbd | |||
| 6cadceba10 | |||
| a0d7a7f307 | |||
| 018ec4df57 | |||
| b98c99d93e | |||
| e4186508b8 | |||
| 153502feca | |||
| 2537c5704f | |||
| 16baa60cb2 | |||
| 19c81ac388 | |||
| 6bda792d2b | |||
| 412c338f4e | |||
| 56338db205 | |||
| 38aa457841 | |||
| 91691750ed | |||
| d0136e941a | |||
| 0bde46f0ff | |||
| ca331ea187 | |||
| f1a4858a29 | |||
| 190606d7e6 | |||
|
cdbe504b2c
|
|||
| 73d3bed2da | |||
|
3e13411fcf
|
|||
| ad60b1da59 | |||
| 652769c46c | |||
|
3a95701d9a
|
|||
|
8548644819
|
|||
| eb46f0ff54 | |||
| 0c5724c2f6 | |||
| 20ad9a46c4 | |||
| e2167ab8e7 | |||
| b3851693c8 | |||
| 23d73e7fb2 | |||
| 86f96f299e |
@@ -183,3 +183,6 @@ agents-test
|
||||
|
||||
# Generated test reports (CI artifacts) — build artifacts, not to be committed
|
||||
test_reports/
|
||||
|
||||
# Auto-added by CleverAgents plan executor
|
||||
plan-output/
|
||||
|
||||
@@ -7,6 +7,33 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- **SubplanExecutionService lazy wiring in CLI** (#10268): Wired `subplan_service`
|
||||
from the DI container into `_get_plan_executor()` so `PlanExecutor` can spawn child
|
||||
plans during the Execute phase. When `SubplanService` is available but
|
||||
`SubplanExecutionService` is not explicitly injected, `_execute_subplans()` now
|
||||
lazily creates a `SubplanExecutionService` using the parent plan's `subplan_config`
|
||||
and the `_execute_child_plan` callback. Added recursion guard and strategize result
|
||||
validation to `_execute_child_plan` to prevent re-entrant or orphaned child plan
|
||||
execution.
|
||||
|
||||
- **Actor namespace/name disambiguation** (#11254): When action YAML references
|
||||
actors using `namespace/name` format (e.g. `strategy_actor: local/my-strategist`),
|
||||
the `_parse_actor_name()` functions no longer mistake the namespace prefix for an
|
||||
LLM provider name. A new `_is_known_provider()` utility checks whether the first
|
||||
slash-separated segment matches a known `ProviderType` (e.g. `openai`, `anthropic`);
|
||||
if not, the input is treated as namespace/name. `PlanLifecycleService` now provides
|
||||
`resolve_actor_provider_model()` to resolve namespaced references to their underlying
|
||||
`provider/model` via the actor registry. All affected call sites — `StrategyActor`,
|
||||
`LLMStrategizeActor`, `LLMExecuteActor`, and `SessionWorkflow._resolve_llm()` —
|
||||
pre-resolve actor names before passing them to the LLM provider, fixing the
|
||||
`ValueError: Unknown provider type` crash when using namespaced actor references.
|
||||
|
||||
Data integrity fix: ValidationAttachmentRepository argument swap (#7492): Fixed
|
||||
a critical data integrity issue in `ValidationAttachmentRepository.attach` where
|
||||
`validation_name` and `resource_id` arguments were being silently swapped based on a
|
||||
fragile heuristic (`"/" in resource_id`). Arguments are now passed in the correct order,
|
||||
ensuring data is stored with proper parameter values.
|
||||
|
||||
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
|
||||
requires whole-word closing keywords (avoids false positives like
|
||||
"prefixes #12"), TDD bug tag discovery now uses exact token matching
|
||||
@@ -122,6 +149,16 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
- **Plan Rollback Command** (#8557): Implemented `agents plan rollback <plan-id> [<checkpoint-id>]` for checkpoint-based plan state restoration in Epic #8493. The command restores a plan's sandbox to the state captured at a given checkpoint, discarding all decisions made after that checkpoint. The checkpoint can be specified as an optional positional second argument or via the `--to-checkpoint` named option. Supports `--yes/-y` flag to skip confirmation prompts and `--format/-f` for output format selection (rich/plain/json/yaml). Included with comprehensive BDD test coverage (>= 97%) and spec-aligned output formatting showing rollback summary, changes reverted, impact analysis, and post-rollback state panels.
|
||||
|
||||
### Fixed
|
||||
- **ACMS execute-phase assembler respects project-level hot_max_tokens** (#11035): Fixed
|
||||
``_resolve_hot_max_tokens()`` to read ``hot_max_tokens`` from
|
||||
``context_policy_json["acms_config"]["hot_max_tokens"]`` — the correct sub-key written
|
||||
by ``agents project context set --hot-max-tokens``. The previous implementation read
|
||||
from the top-level key (``config_dict.get("hot_max_tokens")``), which was always
|
||||
``None``, causing the assembler to silently fall back to the global 16K default even
|
||||
when a project-level override was configured. Also adds two Behave regression scenarios
|
||||
with ``@tdd_issue @tdd_issue_11035`` tags that exercise the DB query code path and
|
||||
verify the project-level budget is applied to ``CoreContextBudget`` and
|
||||
``ContextRequest``.
|
||||
- **Guard cleanup_stale against execute/processing and execute/complete plans** (#11121):
|
||||
``_create_sandbox_for_plan()`` in ``src/cleveragents/cli/commands/plan.py`` now
|
||||
skips ``GitWorktreeSandbox.cleanup_stale()`` when the plan is in
|
||||
|
||||
@@ -89,7 +89,7 @@ def _make_service(plan: MagicMock, changeset: SpecChangeSet) -> PlanApplyService
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle.complete_apply.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
store = MagicMock()
|
||||
store.get.return_value = changeset
|
||||
return PlanApplyService(lifecycle_service=lifecycle, changeset_store=store)
|
||||
|
||||
@@ -62,7 +62,7 @@ def _make_plan() -> MagicMock:
|
||||
def _make_service(plan: MagicMock) -> ErrorRecoveryService:
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
return ErrorRecoveryService(
|
||||
lifecycle_service=lifecycle,
|
||||
auto_retry_threshold=0.0,
|
||||
|
||||
@@ -98,7 +98,7 @@ def _create_plan_and_service() -> tuple[str, PlanApplyService, InMemoryChangeSet
|
||||
plan = lifecycle.get_plan(plan_id)
|
||||
plan.changeset_id = cs.changeset_id
|
||||
plan.sandbox_refs = ["sandbox-bench-001"]
|
||||
lifecycle._commit_plan(plan)
|
||||
lifecycle.commit_plan(plan)
|
||||
|
||||
return plan_id, apply_svc, store
|
||||
|
||||
|
||||
@@ -66,7 +66,7 @@ def _make_errored_plan(lifecycle: PlanLifecycleService, action_name: str) -> str
|
||||
lifecycle.start_strategize(pid)
|
||||
p = lifecycle.get_plan(pid)
|
||||
p.decision_root_id = str(ULID())
|
||||
lifecycle._commit_plan(p)
|
||||
lifecycle.commit_plan(p)
|
||||
lifecycle.complete_strategize(pid)
|
||||
lifecycle.execute_plan(pid)
|
||||
lifecycle.start_execute(pid)
|
||||
@@ -104,7 +104,7 @@ class TimeCheckpointRecording:
|
||||
self.lifecycle.start_strategize(pid)
|
||||
p = self.lifecycle.get_plan(pid)
|
||||
p.decision_root_id = str(ULID())
|
||||
self.lifecycle._commit_plan(p)
|
||||
self.lifecycle.commit_plan(p)
|
||||
self.lifecycle.complete_strategize(pid)
|
||||
self.lifecycle.execute_plan(pid)
|
||||
self.lifecycle.start_execute(pid)
|
||||
|
||||
@@ -25,6 +25,15 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Added
|
||||
|
||||
- **SubplanExecutionService lazy wiring in CLI** (#10268): Wired `subplan_service`
|
||||
from the DI container into `_get_plan_executor()` so `PlanExecutor` can spawn child
|
||||
plans during the Execute phase. When `SubplanService` is available but
|
||||
`SubplanExecutionService` is not explicitly injected, `_execute_subplans()` now
|
||||
lazily creates a `SubplanExecutionService` using the parent plan's `subplan_config`
|
||||
and the `_execute_child_plan` callback. Added recursion guard and strategize result
|
||||
validation to `_execute_child_plan` to prevent re-entrant or orphaned child plan
|
||||
execution.
|
||||
|
||||
- **TDD: MCPToolAdapter.infer_resource_slots() TypeError with null properties** (#10470):
|
||||
Added a TDD issue-capture Behave scenario that reproduces the bug where
|
||||
`MCPToolAdapter.infer_resource_slots()` raises `TypeError` when the input schema
|
||||
@@ -50,6 +59,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Actor namespace/name disambiguation** (#11254): When action YAML references
|
||||
actors using `namespace/name` format (e.g. `strategy_actor: local/my-strategist`),
|
||||
the `_parse_actor_name()` functions no longer mistake the namespace prefix for an
|
||||
LLM provider name. A new `_is_known_provider()` utility checks whether the first
|
||||
slash-separated segment matches a known `ProviderType` (e.g. `openai`, `anthropic`);
|
||||
if not, the input is treated as namespace/name. `PlanLifecycleService` now provides
|
||||
`resolve_actor_provider_model()` to resolve namespaced references to their underlying
|
||||
`provider/model` via the actor registry. All affected call sites — `StrategyActor`,
|
||||
`LLMStrategizeActor`, `LLMExecuteActor`, and `SessionWorkflow._resolve_llm()` —
|
||||
pre-resolve actor names before passing them to the LLM provider, fixing the
|
||||
`ValueError: Unknown provider type` crash when using namespaced actor references.
|
||||
|
||||
- **Built-in actors v3 YAML format** (#10883): Fixed `agents actor run` failing for
|
||||
built-in actors (e.g., `openai/gpt-4`, `anthropic/claude-3-opus`) due to missing
|
||||
v3 `type` field in stored configuration. `ActorRegistry.ensure_built_in_actors()`
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
@coverage @coverage_actor_run_tool_calling
|
||||
Feature: Actor run tool-calling via ToolCallingRuntime
|
||||
When a skill is attached to `agents actor run` via `--skill`, the actor
|
||||
should perform real LLM tool calls through ToolCallingRuntime.
|
||||
|
||||
Background:
|
||||
Given a minimal ToolCallingAgent fixture
|
||||
|
||||
# ---------- A: single tool call succeeds ----------
|
||||
|
||||
Scenario: Tool call succeeds — mock LLM returns read_file call
|
||||
Given a mock LLM caller that makes one read_file tool call
|
||||
When ToolCallingAgent.process is called with prompt "Review src/auth.py"
|
||||
Then the tool_calls count is 1
|
||||
And the final response is non-empty
|
||||
|
||||
# ---------- B: multi-turn tool loop ----------
|
||||
|
||||
Scenario: Multi-turn tool loop — mock LLM makes two consecutive tool calls
|
||||
Given a mock LLM caller that makes 2 consecutive tool calls before finishing
|
||||
When ToolCallingAgent.process is called with prompt "Analyse two files"
|
||||
Then the tool_calls count is 2
|
||||
And the loop ran for at least 3 iterations
|
||||
|
||||
# ---------- C: no-skill plain LLM regression ----------
|
||||
|
||||
Scenario: No-skill plain LLM — SimpleLLMAgent used, no tool schemas sent
|
||||
Given a SimpleLLMAgent with a mock LLM
|
||||
When SimpleLLMAgent.process is called with a simple prompt
|
||||
Then SimpleLLMAgent returns the mocked response
|
||||
And no tool schemas were sent to the LLM
|
||||
|
||||
# ---------- D: skill tools not silently dropped ----------
|
||||
|
||||
Scenario: Skill tools not silently dropped when actor has no base tools list
|
||||
Given an actor config with no base tools list
|
||||
And resolved skill tool entries for the actor
|
||||
When _make_agent_instance is called for that actor
|
||||
Then the result is a ToolCallingAgent instance
|
||||
|
||||
# ---------- E: ToolCallingLLMCaller first-call system prompt ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller sends system prompt on first call
|
||||
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
|
||||
When invoke is called for the first time with a user prompt
|
||||
Then the accumulated messages include a SystemMessage
|
||||
And the accumulated messages include a HumanMessage
|
||||
|
||||
# ---------- F: ToolCallingLLMCaller subsequent call appends tool results ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller appends tool results on subsequent call
|
||||
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
|
||||
And a prior first invoke has been made
|
||||
When invoke is called again with tool_results
|
||||
Then a ToolMessage is appended to the accumulated messages
|
||||
|
||||
# ---------- G: GraphExecutor passes context to ToolCallingAgent ----------
|
||||
|
||||
Scenario: GraphExecutor._invoke_agent passes context dict to ToolCallingAgent
|
||||
Given a ToolCallingAgent spy that captures its process() arguments
|
||||
When GraphExecutor._invoke_agent is called with that agent and a context dict
|
||||
Then the context dict was forwarded to ToolCallingAgent.process
|
||||
|
||||
# ---------- H: last_run_tool_calls property reflects run results ----------
|
||||
|
||||
Scenario: ReactiveCleverAgentsApp.last_run_tool_calls reflects tool calls
|
||||
Given a ReactiveCleverAgentsApp with a registered ToolCallingAgent that has last_result
|
||||
When _tally_tool_calls is called on the app
|
||||
Then last_run_tool_calls equals the number of tool call history entries
|
||||
|
||||
# ---------- I: ToolCallingLLMCaller _render_prompt ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller._render_prompt handles empty template
|
||||
Given a fresh ToolCallingLLMCaller
|
||||
When _render_prompt is called with an empty string template
|
||||
Then _render_prompt returns an empty string
|
||||
|
||||
Scenario: ToolCallingLLMCaller._render_prompt renders Jinja2 template
|
||||
Given a fresh ToolCallingLLMCaller
|
||||
When _render_prompt is called with a Jinja2 template and context
|
||||
Then _render_prompt returns the rendered string
|
||||
|
||||
Scenario: ToolCallingLLMCaller._render_prompt handles rendering exception
|
||||
Given a fresh ToolCallingLLMCaller
|
||||
When _render_prompt is called with a template that raises during rendering
|
||||
Then _render_prompt returns the original template
|
||||
|
||||
# ---------- J: ToolCallingLLMCaller _resolve_llm with kwargs ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller._resolve_llm passes temperature max_tokens max_retries to provider
|
||||
Given a ToolCallingLLMCaller with temperature max_tokens max_retries in actor_config
|
||||
When _resolve_llm is called with tool_schemas
|
||||
Then the provider registry create_llm was called with the config kwargs
|
||||
|
||||
Scenario: ToolCallingLLMCaller._resolve_llm skips bind_tools when no schemas
|
||||
Given a ToolCallingLLMCaller with temperature max_tokens max_retries in actor_config
|
||||
When _resolve_llm is called without tool_schemas
|
||||
Then bind_tools was not called on the LLM
|
||||
|
||||
Scenario: ToolCallingLLMCaller._resolve_llm falls back when bind_tools raises
|
||||
Given a ToolCallingLLMCaller with a provider that raises on bind_tools
|
||||
When _resolve_llm is called with tool_schemas for a failing bind
|
||||
Then the plain LLM is used without tool binding
|
||||
|
||||
# ---------- K: ToolCallingLLMCaller invoke no system prompt ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller.invoke without system prompt only adds HumanMessage
|
||||
Given a ToolCallingLLMCaller with an actor config without system_prompt
|
||||
When invoke is called for the first time with a user prompt
|
||||
Then the accumulated messages include a HumanMessage
|
||||
And no SystemMessage was added
|
||||
|
||||
# ---------- L: ToolCallingLLMCaller invoke list content ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller.invoke handles list content in response
|
||||
Given a ToolCallingLLMCaller with an actor config without system_prompt
|
||||
When invoke is called and the LLM returns list content
|
||||
Then the response content joins the list parts
|
||||
|
||||
# ---------- M: ToolCallingLLMCaller invoke failed tool result ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller.invoke uses error field on failed tool result
|
||||
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
|
||||
And a prior first invoke has been made
|
||||
When invoke is called with a failed tool_result
|
||||
Then a ToolMessage with the error text is appended
|
||||
|
||||
# ---------- N: ToolCallingAgent._build_tool_registry edge cases ----------
|
||||
|
||||
Scenario: ToolCallingAgent._build_tool_registry skips empty tool name
|
||||
Given a ToolCallingAgent with an entry that has an empty name
|
||||
When _build_tool_registry is called
|
||||
Then the local registry is empty
|
||||
|
||||
Scenario: ToolCallingAgent._build_tool_registry skips duplicate names
|
||||
Given a ToolCallingAgent with two entries for the same builtin tool
|
||||
When _build_tool_registry is called
|
||||
Then the local registry has exactly one tool registered
|
||||
|
||||
Scenario: ToolCallingAgent._build_tool_registry warns when tool not in builtin registry
|
||||
Given a ToolCallingAgent with an entry not found in builtin registry
|
||||
When _build_tool_registry is called
|
||||
Then the local registry is empty
|
||||
|
||||
# ---------- O: ToolCallingAgent.process_message_sync ----------
|
||||
|
||||
Scenario: ToolCallingAgent.process_message_sync delegates to process
|
||||
Given a ToolCallingAgent with no tools and a mock process method
|
||||
When process_message_sync is called
|
||||
Then the result is the same as calling process directly
|
||||
|
||||
# ---------- P: ToolCallingLLMCaller.invoke returns tool calls from response ----------
|
||||
|
||||
Scenario: ToolCallingLLMCaller.invoke extracts tool calls from LLM response
|
||||
Given a ToolCallingLLMCaller with an actor config without system_prompt
|
||||
When invoke is called and the LLM returns a response with tool call dicts
|
||||
Then the LLMResponse contains the extracted tool calls
|
||||
|
||||
# ---------- Q: _resolve_provider_format maps provider to correct format ----------
|
||||
|
||||
Scenario: _resolve_provider_format returns ANTHROPIC for anthropic provider
|
||||
Given an actor config with provider "anthropic"
|
||||
When _resolve_provider_format is called with that actor config
|
||||
Then _resolve_provider_format returns ProviderFormat.ANTHROPIC
|
||||
|
||||
Scenario: _resolve_provider_format returns OPENAI for openai provider
|
||||
Given an actor config with provider "openai"
|
||||
When _resolve_provider_format is called with that actor config
|
||||
Then _resolve_provider_format returns ProviderFormat.OPENAI
|
||||
|
||||
Scenario: _resolve_provider_format returns OPENAI for unknown provider
|
||||
Given an actor config with provider "google"
|
||||
When _resolve_provider_format is called with that actor config
|
||||
Then _resolve_provider_format returns ProviderFormat.OPENAI
|
||||
|
||||
Scenario: _resolve_provider_format returns OPENAI for empty actor config
|
||||
Given an empty actor config
|
||||
When _resolve_provider_format is called with that actor config
|
||||
Then _resolve_provider_format returns ProviderFormat.OPENAI
|
||||
|
||||
Scenario: _resolve_provider_format returns OPENAI for None actor config
|
||||
When _resolve_provider_format is called with actor_config None
|
||||
Then _resolve_provider_format returns ProviderFormat.OPENAI
|
||||
|
||||
# ---------- R: ToolCallingAgent.process passes provider_format to runtime ----------
|
||||
|
||||
Scenario: ToolCallingAgent.process passes provider_format to ToolCallingRuntime
|
||||
Given a ToolCallingAgent with an anthropic actor config and a file-read tool entry
|
||||
When ToolCallingAgent.process is called with prompt "Read my file"
|
||||
Then ToolCallingRuntime was constructed with ProviderFormat.ANTHROPIC
|
||||
|
||||
# ---------- S: Tool name encoding/decoding ----------
|
||||
|
||||
Scenario: _encode_tool_name replaces colon and slash with uppercase sentinels
|
||||
When _encode_tool_name is called with "server:namespace/tool"
|
||||
Then _encode_tool_name returns "server_C_namespace_S_tool"
|
||||
|
||||
Scenario: _encode_tool_name is idempotent on already-clean names
|
||||
When _encode_tool_name is called with "my-clean_tool"
|
||||
Then _encode_tool_name returns "my-clean_tool"
|
||||
|
||||
Scenario: _decode_tool_name restores colon and slash from sentinels
|
||||
When _decode_tool_name is called with "server_C_namespace_S_tool"
|
||||
Then _decode_tool_name returns "server:namespace/tool"
|
||||
|
||||
Scenario: _decode_tool_name is idempotent on clean names
|
||||
When _decode_tool_name is called with "my-clean_tool"
|
||||
Then _decode_tool_name returns "my-clean_tool"
|
||||
|
||||
Scenario: encode-decode round-trip preserves original name
|
||||
Given a tool name "local/my__tool"
|
||||
When the tool name is encoded then decoded
|
||||
Then the round-trip name equals "local/my__tool"
|
||||
|
||||
Scenario: encode-decode round-trip for server-qualified name
|
||||
Given a tool name "server.example:builtin/git-status"
|
||||
When the tool name is encoded then decoded
|
||||
Then the round-trip name equals "server.example:builtin/git-status"
|
||||
|
||||
# ---------- T: _resolve_llm encodes tool names in schemas ----------
|
||||
|
||||
Scenario: _resolve_llm encodes tool names before calling bind_tools
|
||||
Given a ToolCallingLLMCaller with default actor config
|
||||
When _resolve_llm is called with a schema containing a namespaced tool name
|
||||
Then bind_tools was called with the encoded tool name
|
||||
|
||||
# ---------- U: invoke decodes tool names from LLM response ----------
|
||||
|
||||
Scenario: invoke decodes tool names when extracting tool calls from LLM response
|
||||
Given a ToolCallingLLMCaller with an actor config without system_prompt
|
||||
When invoke is called and the LLM returns a response with encoded tool call names
|
||||
Then the LLMResponse contains the decoded tool calls
|
||||
|
||||
# ---------- S: Tool name encoding/decoding ----------
|
||||
|
||||
Scenario: _encode_tool_name replaces colon and slash with uppercase sentinels
|
||||
When _encode_tool_name is called with "server:namespace/tool"
|
||||
Then _encode_tool_name returns "server_C_namespace_S_tool"
|
||||
|
||||
Scenario: _encode_tool_name is idempotent on already-clean names
|
||||
When _encode_tool_name is called with "my-clean_tool"
|
||||
Then _encode_tool_name returns "my-clean_tool"
|
||||
|
||||
Scenario: _decode_tool_name restores colon and slash from sentinels
|
||||
When _decode_tool_name is called with "server_C_namespace_S_tool"
|
||||
Then _decode_tool_name returns "server:namespace/tool"
|
||||
|
||||
Scenario: _decode_tool_name is idempotent on clean names
|
||||
When _decode_tool_name is called with "my-clean_tool"
|
||||
Then _decode_tool_name returns "my-clean_tool"
|
||||
|
||||
Scenario: encode-decode round-trip preserves original name
|
||||
Given a tool name "local/my__tool"
|
||||
When the tool name is encoded then decoded
|
||||
Then the round-trip name equals "local/my__tool"
|
||||
|
||||
Scenario: encode-decode round-trip for server-qualified name
|
||||
Given a tool name "server.example:builtin/git-status"
|
||||
When the tool name is encoded then decoded
|
||||
Then the round-trip name equals "server.example:builtin/git-status"
|
||||
|
||||
# ---------- T: _resolve_llm encodes tool names in schemas ----------
|
||||
|
||||
Scenario: _resolve_llm encodes tool names before calling bind_tools
|
||||
Given a ToolCallingLLMCaller with default actor config
|
||||
When _resolve_llm is called with a schema containing a namespaced tool name
|
||||
Then bind_tools was called with the encoded tool name
|
||||
|
||||
# ---------- U: invoke decodes tool names from LLM response ----------
|
||||
|
||||
Scenario: invoke decodes tool names when extracting tool calls from LLM response
|
||||
Given a ToolCallingLLMCaller with an actor config without system_prompt
|
||||
When invoke is called and the LLM returns a response with encoded tool call names
|
||||
Then the LLMResponse contains the decoded tool calls
|
||||
|
||||
# ---------- V: ToolCallingLLMCaller options block forwarding (#11243) ----------
|
||||
|
||||
@tdd_issue @tdd_issue_11243
|
||||
Scenario: ToolCallingLLMCaller forwards openai_api_base and openai_api_key from options block
|
||||
Given a ToolCallingLLMCaller with actor config options containing openai_api_base and openai_api_key
|
||||
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
|
||||
Then create_llm was called with openai_api_base forwarded from options
|
||||
And create_llm was called with __api_key_sentinel extracted from options openai_api_key
|
||||
|
||||
@tdd_issue @tdd_issue_11243
|
||||
Scenario: ToolCallingLLMCaller forwards allowed options keys to create_llm
|
||||
Given a ToolCallingLLMCaller with actor config options containing allowed extra keys
|
||||
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
|
||||
Then create_llm was called with the allowed options keys forwarded
|
||||
|
||||
@tdd_issue @tdd_issue_11243
|
||||
Scenario: ToolCallingLLMCaller rejects reserved keys in options block with a warning
|
||||
Given a ToolCallingLLMCaller with actor config options containing reserved key provider_type
|
||||
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
|
||||
Then create_llm was NOT called with the reserved key provider_type
|
||||
|
||||
@tdd_issue @tdd_issue_11243
|
||||
Scenario: ToolCallingLLMCaller rejects unknown keys in options block with a warning
|
||||
Given a ToolCallingLLMCaller with actor config options containing unknown key foo_bar
|
||||
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
|
||||
Then create_llm was NOT called with the unknown key foo_bar
|
||||
|
||||
@tdd_issue @tdd_issue_11243
|
||||
Scenario: ToolCallingLLMCaller top-level config takes precedence over options block
|
||||
Given a ToolCallingLLMCaller with actor config with top-level temperature and options temperature
|
||||
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
|
||||
Then create_llm was called with the top-level temperature value
|
||||
@@ -211,3 +211,24 @@ Feature: v3 Actor YAML schema support in CLI and registry
|
||||
Given a v3 graph actor config dict with skills and lsp as dict
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then each graph node agent should have lsp dict propagated
|
||||
|
||||
# M5: options block forwarded to agent_config in _build_from_v3 (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser propagates options block to agent config
|
||||
Given a v3 LLM actor config dict with options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should contain the options block
|
||||
|
||||
# M5: actor without options block is unaffected (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser handles v3 actor without options block
|
||||
Given a v3 LLM actor config dict without options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should not contain an options key
|
||||
|
||||
# M5: empty options dict is preserved (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: ReactiveConfigParser preserves empty options block
|
||||
Given a v3 LLM actor config dict with empty options block
|
||||
When I parse the v3 config through ReactiveConfigParser
|
||||
Then the agent config should contain an empty options block
|
||||
|
||||
@@ -188,4 +188,4 @@ Feature: Automation Profile CLI commands
|
||||
# Legacy flag removal tests
|
||||
Scenario: Legacy --automation-level flag is rejected on plan use
|
||||
When I invoke plan use with --automation-level "manual"
|
||||
Then the plan output should contain "No such option: --automation-level"
|
||||
Then the plan output should contain "--automation-level"
|
||||
|
||||
@@ -1379,3 +1379,24 @@ Feature: Consolidated Routing
|
||||
When the stream router tool agent processes "keep_me" through operation
|
||||
Then the stream router tool agent operation result should be "keep_me"
|
||||
|
||||
# M5: SimpleLLMAgent._resolve_llm forwards options to registry.create_llm (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent forwards options block to LLM constructor
|
||||
Given a stream router llm agent with options block containing custom base url
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should have received the custom base url
|
||||
|
||||
# M5: actor without options block is unaffected (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent without options block calls LLM constructor without extra kwargs
|
||||
Given a stream router llm agent without options block
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should not have received extra kwargs
|
||||
|
||||
# M5: top-level keys take precedence over duplicates in options (#11223).
|
||||
@tdd_issue @tdd_issue_11223
|
||||
Scenario: SimpleLLMAgent top-level key takes precedence over options duplicate
|
||||
Given a stream router llm agent with top-level temperature and options block
|
||||
When the stream router llm agent resolves the llm
|
||||
Then the llm constructor should have received the top-level temperature
|
||||
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
Feature: Database URL sanitisation — credentials must never be exposed
|
||||
|
||||
As a CleverAgents operator
|
||||
I want database URLs in CLI output to have their credentials masked
|
||||
So that running ``agents info`` never leaks passwords or API tokens
|
||||
|
||||
@tdd_bug_8395
|
||||
Scenario Outline: PostgreSQL URL with user and password — all are masked
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| postgresql://user:secret@localhost/mydb | postgresql://***:***@localhost/mydb |
|
||||
| postgresql://admin:p%40ssw0rd@db.example.com:5432/agents | postgresql://***:***@db.example.com:5432/agents |
|
||||
| postgresql://readonly:r0ad_only@pg.cluster.internal:5433/production | postgresql://***:***@pg.cluster.internal:5433/production |
|
||||
|
||||
Scenario Outline: MySQL URL with credentials — all are masked
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| mysql://app:s3cret@mysql.internal:3306/agents | mysql://***:***@mysql.internal:3306/agents |
|
||||
| mysql+pymysql://root:toor@localhost/testdb | mysql+pymysql://***:***@localhost/testdb |
|
||||
|
||||
Scenario Outline: SQLite URLs — remain unchanged (no credentials)
|
||||
Given the database url is "<url>"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "<expected>"
|
||||
|
||||
Examples:
|
||||
| url | expected |
|
||||
| sqlite:///data/cleveragents.db | sqlite:///data/cleveragents.db |
|
||||
| sqlite:////absolute/path/to/db.sqlite | sqlite:////absolute/path/to/db.sqlite |
|
||||
| memory | memory |
|
||||
|
||||
Scenario: Username-only URL — password is still masked
|
||||
Given the database url is "postgres://deploy@db.example.com/prod"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "postgres://***:***@db.example.com/prod"
|
||||
|
||||
Scenario: SQLite URL without credentials remains unchanged
|
||||
Given the database url is "sqlite:///test.db"
|
||||
When I run sanitise_db_url
|
||||
Then the sanitised database url should be "sqlite:///test.db"
|
||||
|
||||
Scenario: build_info_data returns sanitised database URL
|
||||
Given a mock settings object with database url "postgresql://admin:supersecret@host.example.com:5432/mydb"
|
||||
When I call build_info_data
|
||||
Then the database field in info data should be "postgresql://***:***@host.example.com:5432/mydb"
|
||||
@@ -0,0 +1,38 @@
|
||||
Feature: Decomposition respects max_child_depth limit
|
||||
As a plan orchestrator
|
||||
I want the decomposition service to enforce plan.max-child-depth
|
||||
So that child plans cannot nest deeper than the configured limit
|
||||
|
||||
Background:
|
||||
Given a decomposition service
|
||||
|
||||
Scenario: Decomposition stops recursing when max_child_depth limit is reached
|
||||
Given a project with 500 files across 8 directory levels
|
||||
When I decompose with max_child_depth 2 and max_depth 10
|
||||
Then the decomposition result should have max_depth_reached <= 2
|
||||
And the decomposition warning log should contain "max-child-depth limit"
|
||||
|
||||
Scenario: Decomposition with default max_child_depth allows standard depth
|
||||
Given a project with 2000 files across 5 directory levels
|
||||
When I decompose with default max_child_depth
|
||||
Then the decomposition result should have max_depth_reached >= 1
|
||||
And the decomposition result should have max_depth_reached <= 5
|
||||
|
||||
Scenario: max_child_depth guard triggers before max_depth when more restrictive
|
||||
Given a project with 500 files across 8 directory levels
|
||||
When I decompose with max_child_depth 1 and max_depth 10
|
||||
Then the decomposition result should have max_depth_reached <= 1
|
||||
And the decomposition warning log should contain "max-child-depth limit"
|
||||
|
||||
Scenario: max_depth can be more restrictive than max_child_depth
|
||||
Given a project with 500 files across 8 directory levels
|
||||
When I decompose with max_child_depth 10 and max_depth 2
|
||||
Then the decomposition result should have max_depth_reached <= 2
|
||||
|
||||
Scenario: Default config includes max_child_depth with expected value
|
||||
Given default decomposition config
|
||||
Then max_child_depth should be 5
|
||||
|
||||
Scenario: Invalid max_child_depth value is rejected
|
||||
When I create a config with max_child_depth 0
|
||||
Then a decomposition ValueError should be raised
|
||||
@@ -235,3 +235,21 @@ Feature: Execute-phase context assembler coverage
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the assembled result should be an AssembledContext
|
||||
And epcov the assembled result should have fragments and metadata
|
||||
|
||||
# ---- _resolve_hot_max_tokens: reads context_policy_json from DB ----
|
||||
|
||||
@tdd_issue @tdd_issue_11035
|
||||
Scenario: epcov assemble uses project-level hot_max_tokens from context_policy_json
|
||||
Given epcov an assembler with context_policy_json hot_max_tokens 32000
|
||||
And epcov a plan with project links
|
||||
And epcov scoped fragments that pass all filters
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the pipeline received budget max_tokens of 32000
|
||||
|
||||
@tdd_issue @tdd_issue_11035
|
||||
Scenario: epcov assemble falls back to global hot_max_tokens when context_policy_json has no override
|
||||
Given epcov an assembler with no hot_max_tokens in context_policy_json
|
||||
And epcov a plan with project links
|
||||
And epcov scoped fragments that pass all filters
|
||||
When epcov I call assemble on the plan
|
||||
Then epcov the pipeline received budget max_tokens of 4096
|
||||
|
||||
@@ -114,6 +114,7 @@ Feature: Large-project hierarchical decomposition
|
||||
Scenario: Default config has expected values
|
||||
Given default decomposition config
|
||||
Then max_depth should be 4
|
||||
And max_child_depth should be 5
|
||||
And max_files_per_subplan should be 500
|
||||
And max_tokens_per_subplan should be 100000
|
||||
And min_files_per_subplan should be 10
|
||||
|
||||
@@ -26,6 +26,24 @@ Feature: LLM Actors Coverage
|
||||
Then the parsed provider should be "anthropic"
|
||||
And the parsed model should be "claude-3"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse actor name with namespace/name format
|
||||
When I parse actor name "local/my-strategist"
|
||||
Then the parsed provider should be "openai"
|
||||
And the parsed model should be "local/my-strategist"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse actor name with org namespace/name format
|
||||
When I parse actor name "cleverthis/my-executor"
|
||||
Then the parsed provider should be "openai"
|
||||
And the parsed model should be "cleverthis/my-executor"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse actor name with unknown-provider-like segment
|
||||
When I parse actor name "unknown/gpt-4"
|
||||
Then the parsed provider should be "openai"
|
||||
And the parsed model should be "unknown/gpt-4"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# LLMStrategizeActor.__init__ validation
|
||||
# ---------------------------------------------------------------
|
||||
@@ -201,6 +219,17 @@ Feature: LLM Actors Coverage
|
||||
When I parse file blocks from empty LLM output
|
||||
Then I should get 0 changeset entries
|
||||
|
||||
# M7 fix: <CAFS>/<CAFE> short-form and >>>>>>>> non-conflicting formats
|
||||
# had zero test coverage. These scenarios exercise both new patterns.
|
||||
|
||||
Scenario: Parse file blocks using CAFS short delimiter format
|
||||
When I parse file blocks using the CAFS short delimiter format
|
||||
Then I should get 1 changeset entry with path "src/cafs_example.py"
|
||||
|
||||
Scenario: Parse file blocks using the non-conflicting arrow delimiter format
|
||||
When I parse file blocks using the new arrow delimiter format
|
||||
Then I should get 1 changeset entry with path "src/arrow_example.py"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# LLMExecuteActor._write_to_sandbox
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
@tdd @tdd_issue @tdd_issue_10878
|
||||
Feature: Delimiter parsing does not trip on triple-backticks in LLM output (#10878)
|
||||
Bug #10878 caused architecture reviews to be truncated because the old regex
|
||||
delimiter (triple-backtick markdown code fences ```) would stop at the first
|
||||
triple-backtick appearing *inside* the LLM-generated file content, cutting off
|
||||
all subsequent sections.
|
||||
|
||||
This test suite re-proves that the new delimiter scheme using unique sentinel
|
||||
markers is not vulnerable to markdown collision.
|
||||
|
||||
Forgejo: #10878
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# The old backtick-based parser was broken (reproduction of bug)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: Old backtick delimiter parser stops at first triple-backtick in content
|
||||
Given a string containing ````
|
||||
When I run the command "echo 'backtick-test'"
|
||||
And I create file with LLM output containing code fences in body
|
||||
When I parse the LLM output as if it were old backtick-delimited file blocks
|
||||
Then the parser should return only 2 entries truncated at first triple-backtick
|
||||
|
||||
Scenario: New CLEVERAGENTS_FILE_START delimiter parser handles code fences inside content
|
||||
Given a string containing ````
|
||||
When I run the command "echo 'backtick-test'"
|
||||
And I create file with LLM output using unique delimiters and code fences embedded in body
|
||||
When I parse the LLM output with new CLEVERAGENTS_FILE_START/CLEVERAGENTS_FILE_END delimiters
|
||||
Then the parser should return 6 entries not truncated at any triple-backtick
|
||||
|
||||
|
||||
Scenario: Old backtick-delimiter parser fails when markdown report contains triple backticks
|
||||
Given a string containing ````
|
||||
When I run the command "echo 'old-parser-test'"
|
||||
And I create LLM output where file content contains embedded markdown code blocks
|
||||
When I parse it with the old backtick-style delimiter pattern
|
||||
Then the parser should return only 1 entry instead of the expected 4
|
||||
|
||||
Scenario: New delimited parser handles markdown code blocks inside file content
|
||||
Given a string containing ````
|
||||
When I run the command "echo 'new-parser-test'"
|
||||
And I create LLM output using unique sentinels with embedded ```python code blocks in body
|
||||
When I parse it with CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters
|
||||
Then the parser should return a result count of 4 entries
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Verify new delimiter handles the real-world bug pattern: report + 3+ file blocks each
|
||||
# containing inline code examples that use triple-backticks
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
Scenario: Delimiter survives LLM output with architecture-report header and nested code
|
||||
Given a string containing ````
|
||||
When I run the command "echo 'full-regression-test'"
|
||||
And I create full regression file block output that includes report section plus multiple files each with inline ```python examples
|
||||
When I parse it with new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters
|
||||
Then the parser should return the full entry count of all embedded blocks
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Unit tests on LLMExecuteActor._parse_file_blocks directly (unit-test level)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
@mock_only
|
||||
Scenario: _parse_file_blocks handles file content with triple-backtick code fences inside body
|
||||
Given a mock plan_id "01HQXXXXX"
|
||||
And LLM output containing two FILE blocks each with embedded ```python sections in the content
|
||||
When I call _parse_file_blocks on that LLM output
|
||||
Then it should return 2 changeset entries not 1
|
||||
|
||||
@mock_only
|
||||
Scenario: _parse_file_blocks handles file content with a trailing triple-backtick line
|
||||
Given a mock plan_id "01HYYYYYY"
|
||||
And an llm_output where the file body ends on a line that is exactly ``` after content
|
||||
When I call _parse_file_blocks on that output
|
||||
Then it should return 1 entry
|
||||
|
||||
Scenario: Backwards compatibility — backtick-delimited file blocks still parse correctly (sanitisation)
|
||||
Given a string ````
|
||||
When I run the command "echo 'backtick-sanitise'"
|
||||
And LLM input with backtick-delimited FILE blocks containing code without triple-backticks inside body
|
||||
When I call _parse_file_blocks on that output
|
||||
Then it should return 1 entry
|
||||
@@ -0,0 +1,86 @@
|
||||
@tdd @tdd_issue @tdd_issue_10878
|
||||
Feature: Delimiter _parse_file_blocks handles embedded markdown code fences (#10878)
|
||||
Bug #10878 caused architecture reviews to be truncated because the old
|
||||
backtick delimiter pattern stopped at the first ``` appearing *inside*
|
||||
LLM-generated file content. All subsequent sections were lost - for
|
||||
example a report with 9 TOC entries produced only 3 before truncation.
|
||||
|
||||
The fix introduces unique sentinel markers:
|
||||
<<<<<<< CLEVERAGENTS_FILE_START >>>>>>> / <<<<<<< CLEVERAGENTS_FILE_END >>>>>>>
|
||||
|
||||
This regression test proves the new sentinel delimiters do NOT collide
|
||||
when file bodies contain triple-backtick markdown code fences.
|
||||
|
||||
Forgejo: #10878
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Main regression - embedded ```python code fences inside FILE blocks
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: New delimiter parses all FILE blocks when each body contains embedded ```python code fences
|
||||
Given an LLM response with two FILE blocks, each having ```python sections in the body
|
||||
And a mock plan_id "01HQREGTEST"
|
||||
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
||||
Then it should return 2 changeset entries
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Real-world architecture-review pattern (the original bug)
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: Full regression - report header + three FILE blocks each with inline markdown code fences
|
||||
Given an LLM response mimicking an architecture review with report prose and three FILE blocks each containing markdown code example triple-backticks in their body
|
||||
And a mock plan_id "01HQARCHTEST"
|
||||
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
||||
Then it should return the full entry count of all embedded blocks
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Edge case - trailing ``` at end of file body before END marker
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: A FILE block whose last line before the END sentinel is exactly ``` parses correctly
|
||||
Given an LLM response where a single FILE block body ends on a line that is exactly ``` after code content
|
||||
And a mock plan_id "01HQTRAILT"
|
||||
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
||||
Then it should return 1 changeset entries
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Edge case - sentinel text mentioned inline without full markers
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: LLM body mentions 'CLEVERAGENTS_FILE_END' in prose but does not use the full sentinel marker
|
||||
Given an LLM response where a FILE block body contains the word 'CLEVERAGENTS_FILE_END' referenced in prose within content and another file follows
|
||||
And a mock plan_id "01HQSENTINEL"
|
||||
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
||||
Then it should return 2 changeset entries (second entry has content in body)
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Edge case - git merge-conflict markers alongside new delimiters
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: FILE block bodies contain git merge-conflict-style markers but only the CLEVERAGENTS sentinels match
|
||||
Given an LLM response where FILE block bodies include text like '>>>>>>> some_branch' and '<<<<<<< base' but the actual delimiters are CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END markers
|
||||
And a mock plan_id "01HQGITTEST"
|
||||
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
||||
Then it should return 2 changeset entries
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Escape support for delimiter markers in file content (#262743)
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_10878
|
||||
Scenario: Escaped marker occurrences inside file body are not treated as boundaries
|
||||
Given an LLM response with a single FILE block using legacy markers where the body contains a backslash-escaped <<<<<<< CLEVERAGENTS_FILE_START >>>>>>> sequence that should be preserved as literal text
|
||||
And a mock plan_id "01HQESCTEST"
|
||||
When I parse file blocks using the new delimiter pattern
|
||||
Then it should return 1 changeset entries
|
||||
@@ -0,0 +1,37 @@
|
||||
Feature: CLI main() error handling paths
|
||||
As a developer
|
||||
I want to cover the error-handling branches in cleveragents.cli.main
|
||||
So that convert_exit_code(), subcommand import failure, and config validation are exercised
|
||||
|
||||
@coverage
|
||||
Scenario: invalid --config-path that is a directory is rejected
|
||||
When I run CLI with arguments ["--config-path", "/tmp", "version"]
|
||||
Then the main cli exit code should be 1
|
||||
And the main cli output contains "is not a file"
|
||||
|
||||
@coverage
|
||||
Scenario: unknown command returns exit code 2
|
||||
When I run CLI with arguments ["this_is_not_a_command"]
|
||||
Then the main cli exit code should be 2
|
||||
And the main cli output contains "Invalid command 'this_is_not_a_command'"
|
||||
|
||||
# These tests call convert_exit_code and _print_basic_help directly
|
||||
@coverage
|
||||
Scenario: convert_exit_code(None) returns 0
|
||||
When the main exit-code converter is called with the value "None"
|
||||
Then the main convert_exit_code result is "0"
|
||||
|
||||
@coverage
|
||||
Scenario: convert_exit_code(int -1) returns -1 for negative codes
|
||||
When the main exit-code converter is called with the value "-1"
|
||||
Then the main convert_exit_code result is "-1"
|
||||
|
||||
@coverage
|
||||
Scenario: convert_exit_code(string "abc") returns 1 for unparseable
|
||||
When the main exit-code converter is called with the value "abc"
|
||||
Then the main convert_exit_code result is "1"
|
||||
|
||||
@coverage
|
||||
Scenario: _print_basic_help prints without error
|
||||
When I call the main _print_basic_help
|
||||
Then the main _print_basic_help completes ok
|
||||
@@ -309,13 +309,18 @@ def make_mock_registry(response_content: str) -> SimpleNamespace:
|
||||
|
||||
def make_mock_lifecycle(
|
||||
strategy_actor: str | None = "openai/gpt-4",
|
||||
execution_actor: str | None = None,
|
||||
) -> SimpleNamespace:
|
||||
"""Create a mock lifecycle service."""
|
||||
plan = SimpleNamespace(action_name="test-action")
|
||||
action = SimpleNamespace(strategy_actor=strategy_actor)
|
||||
action = SimpleNamespace(
|
||||
strategy_actor=strategy_actor,
|
||||
execution_actor=execution_actor or strategy_actor,
|
||||
)
|
||||
return SimpleNamespace(
|
||||
get_plan=MagicMock(return_value=plan),
|
||||
get_action=MagicMock(return_value=action),
|
||||
resolve_actor_provider_model=MagicMock(return_value=None),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
Feature: Plan apply correction diff rendering functions
|
||||
As a developer maintaining plan_apply_service.py
|
||||
I want to cover _build_correction_diff_dict, _render_correction_diff_plain, and _render_correction_diff_rich
|
||||
So that all output rendering branches are exercised
|
||||
|
||||
@coverage
|
||||
Scenario: Build correction diff dict with full data including artifacts path
|
||||
Given I have a CorrectionAttemptRecord with mode="revert", original_decision_id="dec1", new_decision_id="dec2", guidance="Test fix", and archived_artifacts_path="/artifacts/zip"
|
||||
When I call _build_correction_diff_dict on the record
|
||||
Then the result should contain "correction_diff", "comparison", and "patch_preview" keys
|
||||
And the comparison should have 1 reverted decision and 1 added decision
|
||||
And the patch preview should include the artifacts path
|
||||
|
||||
@coverage
|
||||
Scenario: Build correction diff dict without new_decision_id or archived artifacts
|
||||
Given I have a CorrectionAttemptRecord with mode="append", original_decision_id="dec3", no new_decision, guidance="Review needed", and no archived artifacts
|
||||
When I call _build_correction_diff_dict on the record
|
||||
Then the result should contain "correction_diff", "comparison", and "patch_preview" keys
|
||||
And the comparison counts are "1" reverted decisions and "0" new ones
|
||||
And the patch preview should indicate correction execution is pending
|
||||
|
||||
@coverage
|
||||
Scenario: Render correction diff dict as plain text with artifacts
|
||||
Given I have a correction diff dict with mode="revert", original_decision_id="dec-100", guidance="Fix logic", completed_at present, and one added decision "dec-200"
|
||||
When I call _render_correction_diff_plain on the dict
|
||||
Then the output should contain "Correction Diff" heading
|
||||
And the output should contain the mode value "revert"
|
||||
And the output should contain "[OK] Correction diff generated" trailer
|
||||
And the output should contain "Completed:" timestamp
|
||||
And the output should contain the added decision line starting with "+ dec-200"
|
||||
|
||||
@coverage
|
||||
Scenario: Render correction diff dict as plain text without completed_at or guidance
|
||||
Given I have a correction diff dict with mode="append", original_decision_id="dec-300", no guidance text, and patch preview note about pending execution
|
||||
When I call _render_correction_diff_plain on the dict
|
||||
Then the output should contain "Correction Diff" heading
|
||||
And the output should NOT contain "Completed:" line
|
||||
And the output should NOT contain a "Guidance:" line
|
||||
And the output should contain "[OK] Correction diff generated" trailer
|
||||
And the output should contain the pending-execution note text
|
||||
|
||||
@coverage
|
||||
Scenario: Render correction diff dict as rich text with artifacts
|
||||
Given I have a correction diff dict with mode="revert", original_decision_id="dec-rc1", guidance="Patch rollback", completed_at present, and one added decision "dec-rc2"
|
||||
When I call _render_correction_diff_rich on the dict
|
||||
Then the output should contain "[bold]Correction Diff[/bold]" heading
|
||||
And the output should contain "[cyan]revert[/cyan]" mode color markup
|
||||
And the output should contain the rich green checkmark marker at end
|
||||
And the output should contain a red-colored reverted decision ID
|
||||
And the output should contain a green-colored added decision ID
|
||||
|
||||
@coverage
|
||||
Scenario: Render correction diff dict as rich text without guidance or completion
|
||||
Given I have a correction diff dict with mode="append", original_decision_id="dec-rc3", no guidance, and patch preview note about pending execution
|
||||
When I call _render_correction_diff_rich on the dict
|
||||
Then the output should contain "[bold]Correction Diff[/bold]" heading
|
||||
And the output should NOT contain a "Guidance" line
|
||||
And the output should contain the pending-execution note in dim markup
|
||||
@@ -13,7 +13,7 @@ Feature: PlanApplyService branch coverage for handle_merge_failure logger path
|
||||
When pas_branch I call handle_merge_failure and capture any exception
|
||||
Then pas_branch a RuntimeError should have been raised
|
||||
And pas_branch the plan error_details should still contain merge_conflict
|
||||
And pas_branch lifecycle _commit_plan should have been invoked before the error
|
||||
And pas_branch lifecycle commit_plan should have been invoked before the error
|
||||
And pas_branch lifecycle fail_apply should have been invoked before the error
|
||||
|
||||
Scenario: handle_merge_failure returns plan when logger.error succeeds
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
@coverage @context
|
||||
Feature: PlanExecutionContext coverage boost
|
||||
As a developer
|
||||
I want complete coverage of PlanExecutionContext and RuntimeExecuteActor
|
||||
So that all uncovered lines in plan_execution_context.py are exercised
|
||||
|
||||
Scenario: PlanExecutionContext rejects empty plan_id
|
||||
Given a pec fresh context
|
||||
When I pec construct PlanExecutionContext with empty plan_id
|
||||
Then a pec ValidationError should be raised with "plan_id must not be empty"
|
||||
|
||||
Scenario: PlanExecutionContext properties return configured values
|
||||
Given a pec PlanExecutionContext with all optional fields set
|
||||
Then pec the automation_profile property should return the configured value
|
||||
And pec the plan_env property should return the configured value
|
||||
And pec the project_env property should return the configured value
|
||||
And pec the project_resources property should return the configured dict
|
||||
And pec the resource_bindings property should return the configured dict
|
||||
And pec the changeset_store property should return the configured store
|
||||
And pec the active_changeset_ids property should return a list
|
||||
|
||||
Scenario: record_change raises PlanError when no changeset is active
|
||||
Given a pec PlanExecutionContext with a valid plan_id
|
||||
When I pec call record_change without starting a changeset
|
||||
Then a pec PlanError should be raised about no active changeset
|
||||
|
||||
Scenario: get_changeset returns None for unknown changeset
|
||||
Given a pec PlanExecutionContext with a valid plan_id
|
||||
When I pec call get_changeset with a nonexistent changeset_id
|
||||
Then pec the result should be None
|
||||
|
||||
Scenario: summarize returns metadata for execution context
|
||||
Given a pec PlanExecutionContext with a valid plan_id
|
||||
When I pec call summarize on the execution context
|
||||
Then pec the summary should contain plan_id and decision_root_id
|
||||
And pec the summary should contain counts for resources and bindings
|
||||
|
||||
Scenario: RuntimeExecuteActor rejects None tool_runner
|
||||
Given a pec valid PlanExecutionContext
|
||||
When I pec construct RuntimeExecuteActor with None tool_runner
|
||||
Then a pec ValidationError should be raised with "tool_runner must not be None"
|
||||
|
||||
Scenario: RuntimeExecuteActor rejects None execution_context
|
||||
Given a pec mock ToolRunner
|
||||
When I pec construct RuntimeExecuteActor with None execution_context
|
||||
Then a pec ValidationError should be raised with "execution_context must not be None"
|
||||
|
||||
Scenario: RuntimeExecuteActor properties return configured values
|
||||
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context
|
||||
Then pec the tool_runner property should return the configured ToolRunner
|
||||
And pec the execution_context property should return the configured PlanExecutionContext
|
||||
|
||||
Scenario: RuntimeExecuteActor execute with stream callback produces events
|
||||
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context with sandbox_root
|
||||
And a pec stream event collector
|
||||
When I pec execute with decisions and stream callback
|
||||
Then pec the stream events should include "runtime_execute_started"
|
||||
And pec the stream events should include "runtime_execute_complete"
|
||||
|
||||
Scenario: RuntimeExecuteActor execute without stream callback still works
|
||||
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context
|
||||
When I pec execute with decisions and no stream callback
|
||||
Then pec the result should be a RuntimeExecuteResult with a valid changeset_id
|
||||
|
||||
Scenario: RuntimeExecuteActor execute with sandbox_root populates sandbox_refs
|
||||
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context with sandbox_root
|
||||
When I pec execute with decisions and no stream callback
|
||||
Then pec the result sandbox_refs should include the sandbox_root
|
||||
@@ -0,0 +1,28 @@
|
||||
@mock_only @subplan @plan_executor @tdd_issue @tdd_issue_10268
|
||||
Feature: PlanExecutor hierarchical execution wiring
|
||||
As a system executing hierarchical plans via the production CLI
|
||||
I want SubplanExecutionService to be available when SubplanService is wired
|
||||
So that child plan execution works without a pre-configured execution service
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SubplanExecutionService lazy creation (Forgejo #10268)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: PlanExecutor lazily creates SubplanExecutionService when SubplanService is present
|
||||
Given a PlanExecutor with SubplanService but no SubplanExecutionService
|
||||
And a parent plan in Execute phase with a subplan_spawn decision
|
||||
When I call run_execute on the parent plan
|
||||
Then SubplanService.get_spawn_decisions should have been called
|
||||
And SubplanService.spawn should have been called with the spawn entries
|
||||
And SubplanExecutionService.execute_all should have been called via lazy creation
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan callback (Forgejo #10268)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _execute_child_plan callback invokes strategize for a child plan
|
||||
Given a PlanExecutor with SubplanService but no SubplanExecutionService
|
||||
And a parent plan in Execute phase with a subplan_spawn decision
|
||||
And a child plan ready for execution in the lifecycle service
|
||||
When I call run_execute on the parent plan
|
||||
Then SubplanExecutionService.execute_all should have been called via lazy creation
|
||||
@@ -0,0 +1,85 @@
|
||||
@subplan @plan_executor @integration @phase4 @tdd_issue @tdd_issue_10270
|
||||
Feature: Plan hierarchy 4-phase lifecycle execution
|
||||
As a system executing hierarchical plans
|
||||
I want child plans to independently complete all 4 phases — Strategize, Decompose, Execute, Validate
|
||||
So that hierarchical decomposition produces fully executed, verifiable, and aggregated results
|
||||
|
||||
Background:
|
||||
Given a plan lifecycle service with in-memory storage
|
||||
And a DecisionService for recording spawn decisions
|
||||
And a SubplanService backed by the DecisionService
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Happy path — 2-level hierarchy (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Parent plan with 2-level hierarchy executes all phases for parent and children
|
||||
Given a parent plan "parent-happy" in Strategize phase with a spawn decision recorded
|
||||
When I create two child plans for "parent-happy" and pre-register them in the lifecycle
|
||||
And I strategize and execute the parent plan "parent-happy"
|
||||
Then the parent plan Strategize phase should be complete
|
||||
And the parent plan Execute phase should be complete
|
||||
And each child plan should have completed Strategize phase
|
||||
And each child plan should have completed Execute phase
|
||||
And the parent plan should have 1 subplan statuses
|
||||
And all child subplan statuses should indicate success
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Checkpoint triggers — on_subplan_spawn (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: on_subplan_spawn checkpoint is created before first child execution
|
||||
Given a parent plan "parent-checkpoint" in Strategize phase with a spawn decision recorded
|
||||
And I create one child plan for "parent-checkpoint" and pre-register it in the lifecycle
|
||||
And a CheckpointService is configured for the parent plan "parent-checkpoint"
|
||||
When I strategize and execute the parent plan "parent-checkpoint"
|
||||
Then at least one on_subplan_spawn checkpoint should have been created
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Max-child-depth enforcement (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Decomposition respects max-child-depth during execution
|
||||
Given a decomposition service with max-child-depth 1
|
||||
And a project with 100 files across 5 directory levels
|
||||
When I decompose the project files with stored config
|
||||
Then the decomposition result should have max_depth_reached <= 1
|
||||
|
||||
Scenario: Decomposition with default max-child-depth allows moderate nesting
|
||||
Given a decomposition service with default config
|
||||
And a project with 200 files across 5 directory levels
|
||||
When I decompose the project files with stored config
|
||||
Then the decomposition result should have max_depth_reached >= 1
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Child success aggregation (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Parent plan aggregates child subplan results after execution
|
||||
Given a parent plan "parent-aggregate" in Strategize phase with two spawn decisions recorded
|
||||
When I create two child plans for "parent-aggregate" and pre-register them in the lifecycle
|
||||
And I strategize and execute the parent plan "parent-aggregate"
|
||||
Then the parent plan should have subplan statuses with execution results
|
||||
And the parent plan error_details should not contain failed_subplan_ids
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Child failure handling (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Parent plan captures child plan failure in error_details
|
||||
Given a parent plan "parent-fail" in Strategize phase with a spawn decision recorded without action registration
|
||||
When I strategize and execute the parent plan "parent-fail"
|
||||
Then the parent plan error_details should have failed subplan ids
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Nested execution — 3-level hierarchy (Forgejo #10270)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Grandchild plan in 3-level hierarchy executes and cascades results
|
||||
Given a parent plan "parent-nested" in Strategize phase with a spawn decision recorded
|
||||
And I create child plan "child-nested" for "parent-nested" with its own spawn decision recorded
|
||||
And I create a grandchild plan for "child-nested" and pre-register it in the lifecycle
|
||||
When I strategize and execute the parent plan "parent-nested"
|
||||
Then the parent plan should have 1 subplan statuses
|
||||
And the child plan should complete both Strategize and Execute phases
|
||||
And the grandchild plan should complete both Strategize and Execute phases
|
||||
@@ -0,0 +1,124 @@
|
||||
@plan @executor @subplan @coverage
|
||||
Feature: PlanExecutor child plan execution coverage
|
||||
As a developer
|
||||
I want comprehensive tests for _execute_child_plan, _execute_subplans fallback,
|
||||
_apply_subplan_results_to_plan exec_result=None, and checkpoint PlanError handling
|
||||
So that uncovered lines in plan_executor.py are fully exercised
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan — success path (lines 606, 616-649, 662)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Execute child plan succeeds in stub mode
|
||||
Given a child3 PlanExecutor for child plan execution
|
||||
And child3 a SubplanStatus for a valid child plan
|
||||
When I child3 call _execute_child_plan on the executor
|
||||
Then child3 the SubplanExecutionOutput should indicate success
|
||||
And child3 the SubplanExecutionOutput should have a changeset_summary
|
||||
And child3 the running_plan_ids should be empty
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan — circular detection (lines 607-615)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Execute child plan detects circular re-entrant execution
|
||||
Given a child3 PlanExecutor for child plan execution
|
||||
And child3 a SubplanStatus for a valid child plan
|
||||
And I child3 pre-add the subplan_id to running_plan_ids
|
||||
When I child3 call _execute_child_plan on the executor
|
||||
Then child3 the SubplanExecutionOutput should indicate failure
|
||||
And child3 the SubplanExecutionOutput error should mention circular
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan — plan not found (lines 620-624)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Execute child plan returns failure when child plan not found
|
||||
Given a child3 PlanExecutor with lifecycle returning None for child plan
|
||||
And child3 a SubplanStatus for a valid child plan
|
||||
When I child3 call _execute_child_plan on the executor
|
||||
Then child3 the SubplanExecutionOutput should indicate failure
|
||||
And child3 the SubplanExecutionOutput error should mention "not found"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan — strategize produces no decisions (lines 626-634)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Execute child plan returns failure when strategize produces no decisions
|
||||
Given a child3 PlanExecutor for child plan with empty definition
|
||||
And child3 a SubplanStatus for a valid child plan
|
||||
When I child3 call _execute_child_plan on the executor
|
||||
Then child3 the SubplanExecutionOutput should indicate failure
|
||||
And child3 the SubplanExecutionOutput error should mention "no decisions"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_child_plan — exception during execution (lines 650-660)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Execute child plan catches exception and returns failure output
|
||||
Given a child3 PlanExecutor whose lifecycle start_strategize raises RuntimeError
|
||||
And child3 a SubplanStatus for a valid child plan
|
||||
When I child3 call _execute_child_plan on the executor
|
||||
Then child3 the SubplanExecutionOutput should indicate failure
|
||||
And child3 the SubplanExecutionOutput error should contain the exception message
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_subplans — fallback when subplan_execution_service is None
|
||||
# but subplan_service is available (lines 521-527)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _execute_subplans creates SubplanExecutionService when not injected
|
||||
Given a child3 PlanExecutor with subplan_service but no subplan_execution_service
|
||||
And child3 a SpawnResult with one spawned status
|
||||
When I child3 call _execute_subplans on the executor
|
||||
Then child3 the SubplanExecutionResult should not be None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_subplans — returns None when no subplan service (line 521)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _execute_subplans returns None when both services are missing
|
||||
Given a child3 PlanExecutor with no subplan services at all
|
||||
And child3 a SpawnResult with one spawned status
|
||||
When I child3 call _execute_subplans on the executor
|
||||
Then child3 the SubplanExecutionResult should be None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _execute_subplans — returns None when statuses is empty (line 532)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _execute_subplans returns None when spawned_statuses is empty
|
||||
Given a child3 PlanExecutor with subplan_service and success executor
|
||||
And child3 a SpawnResult with no spawned statuses
|
||||
When I child3 call _execute_subplans on the executor
|
||||
Then child3 the SubplanExecutionResult should be None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _apply_subplan_results_to_plan — exec_result is None (lines 569-570)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _apply_subplan_results keeps spawned statuses when exec_result is None
|
||||
Given a child3 PlanExecutor for apply subplan results
|
||||
And child3 a SpawnResult with one spawned status
|
||||
When I child3 call _apply_subplan_results_to_plan with exec_result None
|
||||
Then child3 the plan subplan_statuses should match the spawn result statuses
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Property accessors for optional services (lines 433, 438, 443)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Optional service properties return configured values
|
||||
Given a child3 PlanExecutor with all optional services configured
|
||||
Then child3 the fix_revalidate_orchestrator property should return the configured instance
|
||||
And child3 the subplan_service property should return the configured instance
|
||||
And child3 the subplan_execution_service property should return the configured instance
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# _try_create_checkpoint — PlanError re-raise (lines 731-741, 745)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: _try_create_checkpoint re-raises PlanError from plan update failure
|
||||
Given a child3 PlanExecutor with checkpoint manager and lifecycle that fails on commit_plan
|
||||
And child3 a sandbox that returns a valid checkpoint
|
||||
When I child3 attempt _try_create_checkpoint expecting PlanError
|
||||
Then a child3 PlanError should be raised about persisting checkpoint metadata
|
||||
@@ -225,7 +225,7 @@ Feature: PlanExecutor comprehensive coverage
|
||||
Then the cov2 strategize run result should be a StrategizeResult
|
||||
And the cov2 lifecycle should have called start_strategize
|
||||
And the cov2 lifecycle should have called complete_strategize
|
||||
And the cov2 lifecycle should have called _commit_plan
|
||||
And the cov2 lifecycle should have called commit_plan
|
||||
|
||||
Scenario: run_strategize sets decision_root_id on execution context
|
||||
Given a cov2 mock lifecycle service
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
@tdd @tdd_issue @tdd_issue_10938 @tier-hydration
|
||||
Feature: Tier hydrator integration in PlanExecutor run_strategize (#10938)
|
||||
Verifies that PlanExecutor.run_strategize correctly:
|
||||
- Calls hydrate_tiers_for_plan when tier_service, project_repository, and
|
||||
resource_registry are all wired
|
||||
- Skips hydration if this plan_id has already been hydrated by this executor
|
||||
instance (plan-id-scoped cache, prevents cross-plan contamination)
|
||||
- Catches hydration failures without aborting the strategize phase
|
||||
|
||||
Forgejo: #10938
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Success — hydrate_tiers_for_plan is called when all dependencies exist
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
Scenario: Hydration is invoked when tier_service, project_repository and resource_registry are present
|
||||
Given a mock PlanExecutor with tier_service, project_repository and resource_registry wired in
|
||||
And a plan in the Strategize phase
|
||||
When I call run_strategize on that PlanExecutor
|
||||
Then hydrate_tiers_for_plan should have been called once
|
||||
And the plan executor strategize result should be a valid run_strategize response
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Cache — skip already-hydrated tiers (new block from PR #10938)
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
Scenario: Hydration is skipped when plan_id has already been hydrated by this executor
|
||||
Given a mock PlanExecutor with tier_service and plan_id already marked as hydrated
|
||||
And a plan in the Strategize phase
|
||||
When I call run_strategize on that PlanExecutor
|
||||
Then hydrate_tiers_for_plan should not have been called because the plan was already hydrated
|
||||
And the plan executor strategize result should be a valid run_strategize response
|
||||
|
||||
Scenario: Hydration runs when hot tier is empty
|
||||
Given a mock PlanExecutor with tier_service but no pre-existing hot fragments
|
||||
And a plan in the Strategize phase
|
||||
When I call run_strategize on that PlanExecutor
|
||||
Then hydrate_tiers_for_plan should have been called once
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Failure — hydration failure does not block strategize (new try/except from PR #10938)
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
Scenario: Hydration OSError is caught non-fatally and strategize continues
|
||||
Given a mock PlanExecutor with tier_service, project_repository and resource_registry where hydrate fails with OSError
|
||||
And a plan in the Strategize phase
|
||||
When I call run_strategize on that PlanExecutor
|
||||
Then hydrate_tiers_for_plan should have been called once
|
||||
Then the strategy result should contain decisions (strategize succeeded despite hydration failure)
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# No-op — tier_service not wired does not crash
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
Scenario: Strategize proceeds when tier_service is None (no hydration attempted)
|
||||
Given a mock PlanExecutor without tier_service
|
||||
And a plan in the Strategize phase
|
||||
When I call run_strategize on that PlanExecutor
|
||||
Then hydrate_tiers_for_plan should not have been called
|
||||
And the plan executor strategize result should be a valid run_strategize response
|
||||
@@ -160,4 +160,45 @@ Scenario: _handle_correction_applied skips event with no plan_id
|
||||
Given the plan lifecycle service is configured for r4
|
||||
And get_plan will fail for the plan for r4
|
||||
When I handle a CORRECTION_APPLIED event when get_plan fails for r4
|
||||
Then no exception should be raised for r4
|
||||
Then no exception should be raised for r4
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# resolve_actor_provider_model — namespace/name → provider/model
|
||||
# Lines 731-764 (factor resolution for #11254)
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model returns provider/model for known provider
|
||||
Given the plan lifecycle service has a unit of work for actor resolution r4
|
||||
When I resolve actor provider model for "openai/gpt-4"
|
||||
Then the resolved actor provider model should be "openai/gpt-4"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model resolves namespace/name via actor lookup
|
||||
Given the plan lifecycle service has a unit of work with a known actor "local/strategist" using "openai/gpt-4" for r4
|
||||
When I resolve actor provider model for "local/strategist"
|
||||
Then the resolved actor provider model should be "openai/gpt-4"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model returns None when actor is not found
|
||||
Given the plan lifecycle service has a unit of work without actor "local/missing-actor" for r4
|
||||
When I resolve actor provider model for "local/missing-actor"
|
||||
Then the resolved actor provider model should be None
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model catches exception from actor lookup
|
||||
Given the plan lifecycle service has a unit of work that raises on actor lookup for r4
|
||||
When I resolve actor provider model for "local/crash-actor"
|
||||
Then the resolved actor provider model should be None
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model returns None when unit_of_work is absent
|
||||
Given the plan lifecycle service has no unit of work for r4
|
||||
When I resolve actor provider model for "local/strategist"
|
||||
Then the resolved actor provider model should be None
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: resolve_actor_provider_model returns None for empty actor name
|
||||
Given the plan lifecycle service has a unit of work for actor resolution r4
|
||||
When I resolve actor provider model for ""
|
||||
Then the resolved actor provider model should be None
|
||||
@@ -87,10 +87,10 @@ Feature: Reactive application coverage boost
|
||||
Then a CleverAgentsException should be raised for resolution failure
|
||||
|
||||
@coverage
|
||||
Scenario: Reactive app skill injection skips LLM agents without tools
|
||||
Scenario: Reactive app skill injection upgrades LLM agents to ToolCallingAgent
|
||||
Given a reactive app with skill tools and a config with an LLM agent
|
||||
When agents are registered from config with skill tools present
|
||||
Then the LLM agent should remain a SimpleLLMAgent not a SimpleToolAgent
|
||||
Then the LLM agent should be a ToolCallingAgent not a SimpleLLMAgent
|
||||
|
||||
@coverage
|
||||
Scenario: Reactive app raises error for skill resolution ValueError
|
||||
|
||||
@@ -33,6 +33,8 @@ def _make_app(
|
||||
app_exec = MagicMock()
|
||||
app_exec.config = SimpleNamespace(global_context=dict(config_global_context or {}))
|
||||
app_exec.run_single_shot = AsyncMock(return_value=result)
|
||||
# last_run_tool_calls is an int property; set to 0 so comparisons work in tests
|
||||
app_exec.last_run_tool_calls = 0
|
||||
if run_side_effect is not None:
|
||||
app_exec.run_single_shot.side_effect = run_side_effect
|
||||
return app_exec
|
||||
|
||||
@@ -41,6 +41,8 @@ def _make_app(
|
||||
app_exec = MagicMock()
|
||||
app_exec.config = SimpleNamespace(global_context=dict(config_global_context or {}))
|
||||
app_exec.run_single_shot = AsyncMock(return_value=result)
|
||||
# last_run_tool_calls is an int property; set to 0 so comparisons work in tests
|
||||
app_exec.last_run_tool_calls = 0
|
||||
return app_exec
|
||||
|
||||
|
||||
@@ -165,7 +167,7 @@ def step_resolve_with_no_config_data(context: Any) -> None:
|
||||
try:
|
||||
resolve_config_files("local/empty-actor", [])
|
||||
context.resolve_exit_code = 0
|
||||
except (SystemExit, click.exceptions.Exit) as exc:
|
||||
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
|
||||
context.resolve_exit_code = getattr(
|
||||
exc, "exit_code", getattr(exc, "code", 1)
|
||||
)
|
||||
@@ -209,7 +211,7 @@ def step_resolve_unknown_actor_directly(context: Any) -> None:
|
||||
try:
|
||||
resolve_config_files("nonexistent/actor", [])
|
||||
context.resolve_exit_code = 0
|
||||
except (SystemExit, click.exceptions.Exit) as exc:
|
||||
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
|
||||
context.resolve_exit_code = getattr(
|
||||
exc, "exit_code", getattr(exc, "code", 1)
|
||||
)
|
||||
@@ -260,7 +262,7 @@ def step_resolve_with_empty_config_blob(context: Any) -> None:
|
||||
try:
|
||||
resolve_config_files("local/empty-blob-actor", [])
|
||||
context.resolve_exit_code = 0
|
||||
except (SystemExit, click.exceptions.Exit) as exc:
|
||||
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
|
||||
context.resolve_exit_code = getattr(
|
||||
exc, "exit_code", getattr(exc, "code", 1)
|
||||
)
|
||||
@@ -356,7 +358,7 @@ def step_resolve_with_unserializable_config_blob(context: Any) -> None:
|
||||
try:
|
||||
resolve_config_files("local/bad-blob-actor", [])
|
||||
context.resolve_exit_code = 0
|
||||
except (SystemExit, click.exceptions.Exit) as exc:
|
||||
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
|
||||
context.resolve_exit_code = getattr(
|
||||
exc, "exit_code", getattr(exc, "code", 1)
|
||||
)
|
||||
|
||||
@@ -49,7 +49,7 @@ def step_resolve_with_control_character_name(context: Any) -> None:
|
||||
try:
|
||||
resolve_config_files(actor_name, [])
|
||||
context.resolve_exit_code = 0
|
||||
except (SystemExit, click.exceptions.Exit) as exc:
|
||||
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
|
||||
context.resolve_exit_code = getattr(
|
||||
exc, "exit_code", getattr(exc, "code", 1)
|
||||
)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -14,6 +14,7 @@ from unittest.mock import patch
|
||||
|
||||
import yaml
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.actor.compiler import ActorCompilationError
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
@@ -781,3 +782,77 @@ def step_graph_nodes_have_lsp_dict(context: Any) -> None:
|
||||
assert lsp.get("auto") is True, (
|
||||
f"Node '{node_id}' expected lsp.auto=True, got {lsp}"
|
||||
)
|
||||
|
||||
|
||||
# ── M5: options block forwarding (#11223) ─────────────────────────────────
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict with options block")
|
||||
def step_v3_llm_config_with_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/my-llama-actor",
|
||||
"type": "llm",
|
||||
"description": "Local llama.cpp actor",
|
||||
"provider": "openai",
|
||||
"model": "my-local-model",
|
||||
"options": {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
"temperature": 1.0,
|
||||
},
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict without options block")
|
||||
def step_v3_llm_config_without_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/standard-actor",
|
||||
"type": "llm",
|
||||
"description": "Standard actor without options",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@then("the agent config should contain the options block")
|
||||
def step_agent_config_has_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
options = agent.config.get("options")
|
||||
assert options == {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
"temperature": 1.0,
|
||||
}, f"Expected exact options dict, got {options}"
|
||||
|
||||
|
||||
@then("the agent config should not contain an options key")
|
||||
def step_agent_config_no_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
assert "options" not in agent.config, (
|
||||
f"Expected no 'options' key in agent config, got {agent.config}"
|
||||
)
|
||||
|
||||
|
||||
@given("a v3 LLM actor config dict with empty options block")
|
||||
def step_v3_llm_config_with_empty_options(context: Context) -> None:
|
||||
context.v3_config = {
|
||||
"name": "local/empty-options-actor",
|
||||
"type": "llm",
|
||||
"description": "Actor with empty options",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"options": {},
|
||||
"system_prompt": "You are a helpful assistant.",
|
||||
}
|
||||
|
||||
|
||||
@then("the agent config should contain an empty options block")
|
||||
def step_agent_config_has_empty_options(context: Context) -> None:
|
||||
agents = context.reactive_config.agents
|
||||
agent = next(iter(agents.values()))
|
||||
options = agent.config.get("options")
|
||||
assert options == {}, f"Expected empty options dict, got {options}"
|
||||
|
||||
@@ -89,7 +89,7 @@ def _make_service(
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle.complete_apply.return_value = plan
|
||||
lifecycle.constrain_apply.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
|
||||
store = MagicMock()
|
||||
if changeset is not None:
|
||||
|
||||
@@ -921,7 +921,7 @@ def _make_guardrail_mock_lifecycle(plan: MagicMock) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ def _make_lifecycle_mock(plan: Any) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ def step_then_tier_svc_singleton(context: Any) -> None:
|
||||
def step_then_settings_hot(context: Any) -> None:
|
||||
s = Settings()
|
||||
assert hasattr(s, "context_max_tokens_hot")
|
||||
assert s.context_max_tokens_hot == 16000
|
||||
assert s.context_max_tokens_hot == 32000 # Doubled from 16000 to 32000 (M5 fix)
|
||||
|
||||
|
||||
@then("settings should have context_max_decisions_warm")
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Step definitions for db_url_sanitisation.feature.
|
||||
|
||||
Tests the ``_sanitise_db_url`` helper and its use in ``build_info_data``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from tempfile import mkdtemp
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('the database url is "{url}"')
|
||||
def step_db_url_given(context: Context, url: str) -> None:
|
||||
"""Store the raw database URL for processing."""
|
||||
context.raw_db_url = url
|
||||
|
||||
|
||||
@given('a mock settings object with database url "{db_url}"')
|
||||
def step_mock_settings_with_db_url(context: Context, db_url: str) -> None:
|
||||
"""Build a full mock Settings that ``build_info_data`` expects."""
|
||||
tmpdir = mkdtemp()
|
||||
mock_settings = MagicMock()
|
||||
mock_settings.database_url = db_url
|
||||
mock_settings.data_dir = Path(tmpdir)
|
||||
mock_settings.storage_path = Path(tmpdir)
|
||||
mock_settings.config_path = Path(tmpdir) / "config.toml"
|
||||
mock_settings.log_dir = Path(tmpdir) / "logs"
|
||||
mock_settings.default_automation_profile = "auto"
|
||||
mock_settings.has_provider_configured = MagicMock(return_value=False)
|
||||
mock_settings.configured_provider_names = MagicMock(return_value=[])
|
||||
mock_settings.debug_enabled = False
|
||||
context.mock_settings = mock_settings
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I run sanitise_db_url")
|
||||
def step_run_sanitise(context: Context) -> None:
|
||||
"""Call _sanitise_db_url with the stored raw URL."""
|
||||
from cleveragents.cli.commands.system import _sanitise_db_url
|
||||
|
||||
context.sanitised_url = _sanitise_db_url(context.raw_db_url)
|
||||
|
||||
|
||||
@when("I call build_info_data")
|
||||
def step_call_build_info(context: Context) -> None:
|
||||
"""Call ``build_info_data`` with the mock settings."""
|
||||
from cleveragents.cli.commands.system import build_info_data
|
||||
|
||||
with patch(
|
||||
"cleveragents.config.settings.get_settings",
|
||||
return_value=context.mock_settings,
|
||||
):
|
||||
context.info_data = build_info_data()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the sanitised database url should be "{expected}"')
|
||||
def step_assert_sanitised_url(context: Context, expected: str) -> None:
|
||||
"""Verify the sanitised URL matches expectation."""
|
||||
actual = context.sanitised_url
|
||||
assert actual == expected, f"Expected '{expected}', got '{actual}'"
|
||||
|
||||
|
||||
@then('the database field in info data should be "{expected}"')
|
||||
def step_assert_info_db_field(context: Context, expected: str) -> None:
|
||||
"""Verify the sanitised URL appears correctly in build_info_data output."""
|
||||
actual = context.info_data["database"]
|
||||
assert actual == expected, f"Expected database field '{expected}', got '{actual}'"
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Step implementations for decomposition_max_child_depth.feature.
|
||||
|
||||
Tests the enforcement of max_child_depth in DecompositionService._build_hierarchy.
|
||||
Reuses existing steps from large_project_decomposition_steps.py where applicable.
|
||||
Only new, unique step patterns are defined here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import then, when # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.application.services.decomposition_models import (
|
||||
DecompositionConfig,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_DECOMP_SVC_LOGGER = "cleveragents.application.services.decomposition_service.logger"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Whens
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I decompose with max_child_depth {mcd:d} and max_depth {md:d}")
|
||||
def step_when_decompose_max_child_depth_and_max_depth(
|
||||
context: Any,
|
||||
mcd: int,
|
||||
md: int,
|
||||
) -> None:
|
||||
cfg = DecompositionConfig(
|
||||
max_child_depth=mcd,
|
||||
max_depth=md,
|
||||
max_files_per_subplan=30,
|
||||
)
|
||||
with patch(_DECOMP_SVC_LOGGER) as mock_logger:
|
||||
context.result = context.svc.decompose(context.files, cfg)
|
||||
context.mock_logger = mock_logger
|
||||
|
||||
|
||||
@when("I decompose with default max_child_depth")
|
||||
def step_when_decompose_default_mcd(context: Any) -> None:
|
||||
cfg = DecompositionConfig(
|
||||
max_child_depth=5,
|
||||
max_files_per_subplan=30,
|
||||
)
|
||||
with patch(_DECOMP_SVC_LOGGER) as mock_logger:
|
||||
context.result = context.svc.decompose(context.files, cfg)
|
||||
context.mock_logger = mock_logger
|
||||
|
||||
|
||||
@when("I create a config with max_child_depth {n:d}")
|
||||
def step_when_create_bad_max_child_depth(context: Any, n: int) -> None:
|
||||
try:
|
||||
DecompositionConfig(max_child_depth=n)
|
||||
except ValueError as exc:
|
||||
context.error = exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thens
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the decomposition warning log should contain "{text}"')
|
||||
def step_then_warning_log_contains(context: Any, text: str) -> None:
|
||||
assert context.mock_logger is not None, "Mock logger was not set"
|
||||
assert context.mock_logger.warning.called, "Expected logger.warning to be called"
|
||||
args_str = str(context.mock_logger.warning.call_args_list)
|
||||
assert text in args_str, f"Expected '{text}' in warning calls, got: {args_str}"
|
||||
|
||||
|
||||
@then("max_child_depth should be {n:d}")
|
||||
def step_then_cfg_max_child_depth(context: Any, n: int) -> None:
|
||||
assert context.decomp_config.max_child_depth == n
|
||||
@@ -63,7 +63,7 @@ def _make_service(
|
||||
"""Create an ErrorRecoveryService with mocked lifecycle."""
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
return ErrorRecoveryService(
|
||||
lifecycle_service=lifecycle,
|
||||
auto_retry_threshold=auto_retry_threshold,
|
||||
@@ -574,7 +574,7 @@ def step_given_executor_with_recovery(context: Context) -> None:
|
||||
plan.definition_of_done = "- Step one\n- Step two"
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle.update_error_details = MagicMock()
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
|
||||
# Build error recovery service wrapping the same lifecycle
|
||||
er_service = ErrorRecoveryService(
|
||||
@@ -646,7 +646,7 @@ def step_given_executor_max_retries(context: Context, n: str) -> None:
|
||||
plan.definition_of_done = "- Step one"
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle.update_error_details = MagicMock()
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
|
||||
er_service = ErrorRecoveryService(
|
||||
lifecycle_service=lifecycle,
|
||||
|
||||
@@ -365,8 +365,7 @@ def step_use_estimation_action_and_complete_strategize(context: Context) -> None
|
||||
# Manually set processing state to PROCESSING then COMPLETE
|
||||
plan = context.est_lifecycle.get_plan(plan_id)
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
# TODO: Use public save_plan() once test helpers are refactored
|
||||
context.est_lifecycle._commit_plan(plan)
|
||||
context.est_lifecycle.commit_plan(plan)
|
||||
context.est_lifecycle.complete_strategize(plan_id)
|
||||
|
||||
|
||||
|
||||
@@ -85,7 +85,7 @@ def _drive_to_execute(svc: PlanLifecycleService, plan_id: str) -> None:
|
||||
svc.start_strategize(plan_id)
|
||||
plan = svc.get_plan(plan_id)
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
svc.complete_strategize(plan_id)
|
||||
# complete_strategize calls auto_progress; if still in STRATEGIZE/COMPLETE
|
||||
# we need to call execute_plan explicitly.
|
||||
@@ -300,7 +300,7 @@ def step_use_and_complete_strategize_lifecycle(context: Context) -> None:
|
||||
context.lifecycle_svc.start_strategize(plan_id)
|
||||
p = context.lifecycle_svc.get_plan(plan_id)
|
||||
p.processing_state = ProcessingState.PROCESSING
|
||||
context.lifecycle_svc._commit_plan(p)
|
||||
context.lifecycle_svc.commit_plan(p)
|
||||
context.lifecycle_svc.complete_strategize(plan_id)
|
||||
context.lifecycle_plan = context.lifecycle_svc.get_plan(plan_id)
|
||||
|
||||
@@ -426,7 +426,7 @@ def step_use_and_complete_strategize_skip(context: Context) -> None:
|
||||
context.skip_svc.start_strategize(plan_id)
|
||||
p = context.skip_svc.get_plan(plan_id)
|
||||
p.processing_state = ProcessingState.PROCESSING
|
||||
context.skip_svc._commit_plan(p)
|
||||
context.skip_svc.commit_plan(p)
|
||||
context.skip_svc.complete_strategize(plan_id)
|
||||
context.skip_plan = context.skip_svc.get_plan(plan_id)
|
||||
|
||||
|
||||
@@ -86,7 +86,7 @@ def _create_plan_in_phase(
|
||||
plan = svc.use_action(action_name)
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
plan.phase = phase
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
return plan.identity.plan_id
|
||||
|
||||
|
||||
|
||||
@@ -61,7 +61,7 @@ def step_mock_errored_plan(context: object, exc_type: str) -> None:
|
||||
}
|
||||
)
|
||||
|
||||
mock_service._commit_plan.side_effect = fake_commit
|
||||
mock_service.commit_plan.side_effect = fake_commit
|
||||
|
||||
mock_executor = MagicMock()
|
||||
|
||||
@@ -203,7 +203,7 @@ def step_invoke_execute(context: object) -> None:
|
||||
@then("the plan should have been reset to execute/queued for eer")
|
||||
def step_check_reset(context: object) -> None:
|
||||
committed = context.eer_committed_plans
|
||||
assert len(committed) >= 1, "Expected at least one _commit_plan call"
|
||||
assert len(committed) >= 1, "Expected at least one commit_plan call"
|
||||
first = committed[0]
|
||||
assert first["phase"] == PlanPhase.EXECUTE
|
||||
assert first["processing_state"] == ProcessingState.QUEUED
|
||||
|
||||
@@ -12,6 +12,7 @@ Exercises all uncovered lines in
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
@@ -937,3 +938,112 @@ def step_epcov_assembled_has_data(context: Context) -> None:
|
||||
assert result.budget_used >= 0.0, "Expected non-negative budget_used"
|
||||
assert result.context_hash, "Expected non-empty context_hash"
|
||||
assert result.strategies_used, "Expected non-empty strategies_used"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _resolve_hot_max_tokens: reads hot_max_tokens from context_policy_json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_assembler_with_policy_json(
|
||||
policy_json: str | None,
|
||||
) -> ACMSExecutePhaseContextAssembler:
|
||||
"""Build an assembler whose DB session returns a row with *policy_json*."""
|
||||
pr = _make_pipeline_result()
|
||||
pr.fragments = ()
|
||||
pr.total_tokens = 5
|
||||
pr.budget_used = 0.1
|
||||
pr.strategies_used = ("relevance",)
|
||||
pr.context_hash = "hash-policy"
|
||||
pr.preamble = None
|
||||
pr.provenance_map = {}
|
||||
|
||||
mock_pipeline = MagicMock()
|
||||
mock_pipeline.assemble.return_value = pr
|
||||
|
||||
tier_service = MagicMock()
|
||||
# Return a minimal passing fragment so assemble() does not short-circuit
|
||||
frag = _make_tiered_fragment(
|
||||
fragment_id="frag-policy",
|
||||
content="content",
|
||||
token_count=5,
|
||||
metadata={"path": "src/good.py"},
|
||||
)
|
||||
tier_service.get_scoped_view.return_value = [frag]
|
||||
|
||||
# Build a mock row with context_policy_json
|
||||
mock_row = MagicMock()
|
||||
mock_row.context_policy_json = policy_json
|
||||
|
||||
# Mock the session so _resolve_hot_max_tokens can query it
|
||||
mock_session = MagicMock()
|
||||
mock_session.query.return_value.filter_by.return_value.first.return_value = mock_row
|
||||
|
||||
repo = MagicMock()
|
||||
repo._session.return_value = mock_session
|
||||
repo.get_context_policy.return_value = ProjectContextPolicy()
|
||||
|
||||
return ACMSExecutePhaseContextAssembler(
|
||||
context_tier_service=tier_service,
|
||||
project_repository=repo,
|
||||
acms_pipeline=mock_pipeline,
|
||||
hot_max_tokens=4096,
|
||||
)
|
||||
|
||||
|
||||
@given("epcov an assembler with context_policy_json hot_max_tokens 32000")
|
||||
def step_epcov_assembler_policy_json_32k(context: Context) -> None:
|
||||
"""Assembler backed by a DB row with hot_max_tokens=32000 in policy JSON.
|
||||
|
||||
Mirrors the real storage format written by
|
||||
``agents project context set --hot-max-tokens 32000``:
|
||||
``{"acms_config": {"hot_max_tokens": 32000, ...}, ...}``.
|
||||
"""
|
||||
policy_json = json.dumps({"acms_config": {"hot_max_tokens": 32000}})
|
||||
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
|
||||
|
||||
|
||||
@given("epcov an assembler with no hot_max_tokens in context_policy_json")
|
||||
def step_epcov_assembler_policy_json_no_override(context: Context) -> None:
|
||||
"""Assembler backed by a DB row with no hot_max_tokens — global fallback."""
|
||||
policy_json = json.dumps({"acms_config": {"other_setting": "value"}})
|
||||
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
|
||||
|
||||
|
||||
@then("epcov the pipeline received budget max_tokens of 32000")
|
||||
def step_epcov_pipeline_budget_32k(context: Context) -> None:
|
||||
"""Verify the pipeline was called with max_tokens=32000 in the budget."""
|
||||
if context.epcov_error is not None:
|
||||
raise AssertionError(f"Unexpected error: {context.epcov_error}")
|
||||
pipeline = context.epcov_assembler._pipeline
|
||||
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
|
||||
call_args = pipeline.assemble.call_args
|
||||
budget = call_args.kwargs.get("budget") or (
|
||||
call_args[1].get("budget") if call_args[1] else None
|
||||
)
|
||||
if budget is None and call_args[0]:
|
||||
budget = call_args[0][2] if len(call_args[0]) > 2 else None
|
||||
assert budget is not None, "Could not find budget in pipeline.assemble() call"
|
||||
assert budget.max_tokens == 32000, (
|
||||
f"Expected budget.max_tokens=32000 (from context_policy_json), "
|
||||
f"got {budget.max_tokens}"
|
||||
)
|
||||
|
||||
|
||||
@then("epcov the pipeline received budget max_tokens of 4096")
|
||||
def step_epcov_pipeline_budget_global(context: Context) -> None:
|
||||
"""Verify the pipeline fell back to the global hot_max_tokens=4096."""
|
||||
if context.epcov_error is not None:
|
||||
raise AssertionError(f"Unexpected error: {context.epcov_error}")
|
||||
pipeline = context.epcov_assembler._pipeline
|
||||
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
|
||||
call_args = pipeline.assemble.call_args
|
||||
budget = call_args.kwargs.get("budget") or (
|
||||
call_args[1].get("budget") if call_args[1] else None
|
||||
)
|
||||
if budget is None and call_args[0]:
|
||||
budget = call_args[0][2] if len(call_args[0]) > 2 else None
|
||||
assert budget is not None, "Could not find budget in pipeline.assemble() call"
|
||||
assert budget.max_tokens == 4096, (
|
||||
f"Expected budget.max_tokens=4096 (global fallback), got {budget.max_tokens}"
|
||||
)
|
||||
|
||||
@@ -83,7 +83,7 @@ def _eed_make_lifecycle(plan: MagicMock) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
@@ -152,7 +152,7 @@ def step_eed_run_execute(context: Context) -> None:
|
||||
"""Run execute and capture the result."""
|
||||
context.eed_result = context.eed_executor.run_execute(context.eed_plan_id)
|
||||
# Capture the error_details that were committed
|
||||
commit_calls = context.eed_lifecycle._commit_plan.call_args_list
|
||||
commit_calls = context.eed_lifecycle.commit_plan.call_args_list
|
||||
if commit_calls:
|
||||
last_plan = commit_calls[-1][0][0]
|
||||
context.eed_committed_error_details = last_plan.error_details
|
||||
@@ -169,7 +169,7 @@ def step_eed_run_execute_failure(context: Context) -> None:
|
||||
except Exception:
|
||||
context.eed_raised = True
|
||||
# Capture the error_details that were committed
|
||||
commit_calls = context.eed_lifecycle._commit_plan.call_args_list
|
||||
commit_calls = context.eed_lifecycle.commit_plan.call_args_list
|
||||
if commit_calls:
|
||||
last_plan = commit_calls[-1][0][0]
|
||||
context.eed_committed_error_details = last_plan.error_details
|
||||
|
||||
@@ -112,21 +112,21 @@ def _create_plan_at_phase(
|
||||
elif phase == PlanPhase.STRATEGIZE and state == ProcessingState.COMPLETE:
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
elif phase == PlanPhase.STRATEGIZE and state == ProcessingState.PROCESSING:
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
elif phase == PlanPhase.EXECUTE and state == ProcessingState.COMPLETE:
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
# Transition to Execute
|
||||
plan.processing_state = ProcessingState.QUEUED
|
||||
plan.phase = PlanPhase.EXECUTE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
# Advance to complete
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
|
||||
context.plan = svc.get_plan(plan_id)
|
||||
context.plan_id = plan_id
|
||||
|
||||
@@ -92,6 +92,7 @@ def _make_mock_lifecycle(strategy_actor=None, execution_actor=None):
|
||||
lifecycle = SimpleNamespace(
|
||||
get_plan=MagicMock(return_value=plan),
|
||||
get_action=MagicMock(return_value=action),
|
||||
resolve_actor_provider_model=MagicMock(return_value=None),
|
||||
)
|
||||
return lifecycle
|
||||
|
||||
@@ -130,8 +131,8 @@ class _StubContextAssembler:
|
||||
LLM_NUMBERED_RESPONSE = "1. Create the module\n2. Write unit tests\n3. Update docs"
|
||||
|
||||
LLM_FILE_BLOCKS_RESPONSE = (
|
||||
"FILE: src/main.py\n```python\nprint('hello')\n```\n\n"
|
||||
"FILE: tests/test_main.py\n```python\ndef test_main(): pass\n```\n"
|
||||
"FILE: src/main.py\n<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\nprint('hello')\n<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
"FILE: tests/test_main.py\n<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\ndef test_main(): pass\n<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n"
|
||||
)
|
||||
|
||||
|
||||
@@ -681,18 +682,41 @@ def step_execute_prompt_not_contains(context, needle):
|
||||
|
||||
@when("I parse file blocks from LLM output with two files")
|
||||
def step_parse_file_blocks_two_files(context):
|
||||
context.parsed_entries = LLMExecuteActor._parse_file_blocks(
|
||||
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
LLM_FILE_BLOCKS_RESPONSE, "PLAN_TEST"
|
||||
)
|
||||
|
||||
|
||||
@when("I parse file blocks from empty LLM output")
|
||||
def step_parse_file_blocks_empty(context):
|
||||
context.parsed_entries = LLMExecuteActor._parse_file_blocks(
|
||||
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
"No files here.", "PLAN_EMPTY"
|
||||
)
|
||||
|
||||
|
||||
@when("I parse file blocks using the CAFS short delimiter format")
|
||||
def step_parse_file_blocks_cafs(context):
|
||||
"""M7: exercise the <CAFS>/<CAFE> short-form delimiter pattern."""
|
||||
llm_output = "FILE: src/cafs_example.py\n<CAFS>\nprint('cafs format')\n</CAFE>\n"
|
||||
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
llm_output, "PLAN_CAFS"
|
||||
)
|
||||
|
||||
|
||||
@when("I parse file blocks using the new arrow delimiter format")
|
||||
def step_parse_file_blocks_arrow(context):
|
||||
"""M7: exercise the >>>>>>>> CLEVERAGENTS_FILE_START >>>>>>>> format."""
|
||||
llm_output = (
|
||||
"FILE: src/arrow_example.py\n"
|
||||
">>>>>>>> CLEVERAGENTS_FILE_START >>>>>>>>\n"
|
||||
"print('arrow format')\n"
|
||||
">>>>>>>> CLEVERAGENTS_FILE_END >>>>>>>>\n"
|
||||
)
|
||||
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
llm_output, "PLAN_ARROW"
|
||||
)
|
||||
|
||||
|
||||
@then("I should get {count:d} changeset entries")
|
||||
def step_verify_changeset_entry_count(context, count):
|
||||
assert len(context.parsed_entries) == count, (
|
||||
@@ -707,6 +731,16 @@ def step_verify_changeset_entry_path(context, idx, expected):
|
||||
)
|
||||
|
||||
|
||||
@then('I should get 1 changeset entry with path "{expected}"')
|
||||
def step_verify_single_changeset_entry_path(context, expected):
|
||||
assert len(context.parsed_entries) == 1, (
|
||||
f"Expected exactly 1 entry, got {len(context.parsed_entries)}"
|
||||
)
|
||||
assert context.parsed_entries[0].path == expected, (
|
||||
f"Expected path '{expected}', got '{context.parsed_entries[0].path}'"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# LLMExecuteActor._write_to_sandbox
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -725,10 +759,8 @@ def step_create_temp_sandbox(context):
|
||||
|
||||
@when("I write generated files to the sandbox")
|
||||
def step_write_files_to_sandbox(context):
|
||||
entries = LLMExecuteActor._parse_file_blocks(LLM_FILE_BLOCKS_RESPONSE, "PLAN_WR")
|
||||
LLMExecuteActor._write_to_sandbox(
|
||||
entries, context.sandbox_dir, LLM_FILE_BLOCKS_RESPONSE
|
||||
)
|
||||
_, blocks = LLMExecuteActor._parse_file_blocks(LLM_FILE_BLOCKS_RESPONSE, "PLAN_WR")
|
||||
LLMExecuteActor._write_to_sandbox(blocks, context.sandbox_dir)
|
||||
|
||||
|
||||
@then("the sandbox should contain the generated files")
|
||||
@@ -762,13 +794,20 @@ def step_create_readonly_sandbox(context):
|
||||
|
||||
@when("I write generated files to the read-only sandbox")
|
||||
def step_write_to_readonly_sandbox(context):
|
||||
# FILE path puts the file inside "src/" which is read-only
|
||||
llm_output = "FILE: src/fail.py\n```python\nprint('fail')\n```\n"
|
||||
entries = LLMExecuteActor._parse_file_blocks(llm_output, "PLAN_RO")
|
||||
# FILE path puts the file inside "src/" which is read-only.
|
||||
# Use the legacy CLEVERAGENTS sentinel delimiter so _parse_file_blocks
|
||||
# actually produces file blocks (backtick format was removed in #10878 fix).
|
||||
llm_output = (
|
||||
"FILE: src/fail.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"print('fail')\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n"
|
||||
)
|
||||
_, blocks = LLMExecuteActor._parse_file_blocks(llm_output, "PLAN_RO")
|
||||
# Should not raise - OSError is caught and logged inside _write_to_sandbox
|
||||
context.write_exception = None
|
||||
try:
|
||||
LLMExecuteActor._write_to_sandbox(entries, context.sandbox_dir, llm_output)
|
||||
LLMExecuteActor._write_to_sandbox(blocks, context.sandbox_dir)
|
||||
except Exception as exc:
|
||||
context.write_exception = exc
|
||||
|
||||
|
||||
@@ -0,0 +1,371 @@
|
||||
"""Step definitions for llm_delimiter_regression.feature (#10878).
|
||||
|
||||
Re-proves that the CLEVERAGENTS_FILE_START/END sentinel scheme fixes
|
||||
the triple-backtick delimiter collision bug from #10878.
|
||||
|
||||
Two in-line "old" parsers are provided to demonstrate the original bug:
|
||||
* _old_backtick_nongreedy -- non-greedy ```.``` pattern; truncates content
|
||||
at the first triple-backtick *inside* a file body but still finds all
|
||||
FILE blocks (each with truncated content).
|
||||
* _old_backtick_greedy -- greedy ```.``` pattern; swallows subsequent
|
||||
FILE blocks inside the first match's "content", returning far fewer
|
||||
entries than expected.
|
||||
|
||||
The current ``LLMExecuteActor._parse_file_blocks`` uses
|
||||
``<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>`` / ``<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>``
|
||||
(and the newer ``<CAFS>``/``</CAFE>``) markers, which never collide with
|
||||
triple-backtick Markdown fences.
|
||||
|
||||
Forgejo: #10878
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
from behave import given, step, then, when # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.application.services.llm_actors import LLMExecuteActor
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Old (buggy) backtick-based parsers used to reproduce the pre-fix behaviour
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_LEGACY_START = "<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>"
|
||||
_LEGACY_END = "<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>"
|
||||
|
||||
|
||||
def _old_backtick_nongreedy(llm_output: str) -> list[str]:
|
||||
"""Non-greedy old parser.
|
||||
|
||||
Finds the correct number of FILE blocks but truncates their content at
|
||||
the first triple-backtick that appears inside a file body.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r"FILE:\s*(.+?)\s*\n```\n(.*?)\n```",
|
||||
re.DOTALL,
|
||||
)
|
||||
return [m.group(1).strip() for m in pattern.finditer(llm_output)]
|
||||
|
||||
|
||||
def _old_backtick_greedy(llm_output: str) -> list[str]:
|
||||
"""Greedy old parser.
|
||||
|
||||
The greedy ``(.*)`` causes the *first* match to consume all remaining
|
||||
content up to the very last triple-backtick line in the document,
|
||||
swallowing every subsequent FILE block inside the captured "content".
|
||||
Only one (or very few) entries are returned for an input that contains
|
||||
many FILE blocks.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r"FILE:\s*(.+?)\s*\n```\n(.*)\n```",
|
||||
re.DOTALL,
|
||||
)
|
||||
return [m.group(1).strip() for m in pattern.finditer(llm_output)]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helper builders
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_legacy_block(path: str, content: str) -> str:
|
||||
"""FILE block using the CLEVERAGENTS_FILE_START/END markers."""
|
||||
return f"FILE: {path}\n{_LEGACY_START}\n{content}\n{_LEGACY_END}\n\n"
|
||||
|
||||
|
||||
def _make_old_backtick_block(path: str, content: str) -> str:
|
||||
"""FILE block using the old triple-backtick delimiters."""
|
||||
return f"FILE: {path}\n```\n{content}\n```\n\n"
|
||||
|
||||
|
||||
def _fence_content(label: str) -> str:
|
||||
"""File body that embeds a triple-backtick Python code fence."""
|
||||
return (
|
||||
f"# {label}\n"
|
||||
"```python\n"
|
||||
f"def {label.replace(' ', '_')}(): pass\n"
|
||||
"```\n"
|
||||
f"# end of {label}\n"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given — context setup
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a string containing ````")
|
||||
def step_context_with_backtick_string(context): # type: ignore[no-untyped-def]
|
||||
context.backtick_context = "````"
|
||||
|
||||
|
||||
@given("a string ````")
|
||||
def step_short_backtick_ctx(context): # type: ignore[no-untyped-def]
|
||||
context.backtick_context = "````"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQXXXXX"')
|
||||
def step_plan_id_xxxxx(context): # type: ignore[no-untyped-def]
|
||||
context.delimiter_plan_id = "01HQXXXXX"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HYYYYYY"')
|
||||
def step_plan_id_yyyyyy(context): # type: ignore[no-untyped-def]
|
||||
context.delimiter_plan_id = "01HYYYYYY"
|
||||
|
||||
|
||||
# --- Input-creation steps ---
|
||||
|
||||
|
||||
@step("I create file with LLM output containing code fences in body")
|
||||
def step_create_old_2files_with_fences(context): # type: ignore[no-untyped-def]
|
||||
"""Two FILE blocks in old backtick format, each with an embedded code fence.
|
||||
|
||||
The non-greedy old parser will find both files but truncate their content
|
||||
at the first triple-backtick in each body (the opening fence).
|
||||
"""
|
||||
context.delim_llm_output = _make_old_backtick_block(
|
||||
"src/main.py", _fence_content("main")
|
||||
) + _make_old_backtick_block("src/util.py", _fence_content("util"))
|
||||
|
||||
|
||||
@step(
|
||||
"I create file with LLM output using unique delimiters and code fences embedded in body"
|
||||
)
|
||||
def step_create_new_6files_with_fences(context): # type: ignore[no-untyped-def]
|
||||
"""Six FILE blocks using CLEVERAGENTS markers with embedded code fences."""
|
||||
context.delim_llm_output = "".join(
|
||||
_make_legacy_block(f"src/mod_{i}.py", _fence_content(f"mod_{i}"))
|
||||
for i in range(1, 7)
|
||||
)
|
||||
|
||||
|
||||
@step("I create LLM output where file content contains embedded markdown code blocks")
|
||||
def step_create_old_4files_greedy(context): # type: ignore[no-untyped-def]
|
||||
"""Four FILE blocks in old backtick format, each with embedded code fences.
|
||||
|
||||
The *greedy* old parser will match only the first FILE block and
|
||||
consume the remaining three as part of the first block's "content".
|
||||
"""
|
||||
context.delim_llm_output = (
|
||||
_make_old_backtick_block("src/a.py", _fence_content("a"))
|
||||
+ _make_old_backtick_block("src/b.py", _fence_content("b"))
|
||||
+ _make_old_backtick_block("src/c.py", _fence_content("c"))
|
||||
+ _make_old_backtick_block("src/d.py", _fence_content("d"))
|
||||
)
|
||||
|
||||
|
||||
@step(
|
||||
"I create LLM output using unique sentinels with embedded ```python code blocks in body"
|
||||
)
|
||||
def step_create_new_4files_with_fences(context): # type: ignore[no-untyped-def]
|
||||
"""Four FILE blocks using CLEVERAGENTS markers with embedded code fences."""
|
||||
context.delim_llm_output = (
|
||||
_make_legacy_block("src/a.py", _fence_content("a"))
|
||||
+ _make_legacy_block("src/b.py", _fence_content("b"))
|
||||
+ _make_legacy_block("src/c.py", _fence_content("c"))
|
||||
+ _make_legacy_block("src/d.py", _fence_content("d"))
|
||||
)
|
||||
|
||||
|
||||
@step(
|
||||
"I create full regression file block output that includes report section plus"
|
||||
" multiple files each with inline ```python examples"
|
||||
)
|
||||
def step_create_full_regression_output(context): # type: ignore[no-untyped-def]
|
||||
"""Architecture-report header followed by five FILE blocks with CLEVERAGENTS markers."""
|
||||
prose = "## Architecture Report\n\nThis section describes key modules.\n\n"
|
||||
files = [
|
||||
_make_legacy_block(f"src/mod_{i}.py", _fence_content(f"mod_{i}"))
|
||||
for i in range(1, 6)
|
||||
]
|
||||
context.delim_llm_output = prose + "".join(files)
|
||||
context.delim_expected_count = 5
|
||||
|
||||
|
||||
@step(
|
||||
"LLM output containing two FILE blocks each with embedded"
|
||||
" ```python sections in the content"
|
||||
)
|
||||
def step_create_new_2files_with_python_fences(context): # type: ignore[no-untyped-def]
|
||||
"""Two FILE blocks with CLEVERAGENTS markers and embedded Python code fences."""
|
||||
context.delim_llm_output = _make_legacy_block(
|
||||
"mod_a.py", _fence_content("module_a")
|
||||
) + _make_legacy_block("src/mod_b.py", _fence_content("module_b"))
|
||||
|
||||
|
||||
@step(
|
||||
"an llm_output where the file body ends on a line that is exactly ``` after content"
|
||||
)
|
||||
def step_create_trailing_backtick_body(context): # type: ignore[no-untyped-def]
|
||||
"""Single FILE block whose last body line is exactly three backticks."""
|
||||
context.delim_llm_output = (
|
||||
f"FILE: module.py\n{_LEGACY_START}\nline one\nline two\n```\n{_LEGACY_END}\n\n"
|
||||
)
|
||||
|
||||
|
||||
@step(
|
||||
"LLM input with backtick-delimited FILE blocks containing code without"
|
||||
" triple-backticks inside body"
|
||||
)
|
||||
def step_create_single_simple_block(context): # type: ignore[no-untyped-def]
|
||||
"""Single FILE block with CLEVERAGENTS markers and plain content (no code fences).
|
||||
|
||||
This tests backward-compatible parsing of simple single-file blocks.
|
||||
"""
|
||||
context.delim_llm_output = _make_legacy_block(
|
||||
"src/simple.py",
|
||||
"# Simple module\ndef run(): return 0\n",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When — command-like no-ops (side-effect-free shell invocations)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I run the command \"echo 'backtick-test'\"")
|
||||
def step_run_echo_backtick_test(context): # type: ignore[no-untyped-def]
|
||||
subprocess.run(["echo", "backtick-test"], capture_output=True, check=False)
|
||||
|
||||
|
||||
@when("I run the command \"echo 'old-parser-test'\"")
|
||||
def step_run_echo_old_parser_test(context): # type: ignore[no-untyped-def]
|
||||
subprocess.run(["echo", "old-parser-test"], capture_output=True, check=False)
|
||||
|
||||
|
||||
@when("I run the command \"echo 'new-parser-test'\"")
|
||||
def step_run_echo_new_parser_test(context): # type: ignore[no-untyped-def]
|
||||
subprocess.run(["echo", "new-parser-test"], capture_output=True, check=False)
|
||||
|
||||
|
||||
@when("I run the command \"echo 'full-regression-test'\"")
|
||||
def step_run_echo_full_regression(context): # type: ignore[no-untyped-def]
|
||||
subprocess.run(["echo", "full-regression-test"], capture_output=True, check=False)
|
||||
|
||||
|
||||
@when("I run the command \"echo 'backtick-sanitise'\"")
|
||||
def step_run_echo_backtick_sanitise(context): # type: ignore[no-untyped-def]
|
||||
subprocess.run(["echo", "backtick-sanitise"], capture_output=True, check=False)
|
||||
|
||||
|
||||
# --- Parsing steps ---
|
||||
|
||||
|
||||
@when("I parse the LLM output as if it were old backtick-delimited file blocks")
|
||||
def step_parse_old_nongreedy(context): # type: ignore[no-untyped-def]
|
||||
context.delim_entries = _old_backtick_nongreedy(context.delim_llm_output)
|
||||
|
||||
|
||||
@when(
|
||||
"I parse the LLM output with new"
|
||||
" CLEVERAGENTS_FILE_START/CLEVERAGENTS_FILE_END delimiters"
|
||||
)
|
||||
def step_parse_new_cleveragents(context): # type: ignore[no-untyped-def]
|
||||
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
|
||||
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
context.delim_llm_output, plan_id
|
||||
)
|
||||
|
||||
|
||||
@when("I parse it with the old backtick-style delimiter pattern")
|
||||
def step_parse_old_greedy(context): # type: ignore[no-untyped-def]
|
||||
context.delim_entries = _old_backtick_greedy(context.delim_llm_output)
|
||||
|
||||
|
||||
@when("I parse it with CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters")
|
||||
def step_parse_new_4files(context): # type: ignore[no-untyped-def]
|
||||
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
|
||||
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
context.delim_llm_output, plan_id
|
||||
)
|
||||
|
||||
|
||||
@when("I parse it with new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters")
|
||||
def step_parse_new_full_regression(context): # type: ignore[no-untyped-def]
|
||||
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
|
||||
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
context.delim_llm_output, plan_id
|
||||
)
|
||||
|
||||
|
||||
@when("I call _parse_file_blocks on that LLM output")
|
||||
def step_call_parse_file_blocks_llm(context): # type: ignore[no-untyped-def]
|
||||
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
|
||||
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
context.delim_llm_output, plan_id
|
||||
)
|
||||
|
||||
|
||||
@when("I call _parse_file_blocks on that output")
|
||||
def step_call_parse_file_blocks(context): # type: ignore[no-untyped-def]
|
||||
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
|
||||
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
|
||||
context.delim_llm_output, plan_id
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then — assertions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the parser should return only 2 entries truncated at first triple-backtick")
|
||||
def step_assert_2_truncated_entries(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 2, (
|
||||
f"Expected 2 entries (truncated) from old backtick parser, got {len(entries)}: "
|
||||
f"{entries}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parser should return 6 entries not truncated at any triple-backtick")
|
||||
def step_assert_6_full_entries(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 6, (
|
||||
f"Expected 6 complete entries from new CLEVERAGENTS parser, got {len(entries)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parser should return only 1 entry instead of the expected 4")
|
||||
def step_assert_1_instead_of_4(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 1, (
|
||||
f"Expected greedy old parser to return 1 (swallowing other files), "
|
||||
f"got {len(entries)}: {entries}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parser should return a result count of 4 entries")
|
||||
def step_assert_4_entries(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 4, (
|
||||
f"Expected 4 entries from new CLEVERAGENTS parser, got {len(entries)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parser should return the full entry count of all embedded blocks")
|
||||
def step_assert_full_entry_count(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
expected = getattr(context, "delim_expected_count", 5)
|
||||
assert len(entries) == expected, (
|
||||
f"Expected {expected} entries (full regression), got {len(entries)}"
|
||||
)
|
||||
|
||||
|
||||
@then("it should return 2 changeset entries not 1")
|
||||
def step_assert_2_not_1(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 2, (
|
||||
f"Expected 2 entries (new parser handles embedded fences, not 1): "
|
||||
f"got {len(entries)}"
|
||||
)
|
||||
|
||||
|
||||
@then("it should return 1 entry")
|
||||
def step_assert_1_entry(context): # type: ignore[no-untyped-def]
|
||||
entries = context.delim_entries
|
||||
assert len(entries) == 1, f"Expected exactly 1 entry, got {len(entries)}"
|
||||
@@ -0,0 +1,236 @@
|
||||
"""Step definitions for llm_file_parsing_regression.feature (#10878 TDD regression).
|
||||
|
||||
Demonstrates that LLMExecuteActor._parse_file_blocks correctly extracts all FILE
|
||||
blocks when their body contains embedded triple-backtick markdown code fences,
|
||||
proving the new delimiter scheme is not vulnerable to the collision that
|
||||
caused truncated architecture reviews.
|
||||
|
||||
Forgejo: #10878
|
||||
|
||||
Also exercises related edge cases: trailing backtick line, sentinel text in
|
||||
body prose without full markers, and git merge-conflict style markers mixed
|
||||
with sentinel delimiters.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
from behave import given, then, when # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.application.services.llm_actors import LLMExecuteActor
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers - build LLM output strings matching each scenario's description.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _llm_with_fences(n_files, fence_in_each=True, extra_prose="") -> str:
|
||||
"""Return an ``FILE: ... / delimiter`` block string.
|
||||
|
||||
When *fence_in_each* is True the body of every file includes a
|
||||
`````python ```` `code fence` - exactly the pattern that caused the original bug.
|
||||
"""
|
||||
output = extra_prose
|
||||
for i in range(1, n_files + 1):
|
||||
path = f"mod_{i}.py" if i == 1 else f"src/mod_{i}.py"
|
||||
fence_block = "\n```python\nprint('hello')\n```\n" if fence_in_each else ""
|
||||
output += (
|
||||
f"FILE: {path}\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
f"# File {i}\n{fence_block}"
|
||||
"actual code here\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
)
|
||||
return output
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response with two FILE blocks, each having ```python sections in the body"
|
||||
)
|
||||
def step_two_embedded_fences(context): # type: ignore[no-untyped-def]
|
||||
context.input_lfm = _llm_with_fences(2, fence_in_each=True)
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response mimicking an architecture review with report prose and three FILE blocks each containing markdown code example triple-backticks in their body",
|
||||
)
|
||||
def step_arch_review_full(context): # type: ignore[no-untyped-def]
|
||||
context.input_lfm = _llm_with_fences(
|
||||
3, fence_in_each=True, extra_prose="## Module Graph\nSome report prose above.\n"
|
||||
)
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response where a single FILE block body ends on a line that is exactly ``` after code content"
|
||||
)
|
||||
def step_trailing_backtick(context): # type: ignore[no-untyped-def]
|
||||
path = "module.py"
|
||||
context.input_lfm = (
|
||||
f"FILE: {path}\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"line one\nline two\n```\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
)
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response where a FILE block body contains the word 'CLEVERAGENTS_FILE_END' referenced in prose within content and another file follows",
|
||||
)
|
||||
def step_body_mentions_sentinel_text(context): # type: ignore[no-untyped-def]
|
||||
context.input_lfm = (
|
||||
"FILE: mod_a.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"some text with ```python blocks\n"
|
||||
"# The body mentions CLEVERAGENTS_FILE_END as prose (no markers)\n"
|
||||
"rest of content here\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
"FILE: mod_b.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"# second file block\n"
|
||||
"continues after sentinel-text mention\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
)
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response where FILE block bodies include text like '>>>>>>> some_branch' and '<<<<<<< base' but the actual delimiters are CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END markers",
|
||||
)
|
||||
def step_mixed_git_markers(context): # type: ignore[no-untyped-def]
|
||||
context.input_lfm = (
|
||||
"Prose with <<<<<< some_branch mentioned here\n"
|
||||
"FILE: mod_a.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"```python\ndef f1(): pass # <<< and >>> in content are fine\n```\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
"FILE: mod_b.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
">>>>>>> base branch also mentioned here\n"
|
||||
"more code below\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
)
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQREGTEST"')
|
||||
def step_mock_plan_id_regtest(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQREGTEST"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQARCHTEST"')
|
||||
def step_mock_plan_id_archtest(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQARCHTEST"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQTRAILT"')
|
||||
def step_mock_plan_id_trail(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQTRAILT"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQSENTINEL"')
|
||||
def step_mock_plan_id_sentinel(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQSENTINEL"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQGITTEST"')
|
||||
def step_mock_plan_id_gittest(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQGITTEST"
|
||||
|
||||
|
||||
@given('a mock plan_id "01HQESCTEST"')
|
||||
def step_mock_plan_id_esctest(context): # type: ignore[no-untyped-def]
|
||||
context.lfm_plan_id = "01HQESCTEST"
|
||||
|
||||
|
||||
@given(
|
||||
"an LLM response with a single FILE block using legacy markers where the body"
|
||||
" contains a backslash-escaped <<<<<<< CLEVERAGENTS_FILE_START >>>>>>>"
|
||||
" sequence that should be preserved as literal text"
|
||||
)
|
||||
def step_single_block_with_escaped_marker(context): # type: ignore[no-untyped-def]
|
||||
"""FILE block whose body mentions the START marker prefixed with a backslash.
|
||||
|
||||
The backslash-escaped occurrence must NOT be treated as a block boundary
|
||||
by the negative-lookbehind regex in _parse_file_blocks.
|
||||
"""
|
||||
context.input_lfm = (
|
||||
"FILE: example.py\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"# This file discusses the delimiter format:\n"
|
||||
"# \\<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
|
||||
"# The above line is escaped and must not break parsing.\n"
|
||||
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when(
|
||||
"I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern"
|
||||
)
|
||||
def step_parse_with_new_delimiters(context): # type: ignore[no-untyped-def]
|
||||
"""Exercise LLMExecuteActor._parse_file_blocks with real input."""
|
||||
plan_id = getattr(context, "lfm_plan_id", "TEST_PLAN") # type: ignore[attr-defined]
|
||||
input_data = context.input_lfm # type: ignore[attr-defined]
|
||||
context.lfm_entries, _ = LLMExecuteActor._parse_file_blocks(input_data, plan_id)
|
||||
|
||||
|
||||
@when("I parse file blocks using the new delimiter pattern")
|
||||
def step_parse_with_new_delimiters_short(context): # type: ignore[no-untyped-def]
|
||||
"""Alias of the CLEVERAGENTS step for shorter scenario descriptions."""
|
||||
plan_id = getattr(context, "lfm_plan_id", "TEST_PLAN") # type: ignore[attr-defined]
|
||||
input_data = context.input_lfm # type: ignore[attr-defined]
|
||||
context.lfm_entries, _ = LLMExecuteActor._parse_file_blocks(input_data, plan_id)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("it should return {count:d} changeset entries")
|
||||
def step_validate_entry_count(context, count): # type: ignore[no-untyped-def]
|
||||
assert len(context.lfm_entries) == count, ( # type: ignore[attr-defined]
|
||||
f"Expected {count} entry/entries, got {len(context.lfm_entries)}: "
|
||||
f"{[e.path for e in context.lfm_entries]}"
|
||||
)
|
||||
|
||||
|
||||
@then("it should return the full entry count of all embedded blocks")
|
||||
def step_validate_full_count_in_body(context): # type: ignore[no-untyped-def]
|
||||
entries = context.lfm_entries # type: ignore[attr-defined]
|
||||
assert len(entries) == 3, (
|
||||
f"Expected 3 entries for full arch-review output, got {len(entries)}: "
|
||||
f"{[e.path for e in entries]}"
|
||||
)
|
||||
|
||||
|
||||
@then("it should return {count:d} changeset entries (second entry has content in body)")
|
||||
def step_validate_entry_count_with_body_note(context, count): # type: ignore[no-untyped-def]
|
||||
assert len(context.lfm_entries) == count, ( # type: ignore[attr-defined]
|
||||
f"Expected {count} entries, got {len(context.lfm_entries)}: "
|
||||
f"{[e.path for e in context.lfm_entries]}"
|
||||
)
|
||||
|
||||
|
||||
@then("changeset entry {idx:d} path should be a file with ```python block inside")
|
||||
def step_validate_path_has_python_fences(index, context): # type: ignore[no-untyped-def]
|
||||
entries = context.lfm_entries # type: ignore[attr-defined]
|
||||
assert len(entries) > index, (
|
||||
f"Expected at least {index + 1} entries, got {len(entries)}"
|
||||
)
|
||||
|
||||
|
||||
@then("it should NOT trip on the closing triple backtick")
|
||||
def step_validate_no_triple_backtick_stop(context): # type: ignore[no-untyped-def]
|
||||
entries = context.lfm_entries # type: ignore[attr-defined]
|
||||
assert len(entries) >= 1, (
|
||||
f"Expected at least one entry when body contains ```; got {len(entries)}"
|
||||
)
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Step definitions for main() error path coverage tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import subprocess
|
||||
import sys
|
||||
from io import StringIO
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import then, when
|
||||
from behave.runner import Context
|
||||
|
||||
PYTHON = sys.executable
|
||||
MAIN_MODULE = "cleveragents.cli.main"
|
||||
|
||||
|
||||
@when("I run CLI with arguments {args}")
|
||||
def step_run_cli(context: Context, args: str) -> None:
|
||||
"""Run ``python -m cleveragents.cli.main <args>`` via subprocess.
|
||||
|
||||
The *args* argument is a Python-like list literal from the Gherkin
|
||||
feature line, e.g. ``["--config-path", "/tmp"]`` or ``["bad_cmd"]``.
|
||||
We evaluate it with ``ast.literal_eval`` to get a real ``list[str]``,
|
||||
then invoke the module via subprocess.run().
|
||||
|
||||
Pattern follows cli_coverage_steps.py:subprocess.run(shlex.split(cmd))
|
||||
"""
|
||||
cmd_args = ast.literal_eval(args) if isinstance(args, str) else []
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[PYTHON, "-m", MAIN_MODULE, *cmd_args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
context.exit_code = result.returncode
|
||||
context.output = result.stdout + result.stderr
|
||||
except subprocess.TimeoutExpired:
|
||||
context.exit_code = -1
|
||||
context.output = "Command timed out"
|
||||
except Exception as exc:
|
||||
context.exit_code = -2
|
||||
context.output = f"Subprocess error: {exc}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# convert_exit_code helpers — unique step text to avoid collisions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('the main exit-code converter is called with the value "{value}"')
|
||||
def step_convert_exit_value(context: Context, value: str) -> None:
|
||||
"""Call convert_exit_code(value — int, string, or special token).
|
||||
|
||||
Special tokens recognised in-process:
|
||||
* "None" → Python None
|
||||
* "abc" → literal string 'abc' (triggers TypeError path)
|
||||
Everything else is cast to int first (to cover the int/str paths).
|
||||
"""
|
||||
from cleveragents.cli.main import convert_exit_code
|
||||
|
||||
if value == "None":
|
||||
parsed = None
|
||||
elif value in ("abc", "def"):
|
||||
parsed = value # raw string → triggers TypeError path inside int()
|
||||
else:
|
||||
try:
|
||||
parsed = int(value)
|
||||
except ValueError:
|
||||
parsed = value # pass through (int(calling_code) will catch)
|
||||
context.exit_code = convert_exit_code(parsed)
|
||||
|
||||
|
||||
@when("I call the main _print_basic_help")
|
||||
def step_print_basic_help(context: Context) -> None:
|
||||
"""Capture output from calling _print_basic_help()."""
|
||||
from cleveragents.cli.main import _print_basic_help
|
||||
|
||||
buf = StringIO()
|
||||
with patch("typer.echo", side_effect=buf.write):
|
||||
_print_basic_help()
|
||||
|
||||
context.help_output = buf.getvalue()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps — unique step texts to avoid collisions with other suites
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the main cli exit code should be {code:d}")
|
||||
def step_main_exit_code(context: Context, code: int) -> None:
|
||||
"""Verify the CLI subprocess exit code."""
|
||||
assert context.exit_code == code, (
|
||||
f"Expected exit code {code}, got {context.exit_code}"
|
||||
)
|
||||
|
||||
|
||||
@then('the main cli output contains "{text}"')
|
||||
def step_main_output_contains(context: Context, text: str) -> None:
|
||||
"""Verify the CLI subprocess output."""
|
||||
assert text in context.output, f"Expected '{text}' in output:\n{context.output}"
|
||||
|
||||
|
||||
@then('the main convert_exit_code result is "{expect}"')
|
||||
def step_main_convert_result(context: Context, expect: str) -> None:
|
||||
"""Verify the convert_exit_code return value matches a string."""
|
||||
actual = str(context.exit_code)
|
||||
assert actual == expect, f"Expected '{expect}', got '{actual}'"
|
||||
|
||||
|
||||
@then("the main _print_basic_help completes ok")
|
||||
def step_main_print_ok(context: Context) -> None:
|
||||
"""No-op success — the When-step finished without raising."""
|
||||
pass
|
||||
@@ -342,7 +342,10 @@ def step_validate_sql_constructor(context: Any) -> None:
|
||||
assert context.sql_history_calls
|
||||
_, kwargs = context.sql_history_calls[0]
|
||||
assert kwargs["session_id"] == "sql-session"
|
||||
assert kwargs["connection_string"] == context.connection_string
|
||||
actual_connection = kwargs.get("connection") or kwargs.get("connection_string")
|
||||
assert actual_connection == context.connection_string, (
|
||||
f"Expected {context.connection_string}, got {actual_connection}"
|
||||
)
|
||||
assert kwargs.get("table_name") == "message_history"
|
||||
|
||||
|
||||
|
||||
@@ -354,7 +354,8 @@ def step_call_apply_abort_timeout(context: object) -> None:
|
||||
def step_create_failing_sandbox(context: object) -> None:
|
||||
d = tempfile.mkdtemp(prefix="mca-flat-")
|
||||
context.add_cleanup(shutil.rmtree, d, True)
|
||||
sandbox = os.path.join(d, ".cleveragents", "sandbox")
|
||||
plan_id = "01TESTFLATFAIL00000000000000"
|
||||
sandbox = os.path.join(d, "plan-output", plan_id)
|
||||
os.makedirs(sandbox)
|
||||
Path(sandbox, "output.py").write_text("# generated\n")
|
||||
# Create a read-only destination directory to cause copy failure
|
||||
@@ -364,7 +365,7 @@ def step_create_failing_sandbox(context: object) -> None:
|
||||
os.chmod(dst_dir, 0o444)
|
||||
context.add_cleanup(os.chmod, dst_dir, 0o755)
|
||||
context.mca_flat_project = d
|
||||
context.mca_plan_id = "01TESTFLATFAIL00000000000000"
|
||||
context.mca_plan_id = plan_id
|
||||
|
||||
|
||||
@when("I call _apply_sandbox_changes with the failing flat copy for mca")
|
||||
|
||||
@@ -100,7 +100,7 @@ def step_plan_strategize_empty_definition(context: Context) -> None:
|
||||
plan_id = _create_plan_in_strategize(context, "placeholder")
|
||||
plan = context.lifecycle_service.get_plan(plan_id)
|
||||
plan.definition_of_done = None
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
|
||||
|
||||
@given("I have a plan with invariants in strategize queued state")
|
||||
@@ -159,7 +159,7 @@ def step_plan_execute_no_decisions(context: Context) -> None:
|
||||
context.plan = context.lifecycle_service.get_plan(context.plan_id)
|
||||
# Ensure no decision_root_id
|
||||
context.plan.decision_root_id = None
|
||||
context.lifecycle_service._commit_plan(context.plan)
|
||||
context.lifecycle_service.commit_plan(context.plan)
|
||||
|
||||
|
||||
@given("the strategize actor is configured to fail")
|
||||
|
||||
@@ -159,7 +159,7 @@ def _build_lifecycle_with_capturing_bus(
|
||||
lifecycle = MagicMock(spec=PlanLifecycleService)
|
||||
lifecycle.event_bus = event_bus
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
lifecycle._cleanup_devcontainers = MagicMock()
|
||||
lifecycle._logger = structlog.get_logger("test.lifecycle")
|
||||
|
||||
@@ -337,7 +337,7 @@ def step_apply_with_validation_gate(context: Context) -> None:
|
||||
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
|
||||
captured_kwargs: dict[str, Any] = {}
|
||||
|
||||
|
||||
@@ -0,0 +1,495 @@
|
||||
"""Step definitions for PlanApplyService rendering coverage tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
if TYPE_CHECKING:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers — build plain data dicts for the render functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_diff_data(
|
||||
mode: str = "revert",
|
||||
original_decision_id: str = "dec-0",
|
||||
new_decision_id: str | None = "dec-1",
|
||||
guidance: str = "",
|
||||
completed_at: bool = False,
|
||||
patched_hint: str | None = None,
|
||||
artifacts_path: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Build a minimal correction-diff data dict for render function tests."""
|
||||
now_iso = datetime.datetime.now(datetime.UTC).isoformat()
|
||||
|
||||
summary: dict[str, Any] = {
|
||||
"correction": "corr-test-001",
|
||||
"original_decision": original_decision_id,
|
||||
"mode": mode,
|
||||
"state": "complete" if completed_at else "executing",
|
||||
"guidance": guidance or "",
|
||||
"created_at": now_iso,
|
||||
"completed_at": now_iso if completed_at else None,
|
||||
}
|
||||
|
||||
reverted_decisions: list[dict[str, Any]] = []
|
||||
added_decisions: list[dict[str, Any]] = []
|
||||
if original_decision_id:
|
||||
reverted_decisions.append(
|
||||
{"decision_id": original_decision_id, "status": "reverted"}
|
||||
)
|
||||
if new_decision_id:
|
||||
added_decisions.append({"decision_id": new_decision_id, "status": "added"})
|
||||
|
||||
comparison = {
|
||||
"reverted_decisions": reverted_decisions,
|
||||
"added_decisions": added_decisions,
|
||||
"decisions_changed": len(reverted_decisions) + len(added_decisions),
|
||||
"decisions_added": len(added_decisions),
|
||||
"decisions_reverted": len(reverted_decisions),
|
||||
}
|
||||
|
||||
if patched_hint is None and artifacts_path is None:
|
||||
patch_preview = [
|
||||
{
|
||||
"note": (
|
||||
"Patch preview is available after correction execution completes. "
|
||||
"Current state: executing"
|
||||
),
|
||||
}
|
||||
]
|
||||
elif artifacts_path:
|
||||
patch_preview = [
|
||||
{
|
||||
"note": "Corrected execution artifacts available",
|
||||
"artifacts_path": artifacts_path,
|
||||
}
|
||||
]
|
||||
else:
|
||||
patch_preview = [{"note": patched_hint}]
|
||||
|
||||
return {
|
||||
"correction_diff": summary,
|
||||
"comparison": comparison,
|
||||
"patch_preview": patch_preview,
|
||||
}
|
||||
|
||||
|
||||
def _import_render_functions() -> tuple:
|
||||
"""Lazy-import the render functions."""
|
||||
from cleveragents.application.services import plan_apply_service as mod
|
||||
|
||||
return (
|
||||
mod._render_correction_diff_plain,
|
||||
mod._render_correction_diff_rich,
|
||||
mod._build_correction_diff_dict,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
'I have a CorrectionAttemptRecord with mode="{mode}", '
|
||||
'original_decision_id="{orig}", new_decision_id="{new}", '
|
||||
'guidance="{guidance}", and archived_artifacts_path="{artifacts}"'
|
||||
)
|
||||
def step_corr_record_with_artifact(
|
||||
context: Context,
|
||||
mode: str,
|
||||
orig: str,
|
||||
new: str,
|
||||
guidance: str,
|
||||
artifacts: str,
|
||||
) -> None:
|
||||
"""Construct a minimal CorrectionAttemptRecord for _build_correction_diff_dict."""
|
||||
from cleveragents.domain.models.core.correction import (
|
||||
CorrectionAttemptRecord,
|
||||
CorrectionAttemptState,
|
||||
CorrectionMode,
|
||||
)
|
||||
|
||||
record = CorrectionAttemptRecord(
|
||||
correction_attempt_id="test-corr-001",
|
||||
plan_id="plan-001",
|
||||
original_decision_id=orig,
|
||||
new_decision_id=new if new != "none" else None,
|
||||
mode=CorrectionMode.REVERT if mode == "revert" else CorrectionMode.APPEND,
|
||||
guidance=guidance or "",
|
||||
archived_artifacts_path=artifacts if artifacts not in ("", "none") else None,
|
||||
state=CorrectionAttemptState.COMPLETE,
|
||||
)
|
||||
context.corr_record: CORRECTIONATTEMPTRECORD = record # noqa: F821
|
||||
|
||||
|
||||
@given(
|
||||
'I have a CorrectionAttemptRecord with mode="{mode}", '
|
||||
'original_decision_id="{orig}", no new_decision, '
|
||||
'guidance="{guidance}", and no archived artifacts'
|
||||
)
|
||||
def step_corr_record_no_artifact(
|
||||
context: Context, mode: str, orig: str, guidance: str
|
||||
) -> None:
|
||||
"""Construct a CorrectionAttemptRecord without new_decision or archived artifacts."""
|
||||
from cleveragents.domain.models.core.correction import (
|
||||
CorrectionAttemptRecord,
|
||||
CorrectionAttemptState,
|
||||
CorrectionMode,
|
||||
)
|
||||
|
||||
record = CorrectionAttemptRecord(
|
||||
correction_attempt_id="test-corr-002",
|
||||
plan_id="plan-002",
|
||||
original_decision_id=orig,
|
||||
new_decision_id=None,
|
||||
mode=CorrectionMode.REVERT if mode == "revert" else CorrectionMode.APPEND,
|
||||
guidance=guidance or "",
|
||||
archived_artifacts_path=None,
|
||||
state=CorrectionAttemptState.PENDING,
|
||||
)
|
||||
context.corr_record: CORRECTIONATTEMPTRECORD = record # noqa: F821
|
||||
|
||||
|
||||
@given(
|
||||
'I have a correction diff dict with mode="{mode}", '
|
||||
'original_decision_id="{orig}", guidance="{guidance}", '
|
||||
'completed_at present, and one added decision "{added}"'
|
||||
)
|
||||
def step_diff_dict_with_artifacts(
|
||||
context: Context, mode: str, orig: str, guidance: str, added: str
|
||||
) -> None:
|
||||
"""Build a plain data dict for render function tests."""
|
||||
# Use the added value as new_decision_id if provided
|
||||
nid = added if added != "none" and added else None
|
||||
data = _make_diff_data(
|
||||
mode=mode,
|
||||
original_decision_id=orig,
|
||||
new_decision_id=nid,
|
||||
guidance=guidance,
|
||||
completed_at=True,
|
||||
)
|
||||
context.diff_data = data
|
||||
|
||||
|
||||
@given(
|
||||
'I have a correction diff dict with mode="{mode}", '
|
||||
'original_decision_id="{orig}", no guidance text, '
|
||||
'and patch preview note about "{hint}"'
|
||||
)
|
||||
def step_diff_dict_no_guidance(
|
||||
context: Context, mode: str, orig: str, hint: str
|
||||
) -> None:
|
||||
"""Build a plain data dict with missing fields."""
|
||||
data = _make_diff_data(
|
||||
mode=mode,
|
||||
original_decision_id=orig,
|
||||
new_decision_id=None,
|
||||
guidance="",
|
||||
completed_at=False,
|
||||
patched_hint=f"Patch preview is available after correction execution completes. "
|
||||
f"Current state: {hint}",
|
||||
)
|
||||
context.diff_data = data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I call _build_correction_diff_dict on the record")
|
||||
def step_build_corr_dict(context: Context) -> None:
|
||||
"""Call the production _build_correction_diff_dict function & save dict output."""
|
||||
_, _, build_func = _import_render_functions()
|
||||
record = context.corr_record # type: ignore[attr-defined]
|
||||
data = build_func(record)
|
||||
context.diff_data = data
|
||||
|
||||
|
||||
@when("I call _render_correction_diff_plain on the dict")
|
||||
def step_render_plain(context: Context) -> None:
|
||||
"""Call plain-text renderer and save output."""
|
||||
render_plain, _, _ = _import_render_functions()
|
||||
context.render_output = render_plain(context.diff_data) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I call _render_correction_diff_rich on the dict")
|
||||
def step_render_rich(context: Context) -> None:
|
||||
"""Call rich-text renderer and save output."""
|
||||
_, render_rich, _ = _import_render_functions()
|
||||
context.render_output = render_rich(context.diff_data) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps — verify structure & content
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the result should contain "{key1}", "{key2}", and "{key3}" keys')
|
||||
def step_result_contains_keys(
|
||||
context: Context, key1: str, key2: str, key3: str
|
||||
) -> None:
|
||||
"""Verify the data dict has the expected top-level keys."""
|
||||
for k in (key1, key2, key3):
|
||||
assert k in context.diff_data, f"Missing key '{k}' in result"
|
||||
|
||||
|
||||
@then(
|
||||
"the comparison should have {n_reverted:d} reverted decision and "
|
||||
"{n_added:d} added decisions"
|
||||
)
|
||||
def step_comparison_count(context: Context, n_reverted: int, n_added: int) -> None:
|
||||
"""Verify the number of reverted/added decisions in comparison."""
|
||||
cmp = context.diff_data["comparison"]
|
||||
actual_reverted = len(cmp["reverted_decisions"])
|
||||
actual_added = len(cmp["added_decisions"])
|
||||
assert actual_reverted == n_reverted, (
|
||||
f"Expected {n_reverted} reverted, got {actual_reverted}"
|
||||
)
|
||||
assert actual_added == n_added, f"Expected {n_added} added, got {actual_added}"
|
||||
|
||||
|
||||
@then('the patch preview should include the artifacts path "{path}"')
|
||||
def step_patch_has_artifacts(context: Context, path: str) -> None:
|
||||
"""Verify patch-preview contains the specified artifacts path."""
|
||||
patch = context.diff_data["patch_preview"][0]
|
||||
assert patch.get("artifacts_path") == path, (
|
||||
f"Expected artifacts_path='{path}', got {patch}"
|
||||
)
|
||||
|
||||
|
||||
@then("the patch preview should indicate correction execution is pending")
|
||||
def step_patch_pending(context: Context) -> None:
|
||||
"""Verify that the patch-preview note signals a pending/complete state."""
|
||||
patch = context.diff_data["patch_preview"][0]
|
||||
note_lower = patch.get("note", "").lower()
|
||||
assert "pending" in note_lower or "complete" in note_lower, (
|
||||
f"Expected pending/complete hint in: {patch['note']}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain "{text}" heading')
|
||||
def step_output_contains_heading(context: Context, text: str) -> None:
|
||||
"""Verify the rendered output contains a given heading string."""
|
||||
assert text in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '{text}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain "{value}" mode value')
|
||||
def step_output_contains_mode(context: Context, value: str) -> None:
|
||||
"""Verify the rendered output contains a given mode string."""
|
||||
assert value in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '{value}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain "{text}" trailer')
|
||||
def step_output_contains_trailer(context: Context, text: str) -> None:
|
||||
"""Verify the rendered output contains an end marker string."""
|
||||
assert text in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '{text}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain the added decision line starting with "{marker} {id}"')
|
||||
def step_output_contains_added_decision(context: Context, marker: str, id: str) -> None:
|
||||
"""Verify rendered output contains an added-decision line."""
|
||||
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
|
||||
stripped = line.strip()
|
||||
if f"+ {id}" in stripped or f"{marker} {id}" in stripped:
|
||||
break
|
||||
else:
|
||||
raise AssertionError(
|
||||
f"Expected added decision '{marker} {id}' in output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should NOT contain "{text}" line')
|
||||
def step_output_not_contains(context: Context, text: str) -> None:
|
||||
"""Verify the rendered output does NOT contain a given text string."""
|
||||
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
|
||||
assert text not in line.rstrip(), f"Unexpected '{text}' found in line: {line}"
|
||||
|
||||
|
||||
@then("the output should contain the pending-execution note text")
|
||||
def step_output_contains_pending_note(context: Context) -> None:
|
||||
"""Verify render output includes the patch-preview hint."""
|
||||
assert "Patch preview is available after" in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected pending-note: {context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the output should contain a red-colored reverted decision ID")
|
||||
def step_output_contains_red_revert(context: Context) -> None:
|
||||
"""Verify rich-rendered output has the [red] markup for reverted decisions."""
|
||||
assert "[red]" in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '[red]' markup: {context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the output should contain a green-colored added decision ID")
|
||||
def step_output_contains_green_added(context: Context) -> None:
|
||||
"""Verify rich-rendered output has the [green] markup for added decisions."""
|
||||
assert "[green]" in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '[green]' markup: {context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain "{marker} OK {marker}" checkmark marker at end')
|
||||
def step_output_contains_marker(context: Context, marker: str) -> None:
|
||||
"""Verify the rich-rendered output includes the [green]✔ OK footer."""
|
||||
assert "OK" in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected 'OK' footer: {context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the output should contain the pending-execution note in dim markup")
|
||||
def step_pending_in_dim(context: Context) -> None:
|
||||
"""Verify rendered text includes '[dim]' color wrapper."""
|
||||
assert "[dim]" in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '[dim]' markup: {context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Comparison-count shortcut — avoids ambiguity with the generic matcher below.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the comparison counts are "{n_rev}" reverted decisions and "{n_add}" new ones')
|
||||
def step_comparison_counts_shorthand(context: Context, n_rev: str, n_add: str) -> None:
|
||||
"""Verify the comparison dict has the expected revert/add counts.
|
||||
|
||||
This step uses literal string values from the Gherkin to disambiguate
|
||||
from the parameterised "should have {n:d} ..." matcher above.
|
||||
"""
|
||||
cmp = context.diff_data["comparison"]
|
||||
actual_rev = len(cmp["reverted_decisions"])
|
||||
actual_add = len(cmp["added_decisions"])
|
||||
assert str(actual_rev) == n_rev, f"Expected {n_rev} reverted, got {actual_rev}"
|
||||
assert str(actual_add) == n_add, f"Expected {n_add} added, got {actual_add}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Singular / variant steps that the feature file uses
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then(
|
||||
"the comparison should have {n_reverted:d} reverted decision"
|
||||
" and {n_added:d} added decision"
|
||||
)
|
||||
def step_comparison_count_singular(
|
||||
context: Context, n_reverted: int, n_added: int
|
||||
) -> None:
|
||||
"""Singular variant of the comparison count step (feature uses singular)."""
|
||||
cmp = context.diff_data["comparison"]
|
||||
actual_reverted = len(cmp["reverted_decisions"])
|
||||
actual_added = len(cmp["added_decisions"])
|
||||
assert actual_reverted == n_reverted, (
|
||||
f"Expected {n_reverted} reverted, got {actual_reverted}"
|
||||
)
|
||||
assert actual_added == n_added, f"Expected {n_added} added, got {actual_added}"
|
||||
|
||||
|
||||
@then("the patch preview should include the artifacts path")
|
||||
def step_patch_has_any_artifacts(context: Context) -> None:
|
||||
"""Verify patch-preview contains an artifacts_path (any value)."""
|
||||
patch = context.diff_data["patch_preview"][0]
|
||||
assert patch.get("artifacts_path"), (
|
||||
f"Expected artifacts_path to be set, got: {patch}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain the mode value "{value}"')
|
||||
def step_output_contains_mode_value(context: Context, value: str) -> None:
|
||||
"""Verify rendered output contains the given mode value string."""
|
||||
assert value in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected mode value '{value}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should contain "{text}" timestamp')
|
||||
def step_output_contains_timestamp(context: Context, text: str) -> None:
|
||||
"""Verify rendered output contains a given timestamp label string."""
|
||||
assert text in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected '{text}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the output should NOT contain a "{text}" line')
|
||||
def step_output_not_contains_a_line(context: Context, text: str) -> None:
|
||||
"""Verify rendered output does NOT contain a given text (with 'a' article)."""
|
||||
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
|
||||
assert text not in line.rstrip(), f"Unexpected '{text}' found in line: {line}"
|
||||
|
||||
|
||||
@then('the output should contain "{text}" mode color markup')
|
||||
def step_output_contains_mode_markup(context: Context, text: str) -> None:
|
||||
"""Verify rendered output contains the given rich-markup mode string."""
|
||||
assert text in context.render_output, ( # type: ignore[attr-defined]
|
||||
f"Expected mode markup '{text}' in render output:\n{context.render_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the output should contain the rich green checkmark marker at end")
|
||||
def step_output_contains_checkmark(context: Context) -> None:
|
||||
"""Verify rich-rendered output includes the green checkmark footer."""
|
||||
output = context.render_output # type: ignore[attr-defined]
|
||||
assert "[green]" in output or "✔" in output or "OK" in output, (
|
||||
f"Expected green checkmark / OK footer: {output}"
|
||||
)
|
||||
|
||||
|
||||
@given(
|
||||
'I have a correction diff dict with mode="{mode}", '
|
||||
'original_decision_id="{orig}", no guidance text, '
|
||||
"and patch preview note about pending execution"
|
||||
)
|
||||
def step_diff_dict_no_guidance_pending(context: Context, mode: str, orig: str) -> None:
|
||||
"""Build a plain data dict with no guidance and a pending-execution hint."""
|
||||
data = _make_diff_data(
|
||||
mode=mode,
|
||||
original_decision_id=orig,
|
||||
new_decision_id=None,
|
||||
guidance="",
|
||||
completed_at=False,
|
||||
patched_hint=(
|
||||
"Patch preview is available after correction execution completes. "
|
||||
"Current state: executing"
|
||||
),
|
||||
)
|
||||
context.diff_data = data
|
||||
|
||||
|
||||
@given(
|
||||
'I have a correction diff dict with mode="{mode}", '
|
||||
'original_decision_id="{orig}", no guidance, '
|
||||
"and patch preview note about pending execution"
|
||||
)
|
||||
def step_diff_dict_no_guidance_short_pending(
|
||||
context: Context, mode: str, orig: str
|
||||
) -> None:
|
||||
"""Same as the 'no guidance text' variant - 'no guidance' short form."""
|
||||
data = _make_diff_data(
|
||||
mode=mode,
|
||||
original_decision_id=orig,
|
||||
new_decision_id=None,
|
||||
guidance="",
|
||||
completed_at=False,
|
||||
patched_hint=(
|
||||
"Patch preview is available after correction execution completes. "
|
||||
"Current state: executing"
|
||||
),
|
||||
)
|
||||
context.diff_data = data
|
||||
@@ -89,7 +89,7 @@ def _make_lifecycle_mock(plan: MagicMock) -> MagicMock:
|
||||
"""Create a mock PlanLifecycleService that returns *plan*."""
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
lifecycle.fail_apply = MagicMock(return_value=plan)
|
||||
return lifecycle
|
||||
|
||||
@@ -165,10 +165,10 @@ def step_plan_has_merge_conflict(context: Context) -> None:
|
||||
assert details["sandbox_rollback"] == "pending"
|
||||
|
||||
|
||||
@then("pas_branch lifecycle _commit_plan should have been invoked before the error")
|
||||
def step_commit_plan_invoked(context: Context) -> None:
|
||||
"""Assert _commit_plan was called (it runs before the logger call)."""
|
||||
context.branch_lifecycle._commit_plan.assert_called_once()
|
||||
@then("pas_branch lifecycle commit_plan should have been invoked before the error")
|
||||
def stepcommit_plan_invoked(context: Context) -> None:
|
||||
"""Assert commit_plan was called (it runs before the logger call)."""
|
||||
context.branch_lifecycle.commit_plan.assert_called_once()
|
||||
|
||||
|
||||
@then("pas_branch lifecycle fail_apply should have been invoked before the error")
|
||||
|
||||
@@ -101,7 +101,7 @@ def _make_lifecycle_mock(plan: MagicMock | None = None) -> MagicMock:
|
||||
lifecycle = MagicMock()
|
||||
if plan is not None:
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
lifecycle.complete_apply = MagicMock()
|
||||
lifecycle.constrain_apply = MagicMock()
|
||||
lifecycle.fail_apply = MagicMock()
|
||||
|
||||
@@ -191,7 +191,7 @@ def _make_rename_entry(
|
||||
def _make_lifecycle_mock() -> MagicMock:
|
||||
"""Create a mock PlanLifecycleService."""
|
||||
lifecycle = MagicMock()
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
lifecycle.complete_apply = MagicMock()
|
||||
lifecycle.constrain_apply = MagicMock()
|
||||
lifecycle.fail_apply = MagicMock()
|
||||
@@ -750,10 +750,10 @@ def step_plan_error_details_has_key(context: Context, key: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
@then("pas_cov the lifecycle _commit_plan should have been called")
|
||||
def step_commit_plan_called(context: Context) -> None:
|
||||
"""Assert lifecycle._commit_plan was called."""
|
||||
context.pas_lifecycle._commit_plan.assert_called()
|
||||
@then("pas_cov the lifecycle commit_plan should have been called")
|
||||
def stepcommit_plan_called(context: Context) -> None:
|
||||
"""Assert lifecycle.commit_plan was called."""
|
||||
context.pas_lifecycle.commit_plan.assert_called()
|
||||
|
||||
|
||||
# ======================================================================
|
||||
|
||||
@@ -319,7 +319,7 @@ def step_plcov3_mock_use_action(context: Context) -> None:
|
||||
)
|
||||
service.use_action.return_value = plan
|
||||
service.save_plan.return_value = None
|
||||
service._commit_plan.return_value = None
|
||||
service.commit_plan.return_value = None
|
||||
|
||||
_start_patch(context, _PATCH_GET_LIFECYCLE, return_value=service)
|
||||
_start_patch(context, _PATCH_NOTIFY_FACADE)
|
||||
|
||||
@@ -148,7 +148,7 @@ def step_plan_with_changeset(context: Context) -> None:
|
||||
# Set changeset_id on plan
|
||||
plan.changeset_id = changeset.changeset_id
|
||||
plan.sandbox_refs = ["sandbox-ref-001", "sandbox-ref-002"]
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan_id = plan_id
|
||||
context.changeset = changeset
|
||||
|
||||
@@ -171,7 +171,7 @@ def step_plan_with_empty_changeset(context: Context) -> None:
|
||||
)
|
||||
context.changeset_store._store[empty_cs.changeset_id] = empty_cs
|
||||
plan.changeset_id = empty_cs.changeset_id
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan_id = plan_id
|
||||
|
||||
|
||||
@@ -189,7 +189,7 @@ def step_plan_with_validation(context: Context) -> None:
|
||||
"failed": 1,
|
||||
"skipped": 0,
|
||||
}
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan_id = plan_id
|
||||
|
||||
|
||||
@@ -543,7 +543,7 @@ def step_plan_with_apply_metadata(context: Context) -> None:
|
||||
plan.error_details = {}
|
||||
plan.error_details["apply_files_changed"] = "7"
|
||||
plan.error_details["apply_validations_run"] = "4"
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan_id = plan_id
|
||||
|
||||
|
||||
@@ -555,7 +555,7 @@ def step_plan_with_changeset_id_but_no_store_entry(context: Context) -> None:
|
||||
plan = context.lifecycle_service.get_plan(plan_id)
|
||||
# Set changeset_id but do NOT add anything to the store
|
||||
plan.changeset_id = "missing-changeset-999"
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan_id = plan_id
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,340 @@
|
||||
"""Step definitions for plan_execution_context_coverage_boost.feature."""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_execution_context import (
|
||||
PlanExecutionContext,
|
||||
RuntimeExecuteActor,
|
||||
RuntimeExecuteResult,
|
||||
)
|
||||
from cleveragents.core.exceptions import PlanError, ValidationError
|
||||
from cleveragents.domain.models.core.change import ChangeSetStore
|
||||
from cleveragents.tool.runner import ToolRunner
|
||||
|
||||
|
||||
@given("a pec fresh context")
|
||||
def step_pec_fresh_context(context: Context) -> None:
|
||||
pass
|
||||
|
||||
|
||||
@when("I pec construct PlanExecutionContext with empty plan_id")
|
||||
def step_pec_construct_empty_plan_id(context: Context) -> None:
|
||||
context.pec_error = None
|
||||
try:
|
||||
PlanExecutionContext(plan_id="")
|
||||
except Exception as exc:
|
||||
context.pec_error = exc
|
||||
|
||||
|
||||
@then('a pec ValidationError should be raised with "plan_id must not be empty"')
|
||||
def step_pec_validation_error_empty_plan_id(context: Context) -> None:
|
||||
assert isinstance(context.pec_error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
|
||||
)
|
||||
assert "plan_id must not be empty" in str(context.pec_error)
|
||||
|
||||
|
||||
@given("a pec PlanExecutionContext with all optional fields set")
|
||||
def step_pec_context_with_all_fields(context: Context) -> None:
|
||||
context.pec_store = MagicMock(spec=ChangeSetStore)
|
||||
context.pec_ctx = PlanExecutionContext(
|
||||
plan_id="01PECPLAN01TESTPECPLAN01",
|
||||
decision_root_id="01PECDECI01TESTPECDECI01",
|
||||
sandbox_root="/tmp/pec-sandbox",
|
||||
automation_profile="trusted",
|
||||
project_resources={"repo": "my-repo"},
|
||||
resource_bindings={"db": MagicMock()},
|
||||
changeset_store=context.pec_store,
|
||||
plan_env="docker",
|
||||
project_env="kubernetes",
|
||||
)
|
||||
|
||||
|
||||
@then("pec the automation_profile property should return the configured value")
|
||||
def step_pec_automation_profile(context: Context) -> None:
|
||||
assert context.pec_ctx.automation_profile == "trusted"
|
||||
|
||||
|
||||
@then("pec the plan_env property should return the configured value")
|
||||
def step_pec_plan_env(context: Context) -> None:
|
||||
assert context.pec_ctx.plan_env == "docker"
|
||||
|
||||
|
||||
@then("pec the project_env property should return the configured value")
|
||||
def step_pec_project_env(context: Context) -> None:
|
||||
assert context.pec_ctx.project_env == "kubernetes"
|
||||
|
||||
|
||||
@then("pec the project_resources property should return the configured dict")
|
||||
def step_pec_project_resources(context: Context) -> None:
|
||||
assert context.pec_ctx.project_resources == {"repo": "my-repo"}
|
||||
|
||||
|
||||
@then("pec the resource_bindings property should return the configured dict")
|
||||
def step_pec_resource_bindings(context: Context) -> None:
|
||||
assert len(context.pec_ctx.resource_bindings) == 1
|
||||
assert "db" in context.pec_ctx.resource_bindings
|
||||
|
||||
|
||||
@then("pec the changeset_store property should return the configured store")
|
||||
def step_pec_changeset_store(context: Context) -> None:
|
||||
assert context.pec_ctx.changeset_store is context.pec_store
|
||||
|
||||
|
||||
@then("pec the active_changeset_ids property should return a list")
|
||||
def step_pec_active_changeset_ids(context: Context) -> None:
|
||||
assert isinstance(context.pec_ctx.active_changeset_ids, list)
|
||||
|
||||
|
||||
@given("a pec PlanExecutionContext with a valid plan_id")
|
||||
def step_pec_context_valid_plan_id(context: Context) -> None:
|
||||
store = MagicMock(spec=ChangeSetStore)
|
||||
store.get.return_value = None # unknown changeset → None
|
||||
context.pec_ctx = PlanExecutionContext(
|
||||
plan_id="01PECPLAN02TESTPECPLAN02",
|
||||
changeset_store=store,
|
||||
)
|
||||
|
||||
|
||||
@when("I pec call record_change without starting a changeset")
|
||||
def step_pec_record_change_no_changeset(context: Context) -> None:
|
||||
from cleveragents.domain.models.core.change import ChangeEntry, ChangeOperation
|
||||
|
||||
entry = ChangeEntry(
|
||||
plan_id="01PECPLAN02TESTPECPLAN02",
|
||||
resource_id="res-1",
|
||||
tool_name="test-tool",
|
||||
operation=ChangeOperation.MODIFY,
|
||||
path="/tmp/test",
|
||||
)
|
||||
context.pec_error = None
|
||||
try:
|
||||
context.pec_ctx.record_change(entry)
|
||||
except Exception as exc:
|
||||
context.pec_error = exc
|
||||
|
||||
|
||||
@then("a pec PlanError should be raised about no active changeset")
|
||||
def step_pec_no_active_changeset(context: Context) -> None:
|
||||
assert isinstance(context.pec_error, PlanError), (
|
||||
f"Expected PlanError, got {type(context.pec_error)}: {context.pec_error}"
|
||||
)
|
||||
assert "No active changeset" in str(context.pec_error)
|
||||
|
||||
|
||||
@when("I pec call get_changeset with a nonexistent changeset_id")
|
||||
def step_pec_get_changeset_nonexistent(context: Context) -> None:
|
||||
context.pec_get_result = context.pec_ctx.get_changeset("nonexistent-id")
|
||||
|
||||
|
||||
@then("pec the result should be None")
|
||||
def step_pec_result_none(context: Context) -> None:
|
||||
assert context.pec_get_result is None, (
|
||||
f"Expected None, got {context.pec_get_result}"
|
||||
)
|
||||
|
||||
|
||||
@when("I pec call summarize on the execution context")
|
||||
def step_pec_summarize(context: Context) -> None:
|
||||
context.pec_summary = context.pec_ctx.summarize()
|
||||
|
||||
|
||||
@then("pec the summary should contain plan_id and decision_root_id")
|
||||
def step_pec_summary_has_core_fields(context: Context) -> None:
|
||||
s = context.pec_summary
|
||||
assert s["plan_id"] == "01PECPLAN02TESTPECPLAN02"
|
||||
assert "decision_root_id" in s
|
||||
|
||||
|
||||
@then("pec the summary should contain counts for resources and bindings")
|
||||
def step_pec_summary_has_counts(context: Context) -> None:
|
||||
s = context.pec_summary
|
||||
assert "project_resource_count" in s
|
||||
assert "resource_binding_count" in s
|
||||
assert "active_changeset_count" in s
|
||||
assert "active_changeset_ids" in s
|
||||
assert "changeset_summaries" in s
|
||||
|
||||
|
||||
@given("a pec valid PlanExecutionContext")
|
||||
def step_pec_valid_context(context: Context) -> None:
|
||||
store = MagicMock(spec=ChangeSetStore)
|
||||
context.pec_ctx = PlanExecutionContext(
|
||||
plan_id="01PECPLAN03TESTPECPLAN03",
|
||||
changeset_store=store,
|
||||
)
|
||||
|
||||
|
||||
@when("I pec construct RuntimeExecuteActor with None tool_runner")
|
||||
def step_pec_runtime_actor_no_tool_runner(context: Context) -> None:
|
||||
context.pec_error = None
|
||||
try:
|
||||
RuntimeExecuteActor(
|
||||
tool_runner=None,
|
||||
execution_context=context.pec_ctx,
|
||||
)
|
||||
except Exception as exc:
|
||||
context.pec_error = exc
|
||||
|
||||
|
||||
@then('a pec ValidationError should be raised with "tool_runner must not be None"')
|
||||
def step_pec_tool_runner_validation_error(context: Context) -> None:
|
||||
assert isinstance(context.pec_error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
|
||||
)
|
||||
assert "tool_runner must not be None" in str(context.pec_error)
|
||||
|
||||
|
||||
@given("a pec mock ToolRunner")
|
||||
def step_pec_mock_tool_runner(context: Context) -> None:
|
||||
context.pec_tool_runner = MagicMock(spec=ToolRunner)
|
||||
|
||||
|
||||
@when("I pec construct RuntimeExecuteActor with None execution_context")
|
||||
def step_pec_runtime_actor_no_context(context: Context) -> None:
|
||||
context.pec_error = None
|
||||
try:
|
||||
RuntimeExecuteActor(
|
||||
tool_runner=context.pec_tool_runner,
|
||||
execution_context=None,
|
||||
)
|
||||
except Exception as exc:
|
||||
context.pec_error = exc
|
||||
|
||||
|
||||
@then(
|
||||
'a pec ValidationError should be raised with "execution_context must not be None"'
|
||||
)
|
||||
def step_pec_execution_context_validation_error(context: Context) -> None:
|
||||
assert isinstance(context.pec_error, ValidationError), (
|
||||
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
|
||||
)
|
||||
assert "execution_context must not be None" in str(context.pec_error)
|
||||
|
||||
|
||||
@given("a pec RuntimeExecuteActor with valid tool_runner and execution_context")
|
||||
def step_pec_runtime_actor_valid(context: Context) -> None:
|
||||
store = MagicMock(spec=ChangeSetStore)
|
||||
store.start.return_value = "01PECCHG01TESTPECCHG01"
|
||||
tool_runner = MagicMock(spec=ToolRunner)
|
||||
tool_runner.discover.return_value = []
|
||||
|
||||
ctx = PlanExecutionContext(
|
||||
plan_id="01PECPLAN04TESTPECPLAN04",
|
||||
changeset_store=store,
|
||||
)
|
||||
context.pec_store = store
|
||||
context.pec_tool_runner = tool_runner
|
||||
context.pec_actor = RuntimeExecuteActor(
|
||||
tool_runner=tool_runner,
|
||||
execution_context=ctx,
|
||||
)
|
||||
|
||||
|
||||
@given(
|
||||
"a pec RuntimeExecuteActor with valid tool_runner and execution_context "
|
||||
"with sandbox_root"
|
||||
)
|
||||
def step_pec_runtime_actor_with_sandbox(context: Context) -> None:
|
||||
store = MagicMock(spec=ChangeSetStore)
|
||||
store.start.return_value = "01PECCHG02TESTPECCHG02"
|
||||
tool_runner = MagicMock(spec=ToolRunner)
|
||||
tool_runner.discover.return_value = []
|
||||
|
||||
ctx = PlanExecutionContext(
|
||||
plan_id="01PECPLAN05TESTPECPLAN05",
|
||||
sandbox_root="/tmp/pec-sandbox",
|
||||
changeset_store=store,
|
||||
)
|
||||
context.pec_store = store
|
||||
context.pec_tool_runner = tool_runner
|
||||
context.pec_actor = RuntimeExecuteActor(
|
||||
tool_runner=tool_runner,
|
||||
execution_context=ctx,
|
||||
)
|
||||
|
||||
|
||||
@given("a pec stream event collector")
|
||||
def step_pec_stream_collector(context: Context) -> None:
|
||||
events: list[tuple[str, dict]] = []
|
||||
context.pec_events = events
|
||||
context.pec_stream_callback = lambda evt, data: events.append((evt, data))
|
||||
|
||||
|
||||
@then("pec the tool_runner property should return the configured ToolRunner")
|
||||
def step_pec_tool_runner_property(context: Context) -> None:
|
||||
assert context.pec_actor.tool_runner is context.pec_tool_runner
|
||||
|
||||
|
||||
@then(
|
||||
"pec the execution_context property should return the configured "
|
||||
"PlanExecutionContext"
|
||||
)
|
||||
def step_pec_execution_context_property(context: Context) -> None:
|
||||
assert context.pec_actor.execution_context is not None
|
||||
assert context.pec_actor.execution_context.plan_id == "01PECPLAN04TESTPECPLAN04"
|
||||
|
||||
|
||||
class _FakeDecision:
|
||||
def __init__(self, decision_id: str, step_text: str, sequence: int):
|
||||
self.decision_id = decision_id
|
||||
self.step_text = step_text
|
||||
self.sequence = sequence
|
||||
|
||||
|
||||
@when("I pec execute with decisions and stream callback")
|
||||
def step_pec_execute_with_callback(context: Context) -> None:
|
||||
decisions = [
|
||||
_FakeDecision("dec-1", "Step 1", 0),
|
||||
_FakeDecision("dec-2", "Step 2", 1),
|
||||
]
|
||||
context.pec_result = context.pec_actor.execute(
|
||||
decisions=decisions,
|
||||
stream_callback=context.pec_stream_callback,
|
||||
)
|
||||
|
||||
|
||||
@when("I pec execute with decisions and no stream callback")
|
||||
def step_pec_execute_no_callback(context: Context) -> None:
|
||||
decisions = [
|
||||
_FakeDecision("dec-1", "Step 1", 0),
|
||||
]
|
||||
context.pec_result = context.pec_actor.execute(
|
||||
decisions=decisions,
|
||||
stream_callback=None,
|
||||
)
|
||||
|
||||
|
||||
@then('pec the stream events should include "runtime_execute_started"')
|
||||
def step_pec_events_started(context: Context) -> None:
|
||||
event_names = [e[0] for e in context.pec_events]
|
||||
assert "runtime_execute_started" in event_names, (
|
||||
f"Expected runtime_execute_started in {event_names}"
|
||||
)
|
||||
|
||||
|
||||
@then('pec the stream events should include "runtime_execute_complete"')
|
||||
def step_pec_events_complete(context: Context) -> None:
|
||||
event_names = [e[0] for e in context.pec_events]
|
||||
assert "runtime_execute_complete" in event_names, (
|
||||
f"Expected runtime_execute_complete in {event_names}"
|
||||
)
|
||||
|
||||
|
||||
@then("pec the result should be a RuntimeExecuteResult with a valid changeset_id")
|
||||
def step_pec_result_valid_runtime_result(context: Context) -> None:
|
||||
assert isinstance(context.pec_result, RuntimeExecuteResult), (
|
||||
f"Expected RuntimeExecuteResult, got {type(context.pec_result)}"
|
||||
)
|
||||
assert context.pec_result.changeset_id is not None
|
||||
assert len(context.pec_result.changeset_id) > 0
|
||||
|
||||
|
||||
@then("pec the result sandbox_refs should include the sandbox_root")
|
||||
def step_pec_sandbox_refs_include_root(context: Context) -> None:
|
||||
assert "/tmp/pec-sandbox" in context.pec_result.sandbox_refs, (
|
||||
f"Expected /tmp/pec-sandbox in sandbox_refs: {context.pec_result.sandbox_refs}"
|
||||
)
|
||||
@@ -0,0 +1,770 @@
|
||||
"""Step definitions for plan_execution_hierarchical_4phase.feature.
|
||||
|
||||
Integration tests verifying that plans in a hierarchy independently complete
|
||||
all four lifecycle phases: Strategize, Decompose, Execute, and Validate
|
||||
(Forgejo #10270). Uses real (non-mocked) service implementations in in-memory mode.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from ulid import ULID
|
||||
|
||||
from cleveragents.application.services.checkpoint_service import CheckpointService
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
from cleveragents.application.services.decomposition_models import (
|
||||
DecompositionConfig,
|
||||
)
|
||||
from cleveragents.application.services.decomposition_service import (
|
||||
DecompositionService,
|
||||
)
|
||||
from cleveragents.application.services.plan_executor import PlanExecutor
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.application.services.subplan_execution_service import (
|
||||
SubplanExecutionOutput,
|
||||
SubplanExecutionService,
|
||||
)
|
||||
from cleveragents.application.services.subplan_service import (
|
||||
SubplanService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.domain.models.core.decision import (
|
||||
DecisionType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
NamespacedName,
|
||||
Plan,
|
||||
PlanIdentity,
|
||||
PlanPhase,
|
||||
PlanTimestamps,
|
||||
ProcessingState,
|
||||
SubplanConfig,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _execute_child_plan_with_transition(
|
||||
executor: PlanExecutor,
|
||||
lcs: PlanLifecycleService,
|
||||
status: Any,
|
||||
) -> SubplanExecutionOutput:
|
||||
"""Execute a child plan with explicit phase transition from Strategize to Execute.
|
||||
|
||||
The PlanExecutor._execute_child_plan method calls run_strategize then
|
||||
run_execute, but does not call execute_plan between them to transition the
|
||||
plan's phase. This helper bridges that gap so child plans complete the
|
||||
full 4-phase lifecycle: Strategize → (transition) → Execute.
|
||||
"""
|
||||
subplan_id = status.subplan_id
|
||||
|
||||
# Strategize
|
||||
result = executor.run_strategize(subplan_id)
|
||||
if not result.decision_root_id or not result.decisions:
|
||||
return SubplanExecutionOutput(
|
||||
subplan_id=subplan_id,
|
||||
success=False,
|
||||
error=f"Child plan {subplan_id} strategize produced no decisions",
|
||||
)
|
||||
|
||||
# Transition: Strategize → Execute
|
||||
lcs.execute_plan(subplan_id)
|
||||
|
||||
# Execute
|
||||
execute_result = executor.run_execute(subplan_id)
|
||||
files_changed = execute_result.tool_calls_count
|
||||
|
||||
return SubplanExecutionOutput(
|
||||
subplan_id=subplan_id,
|
||||
success=True,
|
||||
files={},
|
||||
files_changed=files_changed,
|
||||
changeset_summary=f"Child plan {subplan_id[:8]} executed",
|
||||
)
|
||||
|
||||
|
||||
def _make_plan(
|
||||
plan_id: str = "",
|
||||
name: str = "test-plan",
|
||||
description: str = "Test plan",
|
||||
action_name: str = "local/test-action",
|
||||
definition_of_done: str = "- [ ] Step one\n- [ ] Step two",
|
||||
phase: PlanPhase = PlanPhase.STRATEGIZE,
|
||||
state: ProcessingState = ProcessingState.QUEUED,
|
||||
parent_id: str | None = None,
|
||||
root_id: str | None = None,
|
||||
) -> Plan:
|
||||
pid = plan_id if plan_id else str(ULID())
|
||||
if root_id is None:
|
||||
root_id = pid
|
||||
return Plan(
|
||||
identity=PlanIdentity(
|
||||
plan_id=pid,
|
||||
parent_plan_id=parent_id,
|
||||
root_plan_id=root_id,
|
||||
),
|
||||
namespaced_name=NamespacedName(namespace="local", name=name),
|
||||
description=description,
|
||||
action_name=action_name,
|
||||
definition_of_done=definition_of_done,
|
||||
phase=phase,
|
||||
processing_state=state,
|
||||
timestamps=PlanTimestamps(),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background: services
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a plan lifecycle service with in-memory storage")
|
||||
def step_given_lifecycle_service(context: Context) -> None:
|
||||
context.lifecycle_service = PlanLifecycleService(settings=Settings())
|
||||
context.plans_by_name: dict[str, Plan] = {}
|
||||
context.child_plan_ids: list[str] = []
|
||||
context.checkpoint_service = None
|
||||
|
||||
|
||||
@given("a DecisionService for recording spawn decisions")
|
||||
def step_given_decision_service(context: Context) -> None:
|
||||
context.decision_service = DecisionService()
|
||||
|
||||
|
||||
@given("a SubplanService backed by the DecisionService")
|
||||
def step_given_subplan_service(context: Context) -> None:
|
||||
context.subplan_service = SubplanService(decision_service=context.decision_service)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: parent plan with spawn decision and NO child action
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
'a parent plan "{plan_name}" in Strategize phase with a spawn decision '
|
||||
"recorded without action registration"
|
||||
)
|
||||
def step_given_parent_with_spawn_no_action(context: Context, plan_name: str) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
ds = context.decision_service
|
||||
|
||||
action = lcs.create_action(
|
||||
name=f"local/parent-{plan_name}",
|
||||
description=f"Parent action for {plan_name}",
|
||||
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children\n- [ ] Aggregate results",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
plan = lcs.use_action(action_name=str(action.namespaced_name))
|
||||
pid = plan.identity.plan_id
|
||||
plan.description = f"Parent plan {plan_name}"
|
||||
plan.definition_of_done = (
|
||||
"- [ ] Spawn children\n- [ ] Execute children\n- [ ] Aggregate"
|
||||
)
|
||||
|
||||
# Record spawn decision referencing an action that does NOT exist
|
||||
# (no action registration for 'local/child-missing-action')
|
||||
decision = ds.record_decision(
|
||||
plan_id=pid,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question="Spawn a child plan?",
|
||||
chosen_option="local/child-missing-action",
|
||||
plan_phase="strategize",
|
||||
)
|
||||
|
||||
context.plans_by_name[plan_name] = plan
|
||||
context.current_parent_plan = plan
|
||||
context.parent_spawn_decisions = [decision]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: parent plan setup
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('a parent plan "{plan_name}" in Strategize phase with a spawn decision recorded')
|
||||
def step_given_parent_plan_with_spawn_decision(
|
||||
context: Context, plan_name: str
|
||||
) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
ds = context.decision_service
|
||||
|
||||
action = lcs.create_action(
|
||||
name=f"local/parent-{plan_name}",
|
||||
description=f"Parent action for {plan_name}",
|
||||
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children\n- [ ] Aggregate results",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
plan = lcs.use_action(action_name=str(action.namespaced_name))
|
||||
pid = plan.identity.plan_id
|
||||
plan.description = f"Parent plan {plan_name}"
|
||||
plan.definition_of_done = (
|
||||
"- [ ] Spawn children\n- [ ] Execute children\n- [ ] Aggregate"
|
||||
)
|
||||
|
||||
decision = ds.record_decision(
|
||||
plan_id=pid,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question="Spawn a child plan?",
|
||||
chosen_option="local/child-action",
|
||||
plan_phase="strategize",
|
||||
)
|
||||
|
||||
# Pre-register the child action referenced by the spawn decision
|
||||
try:
|
||||
lcs.get_action("local/child-action")
|
||||
except Exception:
|
||||
lcs.create_action(
|
||||
name="local/child-action",
|
||||
description="Child action for spawned subplan",
|
||||
definition_of_done="- [ ] Complete child task",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
context.plans_by_name[plan_name] = plan
|
||||
context.current_parent_plan = plan
|
||||
context.parent_spawn_decisions = [decision]
|
||||
|
||||
|
||||
@given(
|
||||
'a parent plan "{plan_name}" in Strategize phase with two spawn decisions recorded'
|
||||
)
|
||||
def step_given_parent_plan_with_two_spawn_decisions(
|
||||
context: Context, plan_name: str
|
||||
) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
ds = context.decision_service
|
||||
|
||||
action = lcs.create_action(
|
||||
name=f"local/parent-{plan_name}",
|
||||
description=f"Parent action for {plan_name}",
|
||||
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
plan = lcs.use_action(action_name=str(action.namespaced_name))
|
||||
pid = plan.identity.plan_id
|
||||
|
||||
d1 = ds.record_decision(
|
||||
plan_id=pid,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question="Spawn child plan 1?",
|
||||
chosen_option="local/child-action-1",
|
||||
plan_phase="strategize",
|
||||
)
|
||||
d2 = ds.record_decision(
|
||||
plan_id=pid,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question="Spawn child plan 2?",
|
||||
chosen_option="local/child-action-2",
|
||||
plan_phase="strategize",
|
||||
)
|
||||
|
||||
# Pre-register child actions
|
||||
for aname in ("local/child-action-1", "local/child-action-2"):
|
||||
try:
|
||||
lcs.get_action(aname)
|
||||
except Exception:
|
||||
lcs.create_action(
|
||||
name=aname,
|
||||
description=f"Child action {aname}",
|
||||
definition_of_done="- [ ] Complete child task",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
context.plans_by_name[plan_name] = plan
|
||||
context.current_parent_plan = plan
|
||||
context.parent_spawn_decisions = [d1, d2]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: child plan creation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _register_child_plans(
|
||||
context: Context,
|
||||
parent_plan: Plan,
|
||||
num_children: int,
|
||||
make_failing: bool = False,
|
||||
) -> list[Plan]:
|
||||
lcs = context.lifecycle_service
|
||||
children: list[Plan] = []
|
||||
|
||||
for i in range(num_children):
|
||||
action_name = f"local/child-action-{i}"
|
||||
# Register child action so pre-flight checks pass
|
||||
try:
|
||||
lcs.get_action(action_name)
|
||||
except Exception:
|
||||
lcs.create_action(
|
||||
name=action_name,
|
||||
description=f"Child action {i}",
|
||||
definition_of_done="- [ ] Complete child task",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
child_plan = _make_plan(
|
||||
name=f"child-plan-{i}",
|
||||
description=f"Child plan {i}",
|
||||
action_name=action_name,
|
||||
definition_of_done="- [ ] Complete child task",
|
||||
parent_id=parent_plan.identity.plan_id,
|
||||
root_id=parent_plan.identity.root_plan_id,
|
||||
)
|
||||
cpid = child_plan.identity.plan_id
|
||||
lcs._plans[cpid] = child_plan
|
||||
context.child_plan_ids.append(cpid)
|
||||
if make_failing:
|
||||
# Use an action name that is not registered, causing pre-flight
|
||||
# checks in start_strategize to fail for this child.
|
||||
child_plan.action_name = "local/nonexistent-action"
|
||||
children.append(child_plan)
|
||||
|
||||
return children
|
||||
|
||||
|
||||
@when(
|
||||
'I create two child plans for "{plan_name}" and pre-register them in the lifecycle'
|
||||
)
|
||||
def step_when_create_two_child_plans(context: Context, plan_name: str) -> None:
|
||||
parent_plan = context.plans_by_name[plan_name]
|
||||
context.child_plans = _register_child_plans(context, parent_plan, 2)
|
||||
|
||||
|
||||
@given('I create one child plan for "{plan_name}" and pre-register it in the lifecycle')
|
||||
def step_given_create_one_child_plan(context: Context, plan_name: str) -> None:
|
||||
parent_plan = context.plans_by_name[plan_name]
|
||||
context.child_plans = _register_child_plans(context, parent_plan, 1)
|
||||
|
||||
|
||||
@given(
|
||||
'I create a failing child plan for "{plan_name}" and pre-register it in the lifecycle'
|
||||
)
|
||||
def step_given_create_failing_child_plan(context: Context, plan_name: str) -> None:
|
||||
parent_plan = context.plans_by_name[plan_name]
|
||||
context.child_plans = _register_child_plans(
|
||||
context, parent_plan, 1, make_failing=True
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: checkpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('a CheckpointService is configured for the parent plan "{plan_name}"')
|
||||
def step_given_checkpoint_service(context: Context, plan_name: str) -> None:
|
||||
cs = CheckpointService()
|
||||
context.checkpoint_service = cs
|
||||
parent_plan = context.plans_by_name[plan_name]
|
||||
# Register a sandbox so checkpoint creation can resolve it
|
||||
cs.register_sandbox(parent_plan.identity.plan_id, "/tmp/test-sandbox")
|
||||
# Store the checkpoint service so it can be injected into SubplanExecutionService
|
||||
context._checkpoint_service = cs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: max-child-depth decomposition
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a decomposition service with max-child-depth {n:d}")
|
||||
def step_given_decomp_with_max_child_depth(context: Context, n: int) -> None:
|
||||
context.svc = DecompositionService()
|
||||
context.tmpdir = tempfile.mkdtemp(prefix="decompose-4phase-")
|
||||
context.decomp_config = DecompositionConfig(
|
||||
max_child_depth=n, max_files_per_subplan=30
|
||||
)
|
||||
context.result = None
|
||||
context.error = None
|
||||
context.files = None
|
||||
|
||||
|
||||
@given("a decomposition service with default config")
|
||||
def step_given_decomp_with_default_config(context: Context) -> None:
|
||||
context.svc = DecompositionService()
|
||||
context.tmpdir = tempfile.mkdtemp(prefix="decompose-default-")
|
||||
context.decomp_config = DecompositionConfig()
|
||||
context.result = None
|
||||
context.error = None
|
||||
context.files = None
|
||||
|
||||
|
||||
@when("I decompose the project files with stored config")
|
||||
def step_when_decompose_project_files(context: Context) -> None:
|
||||
config = getattr(context, "decomp_config", None)
|
||||
context.result = context.svc.decompose(context.files, config)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When: drive phases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I strategize and execute the parent plan "{plan_name}"')
|
||||
def step_when_drive_parent_plan(context: Context, plan_name: str) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
parent_plan = context.plans_by_name[plan_name]
|
||||
pid = parent_plan.identity.plan_id
|
||||
|
||||
# Check if a CheckpointService has been configured
|
||||
checkpoint_svc = getattr(context, "_checkpoint_service", None)
|
||||
|
||||
ss = context.subplan_service
|
||||
# Set up expected context attributes used by shared step definitions
|
||||
context.execute_error = None
|
||||
planner = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
subplan_service=ss,
|
||||
)
|
||||
|
||||
# ----- Strategize -----
|
||||
plan = lcs.get_plan(pid)
|
||||
assert plan.phase == PlanPhase.STRATEGIZE
|
||||
result = planner.run_strategize(pid)
|
||||
assert result.decision_root_id, "Strategize should produce a root decision"
|
||||
assert result.decisions, "Strategize should produce decisions"
|
||||
|
||||
plan = lcs.get_plan(pid)
|
||||
assert plan.state == ProcessingState.COMPLETE
|
||||
|
||||
# ----- Transition: Strategize → Execute -----
|
||||
plan = lcs.execute_plan(pid)
|
||||
assert plan.phase == PlanPhase.EXECUTE
|
||||
assert plan.state == ProcessingState.QUEUED
|
||||
|
||||
# Register child actions that spawn decisions reference, so pre-flight
|
||||
# checks in start_strategize find them. Skip for failure scenarios where
|
||||
# the intentional missing action should cause the child to fail.
|
||||
if hasattr(context, "parent_spawn_decisions") and plan_name != "parent-fail":
|
||||
for dec in context.parent_spawn_decisions:
|
||||
try:
|
||||
lcs.get_action(dec.chosen_option)
|
||||
except Exception:
|
||||
lcs.create_action(
|
||||
name=dec.chosen_option,
|
||||
description="Child action from spawn decision",
|
||||
definition_of_done="- [ ] Complete child task",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
# ----- Execute -----
|
||||
spawn_result = planner._spawn_subplans(plan)
|
||||
|
||||
if spawn_result is not None and spawn_result.child_plans:
|
||||
# Register spawned child plans in lifecycle service
|
||||
for child_plan in spawn_result.child_plans:
|
||||
cpid = child_plan.identity.plan_id
|
||||
if cpid not in lcs._plans:
|
||||
lcs._plans[cpid] = child_plan
|
||||
context.child_plan_ids.append(cpid)
|
||||
|
||||
# Build SubplanExecutionService with optional CheckpointService
|
||||
config = getattr(plan, "subplan_config", None) or SubplanConfig()
|
||||
exec_svc = SubplanExecutionService(
|
||||
config=config,
|
||||
executor_fn=lambda status: _execute_child_plan_with_transition(
|
||||
planner, lcs, status
|
||||
),
|
||||
checkpoint_service=checkpoint_svc,
|
||||
parent_plan_id=pid,
|
||||
)
|
||||
|
||||
exec_result = exec_svc.execute_all(
|
||||
subplan_statuses=spawn_result.spawned_statuses,
|
||||
base_files={},
|
||||
)
|
||||
planner._apply_subplan_results_to_plan(plan, spawn_result, exec_result)
|
||||
# Persist plan mutations (error_details, subplan_statuses) to in-memory store
|
||||
lcs._plans[pid] = plan
|
||||
context.spawn_result = spawn_result
|
||||
context.exec_result = exec_result
|
||||
else:
|
||||
context.spawn_result = spawn_result
|
||||
context.exec_result = None
|
||||
|
||||
lcs.start_execute(pid)
|
||||
lcs.complete_execute(pid)
|
||||
|
||||
plan = lcs.get_plan(pid)
|
||||
context.plans_by_name[plan_name] = plan
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: phase completion
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the parent plan Strategize phase should be complete")
|
||||
def step_then_parent_strategize_complete(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected EXECUTE phase, got {plan.phase.value}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parent plan Execute phase should be complete")
|
||||
def step_then_parent_execute_complete(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
assert plan.state == ProcessingState.COMPLETE, (
|
||||
f"Expected COMPLETE state, got {plan.state.value if plan.state else 'None'}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: child plan phase verification
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("each child plan should have completed Strategize phase")
|
||||
def step_then_each_child_completed_strategize(context: Context) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
for cpid in context.child_plan_ids:
|
||||
child = lcs._plans.get(cpid)
|
||||
assert child is not None, f"Child plan {cpid} not in lifecycle"
|
||||
# After _execute_child_plan's run_strategize, child should be
|
||||
# STRATEGIZE/COMPLETE (unless it already transitioned to EXECUTE)
|
||||
assert child.processing_state in (
|
||||
ProcessingState.COMPLETE,
|
||||
ProcessingState.QUEUED,
|
||||
), (
|
||||
f"Child {cpid[:8]} unexpected state: {child.processing_state.value if child.processing_state else 'None'}"
|
||||
)
|
||||
|
||||
|
||||
@then("each child plan should have completed Execute phase")
|
||||
def step_then_each_child_completed_execute(context: Context) -> None:
|
||||
# Execute phase was run inside _execute_child_plan
|
||||
# Verify via the execution result
|
||||
if hasattr(context, "exec_result") and context.exec_result is not None:
|
||||
assert context.exec_result.all_succeeded, (
|
||||
f"Expected all children to succeed, failed: {context.exec_result.failed_subplan_ids}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: subplan statuses
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the parent plan should have {n:d} subplan statuses")
|
||||
def step_then_parent_has_n_subplan_statuses(context: Context, n: int) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
assert len(plan.subplan_statuses) == n, (
|
||||
f"Expected {n} subplan statuses, got {len(plan.subplan_statuses)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parent plan should have 1 subplan status")
|
||||
def step_then_parent_has_1_subplan_status(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
assert len(plan.subplan_statuses) >= 1, (
|
||||
f"Expected at least 1 subplan status, got {len(plan.subplan_statuses)}"
|
||||
)
|
||||
|
||||
|
||||
@then("all child subplan statuses should indicate success")
|
||||
def step_then_all_child_statuses_success(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
for status in plan.subplan_statuses:
|
||||
assert status.status in (
|
||||
ProcessingState.COMPLETE,
|
||||
ProcessingState.PROCESSING,
|
||||
ProcessingState.QUEUED,
|
||||
), (
|
||||
f"Expected subplan {status.subplan_id[:8]} to be successful, "
|
||||
f"got {status.status.value}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: checkpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("at least one on_subplan_spawn checkpoint should have been created")
|
||||
def step_then_on_subplan_spawn_checkpoint_created(context: Context) -> None:
|
||||
cs = context.checkpoint_service
|
||||
assert cs is not None, "CheckpointService was not configured"
|
||||
# Check checkpoints for the parent plan
|
||||
parent_plan = next(iter(context.plans_by_name.values()))
|
||||
checkpoints = cs.list_checkpoints(parent_plan.identity.plan_id)
|
||||
assert len(checkpoints) > 0, (
|
||||
f"Expected at least one checkpoint, got {len(checkpoints)}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: aggregation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the parent plan should have subplan statuses with execution results")
|
||||
def step_then_parent_has_exec_results(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
assert len(plan.subplan_statuses) > 0, "Expected subplan statuses, got none"
|
||||
|
||||
|
||||
@then("the parent plan error_details should not contain failed_subplan_ids")
|
||||
def step_then_parent_no_failed_ids(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
if plan.error_details:
|
||||
assert "failed_subplan_ids" not in dict(plan.error_details), (
|
||||
f"Expected no failed_subplan_ids, got: {plan.error_details}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: failure handling
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the parent plan error_details should have failed subplan ids")
|
||||
def step_then_parent_has_failed_ids(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
details = dict(plan.error_details or {})
|
||||
assert "failed_subplan_ids" in details, (
|
||||
f"Expected failed_subplan_ids in error_details, got: {details}"
|
||||
)
|
||||
|
||||
|
||||
@then("the failing child plan should be marked as errored")
|
||||
def step_then_failing_child_errored(context: Context) -> None:
|
||||
plan = context.current_parent_plan
|
||||
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
|
||||
errored = [s for s in plan.subplan_statuses if s.status == ProcessingState.ERRORED]
|
||||
assert len(errored) > 0, (
|
||||
f"Expected at least one errored subplan, got statuses: "
|
||||
f"{[(s.subplan_id[:8], s.status.value) for s in plan.subplan_statuses]}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: nested hierarchy
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
'I create child plan "{child_name}" for "{parent_name}" with its own spawn decision recorded'
|
||||
)
|
||||
def step_given_child_plan_with_spawn_decision(
|
||||
context: Context, child_name: str, parent_name: str
|
||||
) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
ds = context.decision_service
|
||||
parent_plan = context.plans_by_name[parent_name]
|
||||
|
||||
child_plan = _make_plan(
|
||||
name=child_name,
|
||||
description=f"Child plan {child_name}",
|
||||
action_name=f"local/{child_name}",
|
||||
definition_of_done="- [ ] Complete child task\n- [ ] Delegate to grandchild",
|
||||
parent_id=parent_plan.identity.plan_id,
|
||||
root_id=parent_plan.identity.root_plan_id,
|
||||
)
|
||||
cpid = child_plan.identity.plan_id
|
||||
lcs._plans[cpid] = child_plan
|
||||
context.child_plan_ids.append(cpid)
|
||||
|
||||
# Record spawn decision for the child (to spawn grandchild)
|
||||
ds.record_decision(
|
||||
plan_id=cpid,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question="Spawn grandchild?",
|
||||
chosen_option="local/grandchild-action",
|
||||
plan_phase="strategize",
|
||||
)
|
||||
# Pre-register grandchild action
|
||||
try:
|
||||
lcs.get_action("local/grandchild-action")
|
||||
except Exception:
|
||||
lcs.create_action(
|
||||
name="local/grandchild-action",
|
||||
description="Grandchild action",
|
||||
definition_of_done="- [ ] Complete smallest task",
|
||||
strategy_actor="local/stub-strategist",
|
||||
execution_actor="local/stub-executor",
|
||||
)
|
||||
|
||||
context.plans_by_name[child_name] = child_plan
|
||||
context.child_plan = child_plan
|
||||
|
||||
|
||||
@given(
|
||||
'I create a grandchild plan for "{child_name}" and pre-register it in the lifecycle'
|
||||
)
|
||||
def step_given_grandchild_plan(context: Context, child_name: str) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
child_plan = context.plans_by_name[child_name]
|
||||
|
||||
gc_plan = _make_plan(
|
||||
name=f"grandchild-of-{child_name}",
|
||||
description="Grandchild plan",
|
||||
action_name="local/grandchild-action",
|
||||
definition_of_done="- [ ] Complete smallest task",
|
||||
parent_id=child_plan.identity.plan_id,
|
||||
root_id=child_plan.identity.root_plan_id,
|
||||
)
|
||||
gcid = gc_plan.identity.plan_id
|
||||
lcs._plans[gcid] = gc_plan
|
||||
context.child_plan_ids.append(gcid)
|
||||
context.grandchild_plan = gc_plan
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: nested verification
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the child plan should complete both Strategize and Execute phases")
|
||||
def step_then_child_completes_both_phases(context: Context) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
child_plan = context.child_plan
|
||||
child = lcs._plans.get(child_plan.identity.plan_id)
|
||||
assert child is not None, "Child plan not found"
|
||||
assert child.processing_state in (
|
||||
ProcessingState.COMPLETE,
|
||||
ProcessingState.QUEUED,
|
||||
), (
|
||||
f"Child plan unexpected state: {child.processing_state.value if child.processing_state else 'None'}"
|
||||
)
|
||||
|
||||
|
||||
@then("the grandchild plan should complete both Strategize and Execute phases")
|
||||
def step_then_grandchild_completes_both_phases(context: Context) -> None:
|
||||
lcs = context.lifecycle_service
|
||||
gc = context.grandchild_plan
|
||||
gc_plan = lcs._plans.get(gc.identity.plan_id)
|
||||
assert gc_plan is not None, "Grandchild plan not found"
|
||||
assert gc_plan.processing_state in (
|
||||
ProcessingState.COMPLETE,
|
||||
ProcessingState.QUEUED,
|
||||
), (
|
||||
f"Grandchild unexpected state: {gc_plan.processing_state.value if gc_plan.processing_state else 'None'}"
|
||||
)
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Step definitions for plan_execution_hierarchical.feature.
|
||||
|
||||
Tests that PlanExecutor lazily creates SubplanExecutionService when
|
||||
SubplanService is wired but SubplanExecutionService is not pre-configured
|
||||
(Forgejo #10268).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.plan_executor import PlanExecutor
|
||||
from cleveragents.application.services.subplan_execution_service import (
|
||||
SubplanExecutionResult,
|
||||
SubplanExecutionService,
|
||||
)
|
||||
from cleveragents.application.services.subplan_service import (
|
||||
SpawnEntry,
|
||||
SpawnResult,
|
||||
SubplanService,
|
||||
)
|
||||
from cleveragents.domain.models.core.decision import (
|
||||
ContextSnapshot,
|
||||
Decision,
|
||||
DecisionType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
ExecutionMode,
|
||||
NamespacedName,
|
||||
Plan,
|
||||
PlanIdentity,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
SubplanStatus,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PLAN_ID = "01KNFMGJSG67S6RG9TVXV205TQ"
|
||||
_ROOT_ID = "01KNFMGJSH67S6RG9TVXV205TR"
|
||||
_DEC_ID = "01KNFMGJSH67S6RG9TVXV205TS"
|
||||
_SUBPLAN_ID = "01KNFMGJSH67S6RG9TVXV205TT"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_plan(*, phase=PlanPhase.EXECUTE, state=ProcessingState.QUEUED):
|
||||
return Plan(
|
||||
identity=PlanIdentity(plan_id=_PLAN_ID, root_plan_id=_ROOT_ID),
|
||||
namespaced_name=NamespacedName(namespace="local", name="test-plan"),
|
||||
description="Test plan for hierarchical execution",
|
||||
action_name="local/test-action",
|
||||
phase=phase,
|
||||
processing_state=state,
|
||||
)
|
||||
|
||||
|
||||
def _make_spawn_decision(
|
||||
decision_type: DecisionType = DecisionType.SUBPLAN_SPAWN,
|
||||
) -> Decision:
|
||||
return Decision(
|
||||
decision_id=_DEC_ID,
|
||||
plan_id=_PLAN_ID,
|
||||
decision_type=decision_type,
|
||||
sequence_number=0,
|
||||
question="Spawn a child plan?",
|
||||
chosen_option="local/sub-action",
|
||||
context_snapshot=ContextSnapshot(relevant_resources=[]),
|
||||
)
|
||||
|
||||
|
||||
def _make_spawn_result(
|
||||
execution_mode: str = ExecutionMode.SEQUENTIAL,
|
||||
) -> SpawnResult:
|
||||
sub_status = SubplanStatus(
|
||||
subplan_id=_SUBPLAN_ID,
|
||||
action_name="local/sub-action",
|
||||
)
|
||||
return SpawnResult(
|
||||
spawned_statuses=[sub_status],
|
||||
metadata={},
|
||||
total_spawned=1,
|
||||
execution_mode=execution_mode,
|
||||
child_plans=[],
|
||||
)
|
||||
|
||||
|
||||
def _make_subplan_service(
|
||||
decisions: list[Decision] | None = None,
|
||||
spawn_result: SpawnResult | None = None,
|
||||
) -> MagicMock:
|
||||
svc = MagicMock(spec=SubplanService)
|
||||
svc.get_spawn_decisions.return_value = decisions or []
|
||||
svc.build_spawn_entries.return_value = [
|
||||
SpawnEntry(decision=d, action_name="local/sub-action")
|
||||
for d in (decisions or [])
|
||||
]
|
||||
svc.spawn.return_value = spawn_result or _make_spawn_result()
|
||||
return svc
|
||||
|
||||
|
||||
def _make_lifecycle(plan: Plan) -> MagicMock:
|
||||
lcs = MagicMock()
|
||||
lcs.get_plan.return_value = plan
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a PlanExecutor with SubplanService but no SubplanExecutionService")
|
||||
def step_given_executor_with_subplan_service_only(context: Context) -> None:
|
||||
decision = _make_spawn_decision(DecisionType.SUBPLAN_SPAWN)
|
||||
spawn_result = _make_spawn_result()
|
||||
context.spawn_decision = decision
|
||||
context.spawn_result = spawn_result
|
||||
|
||||
mock_subplan_svc = _make_subplan_service(
|
||||
decisions=[decision], spawn_result=spawn_result
|
||||
)
|
||||
plan = _make_plan()
|
||||
plan.decision_root_id = _ROOT_ID
|
||||
context.plan = plan
|
||||
|
||||
lcs = _make_lifecycle(plan)
|
||||
execute_actor = MagicMock()
|
||||
execute_result = MagicMock()
|
||||
execute_result.changeset_id = "01JSPAWN0000000000000CS0001"
|
||||
execute_result.sandbox_refs = []
|
||||
execute_result.tool_calls_count = 0
|
||||
execute_actor.execute.return_value = execute_result
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
execute_actor=execute_actor,
|
||||
subplan_service=mock_subplan_svc,
|
||||
subplan_execution_service=None,
|
||||
)
|
||||
|
||||
sub_status = SubplanStatus(
|
||||
subplan_id=_SUBPLAN_ID,
|
||||
action_name="local/sub-action",
|
||||
status=ProcessingState.COMPLETE,
|
||||
)
|
||||
patcher = patch.object(SubplanExecutionService, "execute_all")
|
||||
mock_exec_all = patcher.start()
|
||||
mock_exec_all.return_value = SubplanExecutionResult(
|
||||
all_succeeded=True,
|
||||
statuses=[sub_status],
|
||||
merge_result=None,
|
||||
total_duration_ms=5,
|
||||
failed_subplan_ids=[],
|
||||
)
|
||||
context._superb_patcher = patcher
|
||||
context._mock_execute_all = mock_exec_all
|
||||
|
||||
context.executor = executor
|
||||
context.lcs = lcs
|
||||
context.mock_subplan_svc = mock_subplan_svc
|
||||
|
||||
|
||||
@given("a child plan ready for execution in the lifecycle service")
|
||||
def step_given_child_plan_ready(context: Context) -> None:
|
||||
child_plan = Plan(
|
||||
identity=PlanIdentity(
|
||||
plan_id=_SUBPLAN_ID,
|
||||
parent_plan_id=_PLAN_ID,
|
||||
root_plan_id=_ROOT_ID,
|
||||
),
|
||||
namespaced_name=NamespacedName(namespace="local", name="child-plan"),
|
||||
description="Child plan for testing",
|
||||
action_name="local/sub-action",
|
||||
phase=PlanPhase.STRATEGIZE,
|
||||
processing_state=ProcessingState.QUEUED,
|
||||
definition_of_done="- [ ] Step one",
|
||||
)
|
||||
context.lcs.get_plan.side_effect = lambda pid: (
|
||||
child_plan if pid == _SUBPLAN_ID else context.plan
|
||||
)
|
||||
context.child_plan = child_plan
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("SubplanExecutionService.execute_all should have been called via lazy creation")
|
||||
def step_then_execute_all_called_via_lazy(context: Context) -> None:
|
||||
try:
|
||||
context._mock_execute_all.assert_called()
|
||||
finally:
|
||||
context._superb_patcher.stop()
|
||||
@@ -0,0 +1,601 @@
|
||||
"""Step definitions for plan_executor_child_plan_execution.feature.
|
||||
|
||||
Tests that cover previously uncovered code paths:
|
||||
- _execute_child_plan (success, circular, not-found, no-decisions, exception)
|
||||
- _execute_subplans fallback (lines 521-527, 532)
|
||||
- _apply_subplan_results_to_plan exec_result=None (lines 569-570)
|
||||
- Property accessors (lines 433, 438, 443)
|
||||
- _try_create_checkpoint PlanError re-raise (lines 731-741, 745)
|
||||
"""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from ulid import ULID
|
||||
|
||||
from cleveragents.application.services.plan_executor import (
|
||||
PlanExecutor,
|
||||
)
|
||||
from cleveragents.application.services.subplan_service import (
|
||||
SpawnResult,
|
||||
)
|
||||
from cleveragents.core.exceptions import PlanError
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
NamespacedName,
|
||||
Plan,
|
||||
PlanIdentity,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
SubplanStatus,
|
||||
)
|
||||
from cleveragents.infrastructure.sandbox.checkpoint import (
|
||||
CheckpointManager,
|
||||
SandboxCheckpoint,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PLAN_ID = str(ULID())
|
||||
_SUBPLAN_ID = str(ULID())
|
||||
_ROOT_PLAN_ID = str(ULID())
|
||||
|
||||
|
||||
def _make_subplan_status(
|
||||
subplan_id: str | None = None,
|
||||
action_name: str = "local/child-action",
|
||||
) -> SubplanStatus:
|
||||
return SubplanStatus(
|
||||
subplan_id=subplan_id or _SUBPLAN_ID,
|
||||
action_name=action_name,
|
||||
)
|
||||
|
||||
|
||||
def _make_plan(
|
||||
plan_id: str | None = None,
|
||||
phase: PlanPhase = PlanPhase.STRATEGIZE,
|
||||
state: ProcessingState = ProcessingState.QUEUED,
|
||||
definition_of_done: str | None = "Do the thing",
|
||||
) -> Plan:
|
||||
return Plan(
|
||||
identity=PlanIdentity(
|
||||
plan_id=plan_id or _PLAN_ID,
|
||||
root_plan_id=_ROOT_PLAN_ID,
|
||||
),
|
||||
namespaced_name=NamespacedName(namespace="local", name="test-child-plan"),
|
||||
description="Test child plan",
|
||||
action_name="local/child-action",
|
||||
definition_of_done=definition_of_done,
|
||||
phase=phase,
|
||||
processing_state=state,
|
||||
)
|
||||
|
||||
|
||||
def _make_spawn_result(
|
||||
num_statuses: int = 1,
|
||||
) -> SpawnResult:
|
||||
statuses = [
|
||||
SubplanStatus(
|
||||
subplan_id=str(ULID()),
|
||||
action_name="local/spawned-action",
|
||||
)
|
||||
for i in range(num_statuses)
|
||||
]
|
||||
return SpawnResult(
|
||||
spawned_statuses=statuses,
|
||||
metadata={},
|
||||
total_spawned=num_statuses,
|
||||
execution_mode="sequential",
|
||||
child_plans=[],
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: _execute_child_plan success
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_child_plan_execution_lifecycle(
|
||||
definition_of_done: str | None = "Do the thing",
|
||||
) -> MagicMock:
|
||||
"""Build a mock lifecycle that returns plans in the correct phase order.
|
||||
|
||||
``_execute_child_plan`` calls ``run_strategize`` (needs STRATEGIZE plan)
|
||||
and then ``run_execute`` (needs EXECUTE plan). Each of those calls
|
||||
``get_plan`` multiple times. We use a counter-based side_effect to
|
||||
return STRATEGIZE plans for the first 3 calls, then EXECUTE plans with
|
||||
decision_root_id for the remaining calls.
|
||||
"""
|
||||
lcs = MagicMock()
|
||||
call_counter = {"count": 0}
|
||||
|
||||
def _get_plan_side_effect(plan_id: str) -> Any:
|
||||
count = call_counter["count"]
|
||||
call_counter["count"] = count + 1
|
||||
if count < 3:
|
||||
return _make_plan(
|
||||
phase=PlanPhase.STRATEGIZE,
|
||||
state=ProcessingState.QUEUED,
|
||||
definition_of_done=definition_of_done,
|
||||
)
|
||||
p = _make_plan(
|
||||
phase=PlanPhase.EXECUTE,
|
||||
state=ProcessingState.QUEUED,
|
||||
definition_of_done=definition_of_done,
|
||||
)
|
||||
p.decision_root_id = str(ULID())
|
||||
return p
|
||||
|
||||
lcs.get_plan.side_effect = _get_plan_side_effect
|
||||
lcs.start_strategize = MagicMock()
|
||||
lcs.complete_strategize = MagicMock()
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor for child plan execution")
|
||||
def step_given_child3_executor_for_child_plan(context: Context) -> None:
|
||||
lcs = _make_child_plan_execution_lifecycle()
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
context.child3_plan = _make_plan(phase=PlanPhase.STRATEGIZE)
|
||||
|
||||
|
||||
@given("child3 a SubplanStatus for a valid child plan")
|
||||
def step_given_child3_subplan_status(context: Context) -> None:
|
||||
context.child3_status = _make_subplan_status()
|
||||
|
||||
|
||||
@given("I child3 pre-add the subplan_id to running_plan_ids")
|
||||
def step_given_child3_pre_add_running(context: Context) -> None:
|
||||
context.child3_executor._running_plan_ids.add(_SUBPLAN_ID)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: plan not found
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor with lifecycle returning None for child plan")
|
||||
def step_given_child3_executor_lifecycle_returns_none(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
lcs.get_plan.return_value = None
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: strategize produces no decisions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor for child plan with empty definition")
|
||||
def step_given_child3_executor_empty_definition(context: Context) -> None:
|
||||
lcs = _make_child_plan_execution_lifecycle(definition_of_done="")
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
# Force run_strategize to return a result with no decisions so that
|
||||
# the "no decisions" error path in _execute_child_plan is exercised.
|
||||
from cleveragents.application.services.plan_executor import StrategizeResult
|
||||
|
||||
empty_result = StrategizeResult(
|
||||
decision_root_id="",
|
||||
decisions=[],
|
||||
)
|
||||
executor.run_strategize = MagicMock(return_value=empty_result)
|
||||
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
context.child3_plan = _make_plan(phase=PlanPhase.STRATEGIZE, definition_of_done="")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: exception during execution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor whose lifecycle start_strategize raises RuntimeError")
|
||||
def step_given_child3_executor_start_strategize_raises(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
plan = _make_plan(phase=PlanPhase.STRATEGIZE)
|
||||
lcs.get_plan.return_value = plan
|
||||
lcs.start_strategize.side_effect = RuntimeError("boom from start_strategize")
|
||||
lcs.fail_strategize = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: _execute_subplans fallback
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor with subplan_service but no subplan_execution_service")
|
||||
def step_given_child3_executor_subplan_svc_only(context: Context) -> None:
|
||||
lcs = _make_child_plan_execution_lifecycle()
|
||||
mock_subplan_svc = MagicMock()
|
||||
mock_subplan_svc.get_spawn_decisions.return_value = []
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
subplan_service=mock_subplan_svc,
|
||||
)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
context.child3_mock_subplan_svc = mock_subplan_svc
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor with no subplan services at all")
|
||||
def step_given_child3_executor_no_subplan_services(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor with subplan_service and success executor")
|
||||
def step_given_child3_executor_subplan_svc_and_exec(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
plan = _make_plan()
|
||||
lcs.get_plan.return_value = plan
|
||||
lcs._commit_plan = MagicMock()
|
||||
|
||||
mock_subplan_svc = MagicMock()
|
||||
mock_subplan_svc.get_spawn_decisions.return_value = []
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
subplan_service=mock_subplan_svc,
|
||||
)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
context.child3_mock_subplan_svc = mock_subplan_svc
|
||||
|
||||
|
||||
@given("child3 a SpawnResult with one spawned status")
|
||||
def step_given_child3_spawn_result_one(context: Context) -> None:
|
||||
context.child3_spawn_result = _make_spawn_result(num_statuses=1)
|
||||
|
||||
|
||||
@given("child3 a SpawnResult with no spawned statuses")
|
||||
def step_given_child3_spawn_result_empty(context: Context) -> None:
|
||||
context.child3_spawn_result = _make_spawn_result(num_statuses=0)
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor for apply subplan results")
|
||||
def step_given_child3_executor_for_apply(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
executor = PlanExecutor(lifecycle_service=lcs)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: property accessors
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a child3 PlanExecutor with all optional services configured")
|
||||
def step_given_child3_executor_with_all_optional(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
mock_fix_orch = MagicMock()
|
||||
mock_subplan_svc = MagicMock()
|
||||
mock_exec_svc = MagicMock()
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
fix_revalidate_orchestrator=mock_fix_orch,
|
||||
subplan_service=mock_subplan_svc,
|
||||
subplan_execution_service=mock_exec_svc,
|
||||
)
|
||||
context.child3_executor = executor
|
||||
context.child3_mock_fix_orch = mock_fix_orch
|
||||
context.child3_mock_subplan_svc = mock_subplan_svc
|
||||
context.child3_mock_exec_svc = mock_exec_svc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given: checkpoint PlanError
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
"a child3 PlanExecutor with checkpoint manager and lifecycle that fails "
|
||||
"on commit_plan"
|
||||
)
|
||||
def step_given_child3_checkpoint_planerror(context: Context) -> None:
|
||||
lcs = MagicMock()
|
||||
plan = _make_plan()
|
||||
lcs.get_plan.return_value = plan
|
||||
lcs.commit_plan.side_effect = RuntimeError("commit failed")
|
||||
|
||||
mock_checkpoint_mgr = MagicMock(spec=CheckpointManager)
|
||||
mock_checkpoint = MagicMock(spec=SandboxCheckpoint)
|
||||
mock_checkpoint.checkpoint_id = "CKPT01"
|
||||
mock_checkpoint_mgr.create_checkpoint.return_value = mock_checkpoint
|
||||
|
||||
# We need the sandbox to be resolvable so we enter the try block
|
||||
# Use execution_context to provide a sandbox manager
|
||||
exec_ctx = MagicMock()
|
||||
exec_ctx.changeset_store = MagicMock()
|
||||
mock_sandbox = MagicMock()
|
||||
mock_sandbox.context = MagicMock()
|
||||
mock_sandbox.context.sandbox_root = "/tmp/sandbox_root"
|
||||
mock_sandbox_mgr = MagicMock()
|
||||
mock_sandbox_mgr.get_sandboxes.return_value = [mock_sandbox]
|
||||
exec_ctx.sandbox_manager = mock_sandbox_mgr
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=lcs,
|
||||
checkpoint_manager=mock_checkpoint_mgr,
|
||||
execution_context=exec_ctx,
|
||||
)
|
||||
context.child3_executor = executor
|
||||
context.child3_lcs = lcs
|
||||
context.child3_checkpoint_mgr = mock_checkpoint_mgr
|
||||
|
||||
|
||||
@given("child3 a sandbox that returns a valid checkpoint")
|
||||
def step_given_child3_sandbox_valid_checkpoint(context: Context) -> None:
|
||||
pass # Already configured in the previous step
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I child3 call _execute_child_plan on the executor")
|
||||
def step_when_child3_call_execute_child_plan(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
status: SubplanStatus = context.child3_status
|
||||
try:
|
||||
context.child3_output = executor._execute_child_plan(status)
|
||||
context.child3_child_error = None
|
||||
except Exception as exc:
|
||||
context.child3_output = None
|
||||
context.child3_child_error = exc
|
||||
|
||||
|
||||
@when("I child3 call _execute_subplans on the executor")
|
||||
def step_when_child3_call_execute_subplans(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
spawn_result: SpawnResult = context.child3_spawn_result
|
||||
plan = getattr(context, "child3_plan", None) or _make_plan()
|
||||
try:
|
||||
context.child3_exec_result = executor._execute_subplans(plan, spawn_result)
|
||||
context.child3_child_error = None
|
||||
except Exception as exc:
|
||||
context.child3_exec_result = None
|
||||
context.child3_child_error = exc
|
||||
|
||||
|
||||
@when("I child3 call _apply_subplan_results_to_plan with exec_result None")
|
||||
def step_when_child3_call_apply_subplan_results(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
spawn_result: SpawnResult = context.child3_spawn_result
|
||||
plan = _make_plan()
|
||||
try:
|
||||
executor._apply_subplan_results_to_plan(plan, spawn_result, None)
|
||||
context.child3_apply_error = None
|
||||
except Exception as exc:
|
||||
context.child3_apply_error = exc
|
||||
context.child3_plan = plan
|
||||
|
||||
|
||||
@when("I child3 attempt _try_create_checkpoint expecting PlanError")
|
||||
def step_when_child3_try_create_checkpoint_planerror(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
context.child3_checkpoint_error = None
|
||||
try:
|
||||
executor._try_create_checkpoint(
|
||||
plan_id=_PLAN_ID,
|
||||
phase="pre_execute",
|
||||
)
|
||||
except PlanError as exc:
|
||||
context.child3_checkpoint_error = exc
|
||||
except Exception as exc:
|
||||
context.child3_checkpoint_error = exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: _execute_child_plan success
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionOutput should indicate success")
|
||||
def step_then_child3_output_success(context: Context) -> None:
|
||||
assert context.child3_child_error is None, (
|
||||
f"Unexpected error: {context.child3_child_error}"
|
||||
)
|
||||
assert context.child3_output is not None, "Expected output but got None"
|
||||
assert context.child3_output.success is True, (
|
||||
f"Expected success, got: {context.child3_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionOutput should have a changeset_summary")
|
||||
def step_then_child3_output_has_summary(context: Context) -> None:
|
||||
assert context.child3_output.changeset_summary is not None, (
|
||||
"Expected changeset_summary to be set"
|
||||
)
|
||||
|
||||
|
||||
@then("child3 the running_plan_ids should be empty")
|
||||
def step_then_child3_running_plan_ids_empty(context: Context) -> None:
|
||||
assert len(context.child3_executor._running_plan_ids) == 0, (
|
||||
f"Expected empty running_plan_ids, got: {context.child3_executor._running_plan_ids}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: circular detection
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionOutput should indicate failure")
|
||||
def step_then_child3_output_failure(context: Context) -> None:
|
||||
assert context.child3_output is not None, "Expected output but got None"
|
||||
assert context.child3_output.success is False, (
|
||||
f"Expected failure, got: {context.child3_output}"
|
||||
)
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionOutput error should mention circular")
|
||||
def step_then_child3_output_error_circular(context: Context) -> None:
|
||||
assert context.child3_output.error is not None, "Expected error but got None"
|
||||
assert "circular" in context.child3_output.error.lower(), (
|
||||
f"Expected error to mention 'circular', got: {context.child3_output.error}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: plan not found
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('child3 the SubplanExecutionOutput error should mention "not found"')
|
||||
def step_then_child3_output_error_not_found(context: Context) -> None:
|
||||
assert context.child3_output.error is not None, "Expected error but got None"
|
||||
assert "not found" in context.child3_output.error.lower(), (
|
||||
f"Expected error to mention 'not found', got: {context.child3_output.error}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: strategize produces no decisions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('child3 the SubplanExecutionOutput error should mention "no decisions"')
|
||||
def step_then_child3_output_error_no_decisions(context: Context) -> None:
|
||||
assert context.child3_output.error is not None, "Expected error but got None"
|
||||
assert "no decisions" in context.child3_output.error.lower(), (
|
||||
f"Expected error to mention 'no decisions', got: {context.child3_output.error}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: exception during execution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionOutput error should contain the exception message")
|
||||
def step_then_child3_output_error_contains_exception(context: Context) -> None:
|
||||
assert context.child3_output.error is not None, "Expected error but got None"
|
||||
assert "boom from start_strategize" in context.child3_output.error, (
|
||||
f"Expected error to contain exception message, got: {context.child3_output.error}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: _execute_subplans
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionResult should not be None")
|
||||
def step_then_child3_exec_result_not_none(context: Context) -> None:
|
||||
assert context.child3_child_error is None, (
|
||||
f"Unexpected error: {context.child3_child_error}"
|
||||
)
|
||||
assert context.child3_exec_result is not None, (
|
||||
"Expected SubplanExecutionResult but got None"
|
||||
)
|
||||
|
||||
|
||||
@then("child3 the SubplanExecutionResult should be None")
|
||||
def step_then_child3_exec_result_none(context: Context) -> None:
|
||||
assert context.child3_child_error is None, (
|
||||
f"Unexpected error: {context.child3_child_error}"
|
||||
)
|
||||
assert context.child3_exec_result is None, (
|
||||
f"Expected None but got: {context.child3_exec_result}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: _apply_subplan_results
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("child3 the plan subplan_statuses should match the spawn result statuses")
|
||||
def step_then_child3_plan_statuses_match_spawn(context: Context) -> None:
|
||||
assert context.child3_apply_error is None, (
|
||||
f"Unexpected error: {context.child3_apply_error}"
|
||||
)
|
||||
plan = context.child3_plan
|
||||
spawn_result: SpawnResult = context.child3_spawn_result
|
||||
assert plan.subplan_statuses is not None, "Expected subplan_statuses to be set"
|
||||
assert len(plan.subplan_statuses) == len(spawn_result.spawned_statuses), (
|
||||
f"Expected {len(spawn_result.spawned_statuses)} statuses, "
|
||||
f"got {len(plan.subplan_statuses)}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: property accessors
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then(
|
||||
"child3 the fix_revalidate_orchestrator property should return the "
|
||||
"configured instance"
|
||||
)
|
||||
def step_then_child3_fix_orch_property(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
result = executor.fix_revalidate_orchestrator
|
||||
assert result is context.child3_mock_fix_orch, (
|
||||
f"Expected {context.child3_mock_fix_orch}, got {result}"
|
||||
)
|
||||
|
||||
|
||||
@then("child3 the subplan_service property should return the configured instance")
|
||||
def step_then_child3_subplan_svc_property(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
result = executor.subplan_service
|
||||
assert result is context.child3_mock_subplan_svc, (
|
||||
f"Expected {context.child3_mock_subplan_svc}, got {result}"
|
||||
)
|
||||
|
||||
|
||||
@then(
|
||||
"child3 the subplan_execution_service property should return the "
|
||||
"configured instance"
|
||||
)
|
||||
def step_then_child3_exec_svc_property(context: Context) -> None:
|
||||
executor: PlanExecutor = context.child3_executor
|
||||
result = executor.subplan_execution_service
|
||||
assert result is context.child3_mock_exec_svc, (
|
||||
f"Expected {context.child3_mock_exec_svc}, got {result}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then: checkpoint PlanError
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("a child3 PlanError should be raised about persisting checkpoint metadata")
|
||||
def step_then_child3_checkpoint_planerror(context: Context) -> None:
|
||||
assert context.child3_checkpoint_error is not None, (
|
||||
"Expected PlanError but none raised"
|
||||
)
|
||||
assert isinstance(context.child3_checkpoint_error, PlanError), (
|
||||
f"Expected PlanError, got: {type(context.child3_checkpoint_error)}"
|
||||
)
|
||||
assert "checkpoint" in str(context.child3_checkpoint_error).lower(), (
|
||||
f"Expected error about checkpoints, got: {context.child3_checkpoint_error}"
|
||||
)
|
||||
@@ -80,7 +80,7 @@ def _cov2_make_lifecycle(plan: Any | None = None) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
@@ -618,10 +618,10 @@ def step_cov2_check_complete_strategize(context: Context) -> None:
|
||||
)
|
||||
|
||||
|
||||
@then("the cov2 lifecycle should have called _commit_plan")
|
||||
def step_cov2_check_commit_plan(context: Context) -> None:
|
||||
"""Verify _commit_plan was called."""
|
||||
assert context.cov2_lifecycle._commit_plan.called
|
||||
@then("the cov2 lifecycle should have called commit_plan")
|
||||
def step_cov2_checkcommit_plan(context: Context) -> None:
|
||||
"""Verify commit_plan was called."""
|
||||
assert context.cov2_lifecycle.commit_plan.called
|
||||
|
||||
|
||||
@then("the cov2 execution context decision_root_id should be set")
|
||||
|
||||
@@ -75,7 +75,7 @@ def _edge3_make_lifecycle(plan: Any | None = None) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ def _make_lifecycle(**overrides: Any) -> MagicMock:
|
||||
lc.start_execute = MagicMock()
|
||||
lc.complete_execute = MagicMock()
|
||||
lc.fail_execute = MagicMock()
|
||||
lc._commit_plan = MagicMock()
|
||||
lc.commit_plan = MagicMock()
|
||||
for key, value in overrides.items():
|
||||
setattr(lc, key, value)
|
||||
return lc
|
||||
|
||||
@@ -153,7 +153,7 @@ def _make_lifecycle(plan: Plan) -> MagicMock:
|
||||
lcs.start_execute = MagicMock()
|
||||
lcs.complete_execute = MagicMock()
|
||||
lcs.fail_execute = MagicMock()
|
||||
lcs._commit_plan = MagicMock()
|
||||
lcs.commit_plan = MagicMock()
|
||||
return lcs
|
||||
|
||||
|
||||
@@ -485,8 +485,8 @@ def step_then_execute_all_called(context: Context) -> None:
|
||||
|
||||
@then("the parent plan subplan_statuses should be updated")
|
||||
def step_then_subplan_statuses_updated(context: Context) -> None:
|
||||
"""Verify the plan's subplan_statuses were updated via _commit_plan."""
|
||||
context.lcs._commit_plan.assert_called()
|
||||
"""Verify the plan's subplan_statuses were updated via commit_plan."""
|
||||
context.lcs.commit_plan.assert_called()
|
||||
|
||||
|
||||
@then("SubplanService.spawn should NOT have been called")
|
||||
@@ -523,10 +523,10 @@ def step_then_no_subplan_spawning(context: Context) -> None:
|
||||
def step_then_error_details_has_failed_ids(context: Context) -> None:
|
||||
"""Verify error_details contains failed_subplan_ids."""
|
||||
assert context.execute_error is None, f"run_execute raised: {context.execute_error}"
|
||||
# The _commit_plan call should have been made with a plan that has
|
||||
# The commit_plan call should have been made with a plan that has
|
||||
# error_details containing failed_subplan_ids
|
||||
commit_calls = context.lcs._commit_plan.call_args_list
|
||||
assert commit_calls, "Expected _commit_plan to be called"
|
||||
commit_calls = context.lcs.commit_plan.call_args_list
|
||||
assert commit_calls, "Expected commit_plan to be called"
|
||||
# Find the call where error_details was set with failed_subplan_ids
|
||||
found = False
|
||||
for c in commit_calls:
|
||||
@@ -537,14 +537,14 @@ def step_then_error_details_has_failed_ids(context: Context) -> None:
|
||||
break
|
||||
assert found, (
|
||||
"Expected error_details to contain 'failed_subplan_ids' in a "
|
||||
f"_commit_plan call. Calls: {commit_calls}"
|
||||
f"commit_plan call. Calls: {commit_calls}"
|
||||
)
|
||||
|
||||
|
||||
@then("the parent plan error_details should contain subplan_execution_failed true")
|
||||
def step_then_error_details_has_exec_failed(context: Context) -> None:
|
||||
"""Verify error_details contains subplan_execution_failed=true."""
|
||||
commit_calls = context.lcs._commit_plan.call_args_list
|
||||
commit_calls = context.lcs.commit_plan.call_args_list
|
||||
found = False
|
||||
for c in commit_calls:
|
||||
committed_plan = c[0][0]
|
||||
@@ -554,5 +554,5 @@ def step_then_error_details_has_exec_failed(context: Context) -> None:
|
||||
break
|
||||
assert found, (
|
||||
"Expected error_details to contain 'subplan_execution_failed'='true' "
|
||||
f"in a _commit_plan call. Calls: {commit_calls}"
|
||||
f"in a commit_plan call. Calls: {commit_calls}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
"""Step definitions for plan_executor_tier_hydration.feature (#10938).
|
||||
|
||||
Verifies that PlanExecutor.run_strategize integrates tier hydration correctly:
|
||||
- Invoking hydrate_tiers_for_plan when tier_service, project_repository and
|
||||
resource_registry are wired in.
|
||||
- Skipping hydration when hot fragments already exist (cache path).
|
||||
- Catching hydration failures non-fatally so strategize continues.
|
||||
|
||||
Forgejo: #10938
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import datetime, UTC
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.application.services.plan_executor import (
|
||||
PlanExecutor,
|
||||
StrategizeResult,
|
||||
StrategyDecision,
|
||||
)
|
||||
from cleveragents.application.services.plan_executor import PlanPhase
|
||||
from cleveragents.domain.models.acms.tiers import TieredFragment
|
||||
from cleveragents.domain.models.core.plan import ProjectLink
|
||||
|
||||
utc = UTC
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_FAKE_PROJECTS = ["local/test-project-1", "local/test-project-2"]
|
||||
|
||||
|
||||
def _make_project_links(): # type: ignore[no-untyped-def]
|
||||
return [ProjectLink(project_name=name) for name in _FAKE_PROJECTS]
|
||||
|
||||
|
||||
def _make_mock_plan(definition_of_done="Build something", invariants=None): # type: ignore[no-untyped-def]
|
||||
return SimpleNamespace(
|
||||
project_links=_make_project_links(),
|
||||
definition_of_done=definition_of_done,
|
||||
phase=PlanPhase.STRATEGIZE,
|
||||
invariants=invariants or [],
|
||||
timestamps=SimpleNamespace(updated_at=datetime.now(tz=utc)),
|
||||
)
|
||||
|
||||
|
||||
def _make_tier(with_fragments=True): # type: ignore[no-untyped-def]
|
||||
"""Create a ContextTierService mock."""
|
||||
tier = MagicMock()
|
||||
if with_fragments:
|
||||
tier.get_hot_fragments.return_value = [
|
||||
TieredFragment(
|
||||
fragment_id=f"frag-{i}",
|
||||
content="file content",
|
||||
tier="hot", # type: ignore[arg-type]
|
||||
resource_id="01TEST00000000000000000001",
|
||||
project_name="local/test-project",
|
||||
token_count=50,
|
||||
metadata={"path": f"mod_{i}.py"},
|
||||
)
|
||||
for i in range(2)
|
||||
]
|
||||
else:
|
||||
tier.get_hot_fragments.return_value = []
|
||||
return tier
|
||||
|
||||
|
||||
def _make_executor(tier_svc=None): # type: ignore[no-untyped-def]
|
||||
"""Build a minimal PlanExecutor from pieces."""
|
||||
mock_lifecycle = MagicMock()
|
||||
mock_ace = MagicMock()
|
||||
# Provide a non-empty decisions list so that the "contains decisions"
|
||||
# Then step can assert len(decisions) > 0 (meaningful assertion, not just
|
||||
# isinstance check which passes even with decisions=[]).
|
||||
mock_decision = StrategyDecision(
|
||||
decision_id="01HQMOCK00000000000000001",
|
||||
step_text="Mock decision for test",
|
||||
sequence=0,
|
||||
)
|
||||
mock_ace.execute.return_value = StrategizeResult(
|
||||
decisions=[mock_decision],
|
||||
decision_root_id="01HQMOCK00000000000000001",
|
||||
)
|
||||
|
||||
executor = PlanExecutor(
|
||||
lifecycle_service=mock_lifecycle,
|
||||
strategize_actor=mock_ace,
|
||||
tier_service=tier_svc,
|
||||
project_repository=MagicMock(),
|
||||
resource_registry=MagicMock(),
|
||||
)
|
||||
|
||||
mock_plan = _make_mock_plan()
|
||||
mock_lifecycle.get_plan.return_value = mock_plan
|
||||
|
||||
return executor
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
"a mock PlanExecutor with tier_service, project_repository and resource_registry wired in"
|
||||
)
|
||||
def step_executor_with_services(context): # type: ignore[no-untyped-def]
|
||||
tier_svc = _make_tier(with_fragments=False)
|
||||
executor = _make_executor(tier_svc=tier_svc)
|
||||
|
||||
# M6 fix: patch hydrate_tiers_for_plan so we can assert it was called.
|
||||
patcher = patch(
|
||||
"cleveragents.application.services.plan_executor.hydrate_tiers_for_plan",
|
||||
)
|
||||
mock_fn = patcher.start()
|
||||
mock_fn._patcher = patcher # type: ignore[attr-defined]
|
||||
|
||||
context.pe_executor = executor # type: ignore[attr-defined]
|
||||
context.pe_tier_svc = tier_svc # type: ignore[attr-defined]
|
||||
context.pe_lazy_patc_hydrate = mock_fn # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a mock PlanExecutor with tier_service and plan_id already marked as hydrated")
|
||||
def step_executor_with_existing_fragments(context): # type: ignore[no-untyped-def]
|
||||
"""M1 fix: skip path is now based on plan_id in _hydrated_plan_ids, not
|
||||
on get_hot_fragments() returning non-empty (which caused cross-plan
|
||||
contamination)."""
|
||||
tier_svc = _make_tier(with_fragments=False)
|
||||
executor = _make_executor(tier_svc=tier_svc)
|
||||
# Pre-populate the plan_id that the When step will use.
|
||||
executor._hydrated_plan_ids.add("01HQZZZZZ_TEST") # type: ignore[attr-defined]
|
||||
|
||||
# M6 fix: patch hydrate_tiers_for_plan so we can assert it was NOT called.
|
||||
patcher = patch(
|
||||
"cleveragents.application.services.plan_executor.hydrate_tiers_for_plan",
|
||||
)
|
||||
mock_fn = patcher.start()
|
||||
mock_fn._patcher = patcher # type: ignore[attr-defined]
|
||||
|
||||
context.pe_executor = executor # type: ignore[attr-defined]
|
||||
context.pe_tier_svc = tier_svc # type: ignore[attr-defined]
|
||||
context.pe_lazy_patc_hydrate = mock_fn # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a mock PlanExecutor with tier_service but no pre-existing hot fragments")
|
||||
def step_executor_empty_tier(context): # type: ignore[no-untyped-def]
|
||||
"""Same as 'wired in' — plan_id not yet hydrated, so hydration runs."""
|
||||
step_executor_with_services(context)
|
||||
|
||||
|
||||
@given(
|
||||
"a mock PlanExecutor with tier_service, project_repository and resource_registry where hydrate fails with OSError",
|
||||
)
|
||||
def step_executor_hydrate_o_error(context): # type: ignore[no-untyped-def]
|
||||
tier_svc = _make_tier(with_fragments=False)
|
||||
|
||||
executor = _make_executor(tier_svc=tier_svc)
|
||||
|
||||
# Patch hydrate_tiers_for_plan to raise OSError inside run_strategize.
|
||||
# Store the mock itself (not the _patch object) so we can assert .called.
|
||||
patcher = patch(
|
||||
"cleveragents.application.services.plan_executor.hydrate_tiers_for_plan",
|
||||
side_effect=OSError("file system error"),
|
||||
)
|
||||
mock_fn = patcher.start()
|
||||
# Attach stop for cleanup (m5: patch leak fix)
|
||||
mock_fn._patcher = patcher # type: ignore[attr-defined]
|
||||
|
||||
context.pe_executor = executor # type: ignore[attr-defined]
|
||||
context.pe_lazy_patc_hydrate = mock_fn # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a mock PlanExecutor without tier_service")
|
||||
def step_executor_no_tier(context): # type: ignore[no-untyped-def]
|
||||
"""tier_service=None means hydration is never attempted."""
|
||||
executor = _make_executor(tier_svc=None)
|
||||
|
||||
context.pe_executor = executor # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a plan in the Strategize phase")
|
||||
def step_plan_in_strategize(context): # type: ignore[no-untyped-def]
|
||||
"""No-op — already created by previous Given steps."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I call run_strategize on that PlanExecutor")
|
||||
def step_call_run_strategize(context): # type: ignore[no-untyped-def]
|
||||
executor = context.pe_executor # type: ignore[attr-defined]
|
||||
result = executor.run_strategize(
|
||||
plan_id="01HQZZZZZ_TEST",
|
||||
stream_callback=None,
|
||||
)
|
||||
context.pe_result = result # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("hydrate_tiers_for_plan should have been called once")
|
||||
def step_verify_hydrate_called(context): # type: ignore[no-untyped-def]
|
||||
# M6 fix: all scenarios now patch hydrate_tiers_for_plan and store the
|
||||
# mock in context.pe_lazy_patc_hydrate — we assert directly on it instead
|
||||
# of checking tier_svc.get_hot_fragments (which was tautological).
|
||||
lazy_patch = getattr(context, "pe_lazy_patc_hydrate", None) # type: ignore[attr-defined]
|
||||
assert lazy_patch is not None, (
|
||||
"Expected hydrate_tiers_for_plan patch to be set in context"
|
||||
)
|
||||
assert lazy_patch.called, "Expected hydrate_tiers_for_plan to have been called once"
|
||||
|
||||
|
||||
@then(
|
||||
"hydrate_tiers_for_plan should not have been called because the plan was already hydrated"
|
||||
)
|
||||
def step_verify_hydrate_skipped(context): # type: ignore[no-untyped-def]
|
||||
# M6 fix: use the patch mock to assert hydrate was NOT called (plan_id
|
||||
# scoped cache prevented it).
|
||||
lazy_patch = getattr(context, "pe_lazy_patc_hydrate", None) # type: ignore[attr-defined]
|
||||
assert lazy_patch is not None, (
|
||||
"Expected hydrate_tiers_for_plan patch to be set in context"
|
||||
)
|
||||
assert not lazy_patch.called, (
|
||||
"Expected hydrate_tiers_for_plan to NOT have been called (plan_id already hydrated)"
|
||||
)
|
||||
|
||||
|
||||
@then("hydrate_tiers_for_plan should not have been called")
|
||||
def step_verify_hydrate_not_called(context): # type: ignore[no-untyped-def]
|
||||
# No-tier scenario: tier_service is None so the hydration code path is
|
||||
# never entered. No patch was set, so verify via executor state.
|
||||
if context.pe_executor is None: # type: ignore[attr-defined]
|
||||
return
|
||||
|
||||
tier_svc = context.pe_executor._tier_service # type: ignore[attr-defined]
|
||||
if tier_svc is None:
|
||||
pass # tier_service=None branch → hydration skipped, this is expected
|
||||
else:
|
||||
# Defensive: if tier_svc is wired but plan_id was hydrated, verify
|
||||
# hydration was not re-triggered.
|
||||
assert not tier_svc.get_hot_fragments.called, (
|
||||
"Expected get_hot_fragments to NOT be called when tier_service is None"
|
||||
)
|
||||
|
||||
|
||||
@then(
|
||||
"the strategy result should contain decisions (strategize succeeded despite hydration failure)"
|
||||
)
|
||||
def step_verify_strategize_result_ok(context): # type: ignore[no-untyped-def]
|
||||
# M6 fix: assert isinstance AND len(decisions) > 0 (previous assertion
|
||||
# passed even when decisions=[] which is meaningless)
|
||||
assert isinstance(context.pe_result, StrategizeResult), (
|
||||
f"Expected StrategizeResult, got {type(context.pe_result)}"
|
||||
)
|
||||
assert len(context.pe_result.decisions) > 0, (
|
||||
"Expected non-empty decisions from strategize"
|
||||
)
|
||||
|
||||
|
||||
@then("the plan executor strategize result should be a valid run_strategize response")
|
||||
def step_verify_exec_result_ok(context): # type: ignore[no-untyped-def]
|
||||
assert isinstance(context.pe_result, StrategizeResult), (
|
||||
f"Expected StrategizeResult, got {type(context.pe_result)}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scenario lifecycle hooks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def after_scenario(context, scenario): # type: ignore[no-untyped-def]
|
||||
"""Stop any started patchers to prevent mock patch leak into later scenarios.
|
||||
|
||||
m5 fix: patcher.start() was called in OSError/KeyError Given steps but
|
||||
patcher.stop() was never called, leaking patches into subsequent scenarios.
|
||||
The mock function stores a reference to its _patcher object so we can stop it.
|
||||
"""
|
||||
lazy_patch = getattr(context, "pe_lazy_patc_hydrate", None)
|
||||
if lazy_patch is not None and hasattr(lazy_patch, "_patcher"):
|
||||
lazy_patch._patcher.stop() # type: ignore[attr-defined]
|
||||
@@ -741,7 +741,7 @@ def step_invoke_execute_plan_auto_progressed(context):
|
||||
|
||||
@then("the lifecycle service should persist the plan overrides")
|
||||
def step_lifecycle_persists_overrides(context):
|
||||
context.lifecycle_service_mock._commit_plan.assert_called_once()
|
||||
context.lifecycle_service_mock.commit_plan.assert_called_once()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -5,7 +5,7 @@ exercised in one direction (True-only or False-only):
|
||||
|
||||
* **Line 100** — ``InvalidPhaseTransitionError.__init__``:
|
||||
``if not message:`` False branch (custom message provided).
|
||||
* **Line 216** — ``_commit_plan``:
|
||||
* **Line 216** — ``commit_plan``:
|
||||
``if self._persisted and self.unit_of_work is not None:`` True branch.
|
||||
* **Line 327** — ``create_action``:
|
||||
``if self._persisted …`` True branch.
|
||||
@@ -219,13 +219,13 @@ def step_r2_check_revert_message(context: Context) -> None:
|
||||
|
||||
|
||||
# ===================================================================
|
||||
# _commit_plan in persisted mode (line 216 True)
|
||||
# commit_plan in persisted mode (line 216 True)
|
||||
# ===================================================================
|
||||
|
||||
|
||||
@when("r2plc-I start strategize on the plan")
|
||||
def step_r2_start_strategize(context: Context) -> None:
|
||||
"""Start strategize — calls _commit_plan internally."""
|
||||
"""Start strategize — calls commit_plan internally."""
|
||||
context.r2_plan = context.r2_service.start_strategize(
|
||||
context.r2_plan.identity.plan_id
|
||||
)
|
||||
|
||||
@@ -884,3 +884,147 @@ def step_handle_correction_get_plan_fails_r4(context: Context) -> None:
|
||||
context.service._handle_correction_applied(event)
|
||||
except Exception as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# =================================================================
|
||||
# resolve_actor_provider_model — namespace/name → provider/model
|
||||
# Lines 731-764
|
||||
# =================================================================
|
||||
|
||||
|
||||
@given("the plan lifecycle service has a unit of work for actor resolution r4")
|
||||
def step_service_with_uow_for_actor_r4(context: Context) -> None:
|
||||
"""Create a service with a basic unit of work for actor resolution."""
|
||||
from unittest.mock import MagicMock
|
||||
from contextlib import contextmanager
|
||||
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
mock_uow = MagicMock()
|
||||
mock_ctx = MagicMock()
|
||||
mock_ctx.actors.get_by_name.return_value = None
|
||||
|
||||
@contextmanager
|
||||
def _fake_transaction():
|
||||
yield mock_ctx
|
||||
|
||||
mock_uow.transaction = _fake_transaction
|
||||
context.service = PlanLifecycleService(settings=settings, unit_of_work=mock_uow)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given(
|
||||
"the plan lifecycle service has a unit of work with a known actor "
|
||||
'"{actor_name}" using "{provider_model}" for r4'
|
||||
)
|
||||
def step_service_with_known_actor_r4(
|
||||
context: Context, actor_name: str, provider_model: str
|
||||
) -> None:
|
||||
"""Create a service where actor lookup succeeds for the given name."""
|
||||
from unittest.mock import MagicMock
|
||||
from contextlib import contextmanager
|
||||
|
||||
provider, model = provider_model.split("/", 1)
|
||||
mock_actor = MagicMock()
|
||||
mock_actor.provider = provider
|
||||
mock_actor.model = model
|
||||
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
mock_uow = MagicMock()
|
||||
mock_ctx = MagicMock()
|
||||
mock_ctx.actors.get_by_name.return_value = mock_actor
|
||||
|
||||
@contextmanager
|
||||
def _fake_transaction():
|
||||
yield mock_ctx
|
||||
|
||||
mock_uow.transaction = _fake_transaction
|
||||
context.service = PlanLifecycleService(settings=settings, unit_of_work=mock_uow)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given(
|
||||
'the plan lifecycle service has a unit of work without actor "{actor_name}" for r4'
|
||||
)
|
||||
def step_service_without_actor_r4(context: Context, actor_name: str) -> None:
|
||||
"""Create a service where actor lookup returns None."""
|
||||
from unittest.mock import MagicMock
|
||||
from contextlib import contextmanager
|
||||
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
mock_uow = MagicMock()
|
||||
mock_ctx = MagicMock()
|
||||
mock_ctx.actors.get_by_name.return_value = None
|
||||
|
||||
@contextmanager
|
||||
def _fake_transaction():
|
||||
yield mock_ctx
|
||||
|
||||
mock_uow.transaction = _fake_transaction
|
||||
context.service = PlanLifecycleService(settings=settings, unit_of_work=mock_uow)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given(
|
||||
"the plan lifecycle service has a unit of work that raises on actor lookup for r4"
|
||||
)
|
||||
def step_service_with_raising_actor_lookup_r4(context: Context) -> None:
|
||||
"""Create a service where actor lookup raises an exception."""
|
||||
from unittest.mock import MagicMock
|
||||
from contextlib import contextmanager
|
||||
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
mock_uow = MagicMock()
|
||||
mock_ctx = MagicMock()
|
||||
mock_ctx.actors.get_by_name.side_effect = RuntimeError("DB failure")
|
||||
|
||||
@contextmanager
|
||||
def _fake_transaction():
|
||||
yield mock_ctx
|
||||
|
||||
mock_uow.transaction = _fake_transaction
|
||||
context.service = PlanLifecycleService(settings=settings, unit_of_work=mock_uow)
|
||||
context.error = None
|
||||
|
||||
|
||||
@given("the plan lifecycle service has no unit of work for r4")
|
||||
def step_service_no_uow_r4(context: Context) -> None:
|
||||
"""Create a service with no unit_of_work set."""
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
unit_of_work=None,
|
||||
)
|
||||
context.error = None
|
||||
|
||||
|
||||
@when('I resolve actor provider model for "{actor_name}"')
|
||||
def step_resolve_actor_provider_model(context: Context, actor_name: str) -> None:
|
||||
"""Call resolve_actor_provider_model directly."""
|
||||
context.resolved_model = context.service.resolve_actor_provider_model(actor_name)
|
||||
|
||||
|
||||
@when('I resolve actor provider model for ""')
|
||||
def step_resolve_actor_provider_model_empty(context: Context) -> None:
|
||||
"""Call resolve_actor_provider_model with empty string."""
|
||||
context.resolved_model = context.service.resolve_actor_provider_model("")
|
||||
|
||||
|
||||
@then('the resolved actor provider model should be "{expected}"')
|
||||
def step_verify_resolved_provider_model(context: Context, expected: str) -> None:
|
||||
"""Verify the resolved provider/model string matches."""
|
||||
assert context.resolved_model == expected, (
|
||||
f"Expected '{expected}', got '{context.resolved_model}'"
|
||||
)
|
||||
|
||||
|
||||
@then("the resolved actor provider model should be None")
|
||||
def step_verify_resolved_provider_model_none(context: Context) -> None:
|
||||
"""Verify the resolved provider/model is None."""
|
||||
assert context.resolved_model is None, (
|
||||
f"Expected None, got '{context.resolved_model}'"
|
||||
)
|
||||
|
||||
@@ -3,9 +3,9 @@
|
||||
Targets lifecycle transition branches in ``plan_lifecycle_service.py``:
|
||||
|
||||
* Non-reusable ``use_action`` archiving (line 576-577).
|
||||
* ``execute_plan`` persisted mode (line 216 True via ``_commit_plan``).
|
||||
* ``apply_plan`` persisted mode (line 216 True via ``_commit_plan``).
|
||||
* ``cancel_plan`` persisted mode (line 216 True via ``_commit_plan``).
|
||||
* ``execute_plan`` persisted mode (line 216 True via ``commit_plan``).
|
||||
* ``apply_plan`` persisted mode (line 216 True via ``commit_plan``).
|
||||
* ``cancel_plan`` persisted mode (line 216 True via ``commit_plan``).
|
||||
* ``pause_plan`` / ``resume_plan`` persisted mode.
|
||||
|
||||
All step text uses the ``r2plc-`` prefix to avoid collisions with
|
||||
@@ -82,7 +82,7 @@ def step_r2_check_plan_created(context: Context) -> None:
|
||||
|
||||
|
||||
# ===================================================================
|
||||
# execute_plan persisted mode (line 216 True via _commit_plan)
|
||||
# execute_plan persisted mode (line 216 True via commit_plan)
|
||||
# ===================================================================
|
||||
|
||||
|
||||
@@ -119,7 +119,7 @@ def step_r2_check_execute_phase(context: Context) -> None:
|
||||
|
||||
|
||||
# ===================================================================
|
||||
# apply_plan persisted mode (line 216 True via _commit_plan)
|
||||
# apply_plan persisted mode (line 216 True via commit_plan)
|
||||
# ===================================================================
|
||||
|
||||
|
||||
@@ -155,7 +155,7 @@ def step_r2_check_apply_phase(context: Context) -> None:
|
||||
|
||||
|
||||
# ===================================================================
|
||||
# cancel_plan persisted mode (line 216 True via _commit_plan)
|
||||
# cancel_plan persisted mode (line 216 True via commit_plan)
|
||||
# ===================================================================
|
||||
|
||||
|
||||
|
||||
@@ -62,7 +62,7 @@ def step_create_coverage_boost_plan(context: Context) -> None:
|
||||
context.cb_lifecycle.start_strategize(plan_id)
|
||||
p = context.cb_lifecycle.get_plan(plan_id)
|
||||
p.decision_root_id = str(ULID())
|
||||
context.cb_lifecycle._commit_plan(p)
|
||||
context.cb_lifecycle.commit_plan(p)
|
||||
context.cb_lifecycle.complete_strategize(plan_id)
|
||||
context.cb_lifecycle.execute_plan(plan_id)
|
||||
context.cb_lifecycle.start_execute(plan_id)
|
||||
|
||||
@@ -79,7 +79,7 @@ def _create_plan_in_state(
|
||||
context.lifecycle_service.start_strategize(plan_id)
|
||||
p = context.lifecycle_service.get_plan(plan_id)
|
||||
p.decision_root_id = str(ULID())
|
||||
context.lifecycle_service._commit_plan(p)
|
||||
context.lifecycle_service.commit_plan(p)
|
||||
context.lifecycle_service.complete_strategize(plan_id)
|
||||
context.lifecycle_service.execute_plan(plan_id)
|
||||
context.lifecycle_service.start_execute(plan_id)
|
||||
@@ -87,7 +87,7 @@ def _create_plan_in_state(
|
||||
context.lifecycle_service.start_strategize(plan_id)
|
||||
p = context.lifecycle_service.get_plan(plan_id)
|
||||
p.decision_root_id = str(ULID())
|
||||
context.lifecycle_service._commit_plan(p)
|
||||
context.lifecycle_service.commit_plan(p)
|
||||
context.lifecycle_service.complete_strategize(plan_id)
|
||||
context.lifecycle_service.execute_plan(plan_id)
|
||||
context.lifecycle_service.start_execute(plan_id)
|
||||
@@ -96,7 +96,7 @@ def _create_plan_in_state(
|
||||
context.lifecycle_service.start_strategize(plan_id)
|
||||
p = context.lifecycle_service.get_plan(plan_id)
|
||||
p.decision_root_id = str(ULID())
|
||||
context.lifecycle_service._commit_plan(p)
|
||||
context.lifecycle_service.commit_plan(p)
|
||||
context.lifecycle_service.complete_strategize(plan_id)
|
||||
context.lifecycle_service.execute_plan(plan_id)
|
||||
context.lifecycle_service.start_execute(plan_id)
|
||||
@@ -108,7 +108,7 @@ def _create_plan_in_state(
|
||||
context.lifecycle_service.start_strategize(plan_id)
|
||||
p = context.lifecycle_service.get_plan(plan_id)
|
||||
p.decision_root_id = str(ULID())
|
||||
context.lifecycle_service._commit_plan(p)
|
||||
context.lifecycle_service.commit_plan(p)
|
||||
context.lifecycle_service.complete_strategize(plan_id)
|
||||
context.lifecycle_service.execute_plan(plan_id)
|
||||
context.lifecycle_service.start_execute(plan_id)
|
||||
@@ -117,7 +117,7 @@ def _create_plan_in_state(
|
||||
context.lifecycle_service.start_apply(plan_id)
|
||||
p2 = context.lifecycle_service.get_plan(plan_id)
|
||||
p2.processing_state = ProcessingState.PROCESSING
|
||||
context.lifecycle_service._commit_plan(p2)
|
||||
context.lifecycle_service.commit_plan(p2)
|
||||
context.lifecycle_service.constrain_apply(plan_id, "constraints violated")
|
||||
|
||||
context.plan_id = plan_id
|
||||
@@ -172,7 +172,7 @@ def step_resume_plan_action_phase(context: Context) -> None:
|
||||
_create_plan_in_state(context, PlanPhase.STRATEGIZE, ProcessingState.QUEUED)
|
||||
plan = context.lifecycle_service.get_plan(context.plan_id)
|
||||
plan.phase = PlanPhase.ACTION
|
||||
context.lifecycle_service._commit_plan(plan)
|
||||
context.lifecycle_service.commit_plan(plan)
|
||||
context.plan = plan
|
||||
|
||||
|
||||
|
||||
@@ -2713,10 +2713,17 @@ def step_check_no_applied_in_pending(context: Context) -> None:
|
||||
def _ensure_memory_database_url(context: Context) -> None:
|
||||
"""Ensure plan service uses an in-memory database for memory tests."""
|
||||
try:
|
||||
if context.plan_service.settings.database_url != "sqlite:///:memory:":
|
||||
context.plan_service.settings.database_url = "sqlite:///:memory:"
|
||||
if (
|
||||
context.plan_service.settings.database_url
|
||||
!= "sqlite:///file::memory:?cache=shared"
|
||||
):
|
||||
context.plan_service.settings.database_url = (
|
||||
"sqlite:///file::memory:?cache=shared"
|
||||
)
|
||||
except AttributeError:
|
||||
context.plan_service.settings = Settings(database_url="sqlite:///:memory:")
|
||||
context.plan_service.settings = Settings(
|
||||
database_url="sqlite:///file::memory:?cache=shared"
|
||||
)
|
||||
|
||||
|
||||
def _request_memory_service_for_session(
|
||||
|
||||
@@ -668,14 +668,15 @@ def step_register_agents_with_skill_tools_and_llm(context: Context) -> None:
|
||||
context.app._register_agents_from_config() # pylint: disable=protected-access
|
||||
|
||||
|
||||
@then("the LLM agent should remain a SimpleLLMAgent not a SimpleToolAgent")
|
||||
def step_llm_agent_not_converted(context: Context) -> None:
|
||||
from cleveragents.reactive.stream_router import SimpleLLMAgent, SimpleToolAgent
|
||||
@then("the LLM agent should be a ToolCallingAgent not a SimpleLLMAgent")
|
||||
def step_llm_agent_upgraded_to_tool_calling(context: Context) -> None:
|
||||
from cleveragents.reactive.stream_router import SimpleToolAgent
|
||||
from cleveragents.reactive.tool_agent import ToolCallingAgent
|
||||
|
||||
llm_agent = context.app.stream_router.agents.get("llm_actor")
|
||||
assert llm_agent is not None, "Expected llm_actor to be registered"
|
||||
assert isinstance(llm_agent, SimpleLLMAgent), (
|
||||
f"Expected SimpleLLMAgent but got {type(llm_agent).__name__}"
|
||||
assert isinstance(llm_agent, ToolCallingAgent), (
|
||||
f"Expected ToolCallingAgent but got {type(llm_agent).__name__}"
|
||||
)
|
||||
|
||||
tool_agent = context.app.stream_router.agents.get("tool_actor")
|
||||
|
||||
@@ -41,6 +41,7 @@ from cleveragents.application.services.strategy_actor import (
|
||||
resolve_strategy_actor,
|
||||
validate_no_cycles,
|
||||
)
|
||||
from cleveragents.application.services.strategy_resolution import _is_known_provider
|
||||
from cleveragents.core.exceptions import PlanError, ValidationError
|
||||
from cleveragents.domain.models.core.decision import DecisionType
|
||||
from cleveragents.domain.models.core.plan import InvariantSource, PlanInvariant
|
||||
@@ -583,6 +584,30 @@ def step_verify_sa_parsed_model(context, expected):
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _is_known_provider helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I check if "{provider}" is a known strategy provider')
|
||||
def step_check_known_strategy_provider(context, provider):
|
||||
context.sa_is_known = _is_known_provider(provider)
|
||||
|
||||
|
||||
@then("the strategy provider check should be true")
|
||||
def step_verify_provider_check_true(context):
|
||||
assert context.sa_is_known is True, (
|
||||
f"Expected provider to be known, got {context.sa_is_known}"
|
||||
)
|
||||
|
||||
|
||||
@then("the strategy provider check should be false")
|
||||
def step_verify_provider_check_false(context):
|
||||
assert context.sa_is_known is False, (
|
||||
f"Expected provider to NOT be known, got {context.sa_is_known}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cyclic dependency detection through LLM execute path (H8)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -6,7 +6,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
@@ -160,3 +161,115 @@ def step_then_tool_agent_operation_result(context: Context, expected: str) -> No
|
||||
assert context.op_result == expected, (
|
||||
f"Expected '{expected}', got '{context.op_result}'"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M5: SimpleLLMAgent._resolve_llm forwards options to create_llm (#11223).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a stream router llm agent with options block containing custom base url")
|
||||
def step_given_llm_agent_with_options(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_options",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "my-local-model",
|
||||
"options": {
|
||||
"openai_api_base": "http://localhost:8080/v1",
|
||||
"openai_api_key": "none",
|
||||
},
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs: dict[str, Any] = {}
|
||||
|
||||
|
||||
@given("a stream router llm agent without options block")
|
||||
def step_given_llm_agent_without_options(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_no_options",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs = {}
|
||||
|
||||
|
||||
@when("the stream router llm agent resolves the llm")
|
||||
def step_when_llm_agent_resolves(context: Context) -> None:
|
||||
mock_llm = MagicMock()
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def fake_create_llm(
|
||||
provider_type: Any = None, model_id: Any = None, **kwargs: Any
|
||||
) -> Any:
|
||||
captured["provider_type"] = provider_type
|
||||
captured["model_id"] = model_id
|
||||
captured["kwargs"] = kwargs
|
||||
return mock_llm
|
||||
|
||||
mock_registry = MagicMock()
|
||||
mock_registry.create_llm.side_effect = fake_create_llm
|
||||
|
||||
with patch(
|
||||
"cleveragents.reactive.stream_router.get_provider_registry",
|
||||
return_value=mock_registry,
|
||||
):
|
||||
context.llm_options_agent._resolve_llm()
|
||||
|
||||
context.create_llm_kwargs = captured
|
||||
|
||||
|
||||
@then("the llm constructor should have received the custom base url")
|
||||
def step_then_llm_received_custom_base_url(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
assert "openai_api_base" in kwargs, (
|
||||
f"Expected 'openai_api_base' in create_llm kwargs, got {kwargs}"
|
||||
)
|
||||
assert kwargs["openai_api_base"] == "http://localhost:8080/v1", (
|
||||
f"Expected openai_api_base='http://localhost:8080/v1', got {kwargs}"
|
||||
)
|
||||
# openai_api_key is routed through __api_key_sentinel so the
|
||||
# registry can distinguish explicit keys from environment defaults.
|
||||
assert kwargs.get("__api_key_sentinel") == "none", (
|
||||
f"Expected __api_key_sentinel='none' (from options.openai_api_key), "
|
||||
f"got {kwargs}"
|
||||
)
|
||||
|
||||
|
||||
@then("the llm constructor should not have received extra kwargs")
|
||||
def step_then_llm_no_extra_kwargs(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
# When neither options nor top-level params are present, nothing
|
||||
# should be forwarded to the LLM constructor.
|
||||
assert kwargs == {}, f"Expected empty kwargs in create_llm call, got {kwargs}"
|
||||
|
||||
|
||||
@given("a stream router llm agent with top-level temperature and options block")
|
||||
def step_given_llm_agent_with_precedence(context: Context) -> None:
|
||||
context.llm_options_agent = SimpleLLMAgent(
|
||||
"test_precedence",
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"temperature": 0.5,
|
||||
"options": {
|
||||
"temperature": 1.0,
|
||||
"max_tokens": 4096,
|
||||
},
|
||||
},
|
||||
)
|
||||
context.create_llm_kwargs: dict[str, Any] = {}
|
||||
|
||||
|
||||
@then("the llm constructor should have received the top-level temperature")
|
||||
def step_then_llm_received_top_level_temperature(context: Context) -> None:
|
||||
kwargs = context.create_llm_kwargs.get("kwargs", {})
|
||||
assert kwargs.get("temperature") == 0.5, (
|
||||
f"Expected temperature=0.5 (top-level precedence over options), got {kwargs}"
|
||||
)
|
||||
# max_tokens from options should still be forwarded.
|
||||
assert kwargs.get("max_tokens") == 4096, (
|
||||
f"Expected max_tokens=4096 from options, got {kwargs}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Step definitions for tdd_tui_prompt_input_live_refresh.feature.
|
||||
|
||||
Regression guard for issue #11249: _PromptTextInput must override
|
||||
virtual_size with layout=False to prevent full layout recalculations
|
||||
on every keystroke, which blocked Textual's WriterThread from flushing
|
||||
stdout and made typed characters invisible until Enter was pressed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then
|
||||
|
||||
|
||||
@given("the _PromptTextInput class is available")
|
||||
def step_prompt_text_input_available(context: Any) -> None:
|
||||
"""Import _PromptTextInput from the prompt module."""
|
||||
from cleveragents.tui.widgets.prompt import _PromptInputBase
|
||||
|
||||
context._prompt_input_cls = _PromptInputBase
|
||||
|
||||
|
||||
@then("its virtual_size reactive should have layout=False")
|
||||
def step_virtual_size_layout_false(context: Any) -> None:
|
||||
"""Assert that _PromptTextInput.virtual_size has layout=False.
|
||||
|
||||
Bug #11249: if layout=True, every keystroke triggers a full layout
|
||||
recalculation that keeps the WriterThread queue non-empty and prevents
|
||||
stdout from flushing, making typed characters invisible until Enter.
|
||||
"""
|
||||
cls = context._prompt_input_cls
|
||||
|
||||
if not hasattr(cls, "virtual_size"):
|
||||
# Fallback class (Textual not available) — skip, no reactive to check
|
||||
return
|
||||
|
||||
vs_reactive = cls.__dict__.get("virtual_size")
|
||||
assert vs_reactive is not None, (
|
||||
"Bug #11249 regression: _PromptTextInput must define its own "
|
||||
"virtual_size reactive (overriding Widget.virtual_size)."
|
||||
)
|
||||
assert not getattr(vs_reactive, "_layout", True), (
|
||||
"Bug #11249 regression: _PromptTextInput.virtual_size must have "
|
||||
"layout=False to prevent refresh(layout=True) on every keystroke. "
|
||||
"Characters will be invisible while typing if layout=True."
|
||||
)
|
||||
|
||||
|
||||
@given("the _TextualPromptInput class is available")
|
||||
def step_textual_prompt_input_available(context: Any) -> None:
|
||||
"""Import _TextualPromptInput (the Horizontal wrapper) from the prompt module."""
|
||||
from cleveragents.tui.widgets import prompt as prompt_mod
|
||||
|
||||
context._textual_prompt_cls = getattr(prompt_mod, "_TextualPromptInput", None)
|
||||
context._prompt_input_base = prompt_mod._PromptInputBase
|
||||
|
||||
|
||||
@then("its inner _input should be a _PromptTextInput instance")
|
||||
def step_inner_input_is_prompt_text_input(context: Any) -> None:
|
||||
"""Assert the inner Input widget is our subclass, not raw textual.Input.
|
||||
|
||||
Verifies that _TextualPromptInput uses _PromptInputBase (the subclass
|
||||
with layout=False virtual_size) rather than the raw _InputBase.
|
||||
"""
|
||||
cls = context._textual_prompt_cls
|
||||
if cls is None:
|
||||
return # Textual not available — skip
|
||||
|
||||
# Instantiate and check the _input type
|
||||
instance = cls.__new__(cls)
|
||||
# Access _input type via the class __init__ source check
|
||||
# (avoid actually instantiating which requires a running app)
|
||||
import inspect
|
||||
|
||||
src = inspect.getsource(cls.__init__)
|
||||
assert "_PromptInputBase" in src, (
|
||||
"Bug #11249 regression: _TextualPromptInput.__init__ must create "
|
||||
"_input using _PromptInputBase (not _InputBase) so the virtual_size "
|
||||
"layout=False override is active."
|
||||
)
|
||||
del instance # avoid unused warning
|
||||
|
||||
|
||||
@given("the TUI CSS file is loaded")
|
||||
def step_css_file_loaded(context: Any) -> None:
|
||||
"""Load the TUI CSS file content."""
|
||||
css_path = (
|
||||
Path(__file__).resolve().parents[2]
|
||||
/ "src"
|
||||
/ "cleveragents"
|
||||
/ "tui"
|
||||
/ "cleveragents.tcss"
|
||||
)
|
||||
context._css_content = css_path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
@then("the prompt Input rule should use a fixed height not auto")
|
||||
def step_prompt_input_fixed_height(context: Any) -> None:
|
||||
"""Assert the #prompt > Input CSS rule uses a fixed height, not height:auto.
|
||||
|
||||
Bug #11249: with ``height: auto``, ``Input.styles.auto_dimensions``
|
||||
returns True, causing ``_watch_value`` to call ``refresh(layout=True)``
|
||||
on every keystroke — a second source of layout recalculations that
|
||||
adds to the WriterThread queue pressure.
|
||||
"""
|
||||
css = context._css_content
|
||||
|
||||
# Find the #prompt > Input block
|
||||
import re
|
||||
|
||||
match = re.search(r"#prompt\s*>\s*Input\s*\{([^}]+)\}", css, re.DOTALL)
|
||||
assert match, "CSS must contain a '#prompt > Input' rule block."
|
||||
|
||||
block = match.group(1)
|
||||
assert "height: auto" not in block, (
|
||||
"Bug #11249 regression: '#prompt > Input' must not use 'height: auto'. "
|
||||
"Auto height sets auto_dimensions=True which triggers refresh(layout=True) "
|
||||
"on every keystroke. Use a fixed height (e.g. 'height: 1')."
|
||||
)
|
||||
assert re.search(r"height:\s*\d", block), (
|
||||
"Bug #11249 regression: '#prompt > Input' must have a fixed numeric "
|
||||
"height (e.g. 'height: 1') to prevent auto_dimensions layout refreshes."
|
||||
)
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Step definitions for TransportSelector coverage tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
|
||||
@given("no transport server URL is configured")
|
||||
def step_no_server_url(context: Context) -> None:
|
||||
"""Prepare a scenario with no server URL (local mode)."""
|
||||
context.transport_server_url = None
|
||||
|
||||
|
||||
@given('a server URL "{url}" is configured')
|
||||
def step_server_url_configured(context: Context, url: str) -> None:
|
||||
"""Prepare a scenario with a specific server URL."""
|
||||
context.transport_server_url = url
|
||||
|
||||
|
||||
@given("an empty string server URL is configured")
|
||||
def step_empty_server_url(context: Context) -> None:
|
||||
"""Prepare a scenario with an empty string as server URL."""
|
||||
context.transport_server_url = ""
|
||||
|
||||
|
||||
@when("I call TransportSelector.select with no server_url")
|
||||
def step_select_no_server_url(context: Context) -> None:
|
||||
"""Call TransportSelector.select() with None (no server_url)."""
|
||||
from cleveragents.a2a.transport_selector import TransportSelector
|
||||
|
||||
context.transport_result = TransportSelector.select(None)
|
||||
|
||||
|
||||
@when('I call TransportSelector.select with server_url ""')
|
||||
def step_select_empty_server_url(context: Context) -> None:
|
||||
"""Call TransportSelector.select() with an empty string URL (local/stdio mode)."""
|
||||
from cleveragents.a2a.transport_selector import TransportSelector
|
||||
|
||||
context.transport_result = TransportSelector.select("")
|
||||
|
||||
|
||||
@when('I call TransportSelector.select with server_url "{url}"')
|
||||
def step_select_with_server_url(context: Context, url: str) -> None:
|
||||
"""Call TransportSelector.select() with the given server URL."""
|
||||
from cleveragents.a2a.transport_selector import TransportSelector
|
||||
|
||||
context.transport_result = TransportSelector.select(url)
|
||||
|
||||
|
||||
@then("the returned transport should be an A2aStdioTransport instance")
|
||||
def step_transport_is_stdio(context: Context) -> None:
|
||||
"""Verify the result is an A2aStdioTransport instance."""
|
||||
from cleveragents.a2a.stdio_transport import A2aStdioTransport
|
||||
|
||||
assert isinstance(context.transport_result, A2aStdioTransport), (
|
||||
f"Expected A2aStdioTransport, got {type(context.transport_result).__qualname__}"
|
||||
)
|
||||
|
||||
|
||||
@then("the returned transport should be an A2aHttpTransport instance")
|
||||
def step_transport_is_http(context: Context) -> None:
|
||||
"""Verify the result is an A2aHttpTransport instance."""
|
||||
from cleveragents.a2a.transport import A2aHttpTransport
|
||||
|
||||
assert isinstance(context.transport_result, A2aHttpTransport), (
|
||||
f"Expected A2aHttpTransport, got {type(context.transport_result).__qualname__}"
|
||||
)
|
||||
@@ -485,6 +485,6 @@ def step_persona_bar_reflects_actor(context: object) -> None:
|
||||
from cleveragents.tui.widgets.persona_bar import PersonaBar
|
||||
|
||||
bar = context._first_run_app.query_one("#persona-bar", PersonaBar)
|
||||
assert "anthropic/claude-4-sonnet" in bar._text, (
|
||||
assert "anthropic/claude-sonnet-4-20250514" in bar._text, (
|
||||
f"Expected actor in persona bar, got: {bar._text}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,429 @@
|
||||
"""Step definitions for tui_llm_dispatch.feature.
|
||||
|
||||
Tests the TUI LLM dispatch wiring without requiring Textual or importlib.reload.
|
||||
|
||||
The core dispatch logic lives in the module-level ``_run_llm_dispatch()``
|
||||
function in ``tui/app.py``. This function is Textual-free and can be
|
||||
imported and tested directly.
|
||||
|
||||
Session creation and facade building helpers (``_create_tui_session`` and
|
||||
``_build_tui_facade`` in ``tui/commands.py``) are also tested directly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
|
||||
from cleveragents.tui.app import _format_worker_outcome, _run_llm_dispatch
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_mock_facade(response_text: str = "Mock LLM reply") -> MagicMock:
|
||||
"""Return a mock A2aLocalFacade returning a canned assistant message."""
|
||||
facade = MagicMock()
|
||||
response = MagicMock()
|
||||
response.error = None
|
||||
response.result = {"assistant_message": response_text}
|
||||
facade.dispatch.return_value = response
|
||||
return facade
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("the TUI app is initialised with a mock A2A facade and session")
|
||||
def step_init_with_facade(context: Any) -> None:
|
||||
"""Set up a mock facade and session_id on the context."""
|
||||
context._facade = _make_mock_facade()
|
||||
context._session_id = "test-session-001"
|
||||
|
||||
|
||||
@given("the TUI app is initialised with a facade that returns empty response")
|
||||
def step_init_with_empty_response_facade(context: Any) -> None:
|
||||
"""Set up a mock facade that returns empty assistant_message."""
|
||||
context._facade = _make_mock_facade(response_text="")
|
||||
context._session_id = "test-session-001"
|
||||
|
||||
|
||||
@given("the TUI app is initialised without a facade")
|
||||
def step_init_without_facade(context: Any) -> None:
|
||||
"""Simulate facade=None (preview fallback path)."""
|
||||
context._facade = None
|
||||
context._session_id = "default"
|
||||
|
||||
|
||||
@given("the facade raises SessionActorNotConfiguredError")
|
||||
def step_facade_raises_no_actor(context: Any) -> None:
|
||||
"""Configure the mock facade to raise SessionActorNotConfiguredError."""
|
||||
from cleveragents.domain.models.core.session import SessionActorNotConfiguredError
|
||||
|
||||
facade = MagicMock()
|
||||
facade.dispatch.side_effect = SessionActorNotConfiguredError("no actor")
|
||||
context._facade = facade
|
||||
context._session_id = "default"
|
||||
|
||||
|
||||
@given("the facade raises SessionNotFoundError")
|
||||
def step_facade_raises_session_not_found(context: Any) -> None:
|
||||
"""Configure the mock facade to raise SessionNotFoundError."""
|
||||
from cleveragents.domain.models.core.session import SessionNotFoundError
|
||||
|
||||
facade = MagicMock()
|
||||
facade.dispatch.side_effect = SessionNotFoundError("session not found")
|
||||
context._facade = facade
|
||||
context._session_id = "default"
|
||||
|
||||
|
||||
@given("the facade raises DatabaseError")
|
||||
def step_facade_raises_database_error(context: Any) -> None:
|
||||
"""Configure the mock facade to raise DatabaseError."""
|
||||
from cleveragents.core.exceptions import DatabaseError
|
||||
|
||||
facade = MagicMock()
|
||||
facade.dispatch.side_effect = DatabaseError("db locked")
|
||||
context._facade = facade
|
||||
context._session_id = "default"
|
||||
|
||||
|
||||
@given("the facade returns an A2A error response")
|
||||
def step_facade_returns_error(context: Any) -> None:
|
||||
"""Configure the mock facade to return an A2A error response."""
|
||||
facade = MagicMock()
|
||||
error = MagicMock()
|
||||
error.message = "internal server error"
|
||||
response = MagicMock()
|
||||
response.error = error
|
||||
response.result = None
|
||||
facade.dispatch.return_value = response
|
||||
context._facade = facade
|
||||
context._session_id = "default"
|
||||
|
||||
|
||||
@given('a mock session service that returns session_id "{sid}"')
|
||||
def step_mock_session_service(context: Any, sid: str) -> None:
|
||||
"""Set up a mock session service returning the given session_id."""
|
||||
session = MagicMock()
|
||||
session.session_id = sid
|
||||
service = MagicMock()
|
||||
service.create.return_value = session
|
||||
context._mock_service = service
|
||||
context._expected_sid = sid
|
||||
|
||||
|
||||
@given("the session service raises an exception")
|
||||
def step_session_service_raises(context: Any) -> None:
|
||||
"""Set up a mock session service that raises on create()."""
|
||||
service = MagicMock()
|
||||
service.create.side_effect = RuntimeError("db unavailable")
|
||||
context._mock_service = service
|
||||
context._expected_sid = "default"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I submit the text "{text}"')
|
||||
def step_submit_text(context: Any, text: str) -> None:
|
||||
"""Invoke _run_llm_dispatch directly with the given text."""
|
||||
context._submitted_text = text # store for parameterized assertions
|
||||
if context._facade is None:
|
||||
# Preview fallback path — no dispatch, just echo
|
||||
context._result_text = text
|
||||
context._app_transcript = []
|
||||
else:
|
||||
context._result_text = _run_llm_dispatch(
|
||||
context._facade, context._session_id, text
|
||||
)
|
||||
# Simulate what on_input_submitted does: pre-escape and store in transcript
|
||||
from rich.markup import escape as _esc
|
||||
|
||||
context._app_transcript = [_esc(f"You: {text}")]
|
||||
|
||||
|
||||
@when('I dispatch "{text}" overriding actor with "{actor}"')
|
||||
def step_dispatch_text_with_actor(context: Any, text: str, actor: str) -> None:
|
||||
"""Invoke _run_llm_dispatch with an explicit actor_override."""
|
||||
context._submitted_text = text
|
||||
context._result_text = _run_llm_dispatch(
|
||||
context._facade, context._session_id, text, actor_override=actor
|
||||
)
|
||||
|
||||
|
||||
@when("the worker completes with a KeyboardInterrupt error")
|
||||
def step_worker_keyboard_interrupt(context: Any) -> None:
|
||||
"""Simulate a worker that completed with a KeyboardInterrupt."""
|
||||
context._worker_error = KeyboardInterrupt("user interrupt")
|
||||
|
||||
|
||||
@when("the worker completes with a SystemExit error")
|
||||
def step_worker_system_exit(context: Any) -> None:
|
||||
"""Simulate a worker that completed with a SystemExit."""
|
||||
context._worker_error = SystemExit(1)
|
||||
|
||||
|
||||
@when("the worker completes with neither result nor error")
|
||||
def step_worker_no_result_no_error(context: Any) -> None:
|
||||
"""Simulate a worker that completed with both result and error as None."""
|
||||
context._worker_outcome = _format_worker_outcome(result=None, error=None)
|
||||
|
||||
|
||||
@when("_build_tui_facade is called with a broken container")
|
||||
def step_build_tui_facade_broken_container(context: Any) -> None:
|
||||
"""Call _build_tui_facade with a container that always raises."""
|
||||
from cleveragents.tui import commands as cmd_mod
|
||||
|
||||
cmd_mod._tui_session_service = None
|
||||
broken = MagicMock()
|
||||
broken.session_service.side_effect = RuntimeError("no db")
|
||||
with patch("cleveragents.a2a.cli_bootstrap.get_facade") as mock_facade:
|
||||
mock_facade.side_effect = ImportError("no a2a")
|
||||
context._facade_result = cmd_mod._build_tui_facade()
|
||||
cmd_mod._tui_session_service = None
|
||||
|
||||
|
||||
@when('the worker completes with error "{error_msg}"')
|
||||
def step_worker_completes_with_error(context: Any, error_msg: str) -> None:
|
||||
"""Simulate a Textual worker that failed with the given exception."""
|
||||
context._worker_outcome = _format_worker_outcome(
|
||||
result=None, error=RuntimeError(error_msg)
|
||||
)
|
||||
|
||||
|
||||
@when('the worker completes with result "{result}"')
|
||||
def step_worker_completes_with_result(context: Any, result: str) -> None:
|
||||
"""Simulate a Textual worker that succeeded with the given result."""
|
||||
context._worker_outcome = _format_worker_outcome(result=result, error=None)
|
||||
|
||||
|
||||
@given('a facade returning assistant text "{reply}"')
|
||||
def step_facade_returning_specific_reply(context: Any, reply: str) -> None:
|
||||
"""Set up a mock facade with a specific assistant reply."""
|
||||
context._facade = _make_mock_facade(response_text=reply)
|
||||
context._session_id = "test-session-001"
|
||||
|
||||
|
||||
@given('a persona state with active actor "{actor}"')
|
||||
def step_persona_state_with_actor(context: Any, actor: str) -> None:
|
||||
"""Set up a mock PersonaState whose active persona has the given actor."""
|
||||
persona = MagicMock()
|
||||
persona.actor = actor
|
||||
state = MagicMock()
|
||||
state.active_persona.return_value = persona
|
||||
context._mock_persona_state = state
|
||||
|
||||
|
||||
@when("I call _create_tui_session with that service")
|
||||
def step_call_create_tui_session(context: Any) -> None:
|
||||
"""Call _create_tui_session patching get_container, no persona state."""
|
||||
from cleveragents.tui import commands as cmd_mod
|
||||
|
||||
container = MagicMock()
|
||||
container.session_service.return_value = context._mock_service
|
||||
# Reset singleton so the patched container is used on this call.
|
||||
cmd_mod._tui_session_service = None
|
||||
with patch.object(cmd_mod, "get_container", return_value=container):
|
||||
context._result_sid = cmd_mod._create_tui_session()
|
||||
cmd_mod._tui_session_service = None # clean up after test
|
||||
|
||||
|
||||
@when("I call _create_tui_session with that service and persona state")
|
||||
def step_call_create_tui_session_with_persona(context: Any) -> None:
|
||||
"""Call _create_tui_session with a persona state so actor is bound."""
|
||||
from cleveragents.tui import commands as cmd_mod
|
||||
|
||||
container = MagicMock()
|
||||
container.session_service.return_value = context._mock_service
|
||||
# Reset singleton so the patched container is used on this call.
|
||||
cmd_mod._tui_session_service = None
|
||||
with patch.object(cmd_mod, "get_container", return_value=container):
|
||||
context._result_sid = cmd_mod._create_tui_session(
|
||||
persona_state=context._mock_persona_state
|
||||
)
|
||||
cmd_mod._tui_session_service = None # clean up after test
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the facade should have received a message/send request")
|
||||
def step_facade_received_request(context: Any) -> None:
|
||||
"""Assert facade.dispatch was called with method=message/send."""
|
||||
context._facade.dispatch.assert_called_once()
|
||||
request = context._facade.dispatch.call_args[0][0]
|
||||
assert request.method == "message/send", (
|
||||
f"Expected method='message/send', got {request.method!r}"
|
||||
)
|
||||
assert request.params["session_id"] == context._session_id
|
||||
assert request.params["message"] == context._submitted_text, (
|
||||
f"Expected message={context._submitted_text!r}, "
|
||||
f"got {request.params['message']!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the conversation should show the assistant response")
|
||||
def step_conversation_shows_response(context: Any) -> None:
|
||||
"""Assert the result contains the assistant response."""
|
||||
assert "Mock LLM reply" in context._result_text, (
|
||||
f"Expected assistant response in result, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the conversation should show the preview text "{expected}"')
|
||||
def step_conversation_shows_preview(context: Any, expected: str) -> None:
|
||||
"""Assert the result is just the echoed input text (no-facade path)."""
|
||||
assert expected in context._result_text, (
|
||||
f"Expected {expected!r} in result, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the conversation should show the no-actor error message")
|
||||
def step_conversation_shows_no_actor(context: Any) -> None:
|
||||
"""Assert the no-actor friendly message is returned."""
|
||||
assert "No actor configured" in context._result_text, (
|
||||
f"Expected no-actor message in result, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the conversation should show an error message")
|
||||
def step_conversation_shows_error(context: Any) -> None:
|
||||
"""Assert an [Error] tag is in the result."""
|
||||
assert "[Error]" in context._result_text, (
|
||||
f"Expected [Error] in result, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the returned session_id should be "{expected}"')
|
||||
def step_returned_session_id(context: Any, expected: str) -> None:
|
||||
"""Assert the session_id from _create_tui_session matches."""
|
||||
assert context._result_sid == expected, (
|
||||
f"Expected session_id={expected!r}, got {context._result_sid!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the facade should have received a message/send request with actor "{actor}"')
|
||||
def step_facade_received_request_with_actor(context: Any, actor: str) -> None:
|
||||
"""Assert the dispatch request included the given actor override."""
|
||||
context._facade.dispatch.assert_called_once()
|
||||
request = context._facade.dispatch.call_args[0][0]
|
||||
assert request.method == "message/send", (
|
||||
f"Expected method='message/send', got {request.method!r}"
|
||||
)
|
||||
assert request.params.get("actor") == actor, (
|
||||
f"Expected actor={actor!r} in params, got {request.params.get('actor')!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the conversation should show a session-not-found error message")
|
||||
def step_conversation_shows_session_not_found(context: Any) -> None:
|
||||
"""Assert the session-not-found friendly message is returned."""
|
||||
assert "Session not found" in context._result_text, (
|
||||
f"Expected session-not-found message, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the conversation should show a database error message")
|
||||
def step_conversation_shows_database_error(context: Any) -> None:
|
||||
"""Assert the database error message is returned."""
|
||||
assert "Database error" in context._result_text, (
|
||||
f"Expected database error message, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the formatted outcome should show an error message")
|
||||
def step_worker_outcome_shows_error(context: Any) -> None:
|
||||
"""Assert _format_worker_outcome produced an [Error] string."""
|
||||
assert context._worker_outcome is not None, "Expected non-None outcome"
|
||||
assert "[Error]" in context._worker_outcome, (
|
||||
f"Expected [Error] in outcome, got: {context._worker_outcome!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the formatted outcome should be "{expected}"')
|
||||
def step_worker_outcome_equals(context: Any, expected: str) -> None:
|
||||
"""Assert _format_worker_outcome returned the expected string."""
|
||||
assert context._worker_outcome == expected, (
|
||||
f"Expected {expected!r}, got {context._worker_outcome!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the conversation should show "(no response)"')
|
||||
def step_conversation_shows_no_response(context: Any) -> None:
|
||||
"""Assert the empty-response fallback text is present."""
|
||||
assert "(no response)" in context._result_text, (
|
||||
f"Expected '(no response)' in result, got: {context._result_text!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the transcript entry should have markup escaped")
|
||||
def step_transcript_entry_escaped(context: Any) -> None:
|
||||
"""Assert the transcript entry stored by on_input_submitted has escaped markup.
|
||||
|
||||
Verifies that _render_transcript receives pre-escaped entries — i.e. that
|
||||
on_input_submitted calls _escape() before storing in the transcript, not
|
||||
just at render time. This replaces the previous test that only validated
|
||||
escape() in isolation without checking the storage path.
|
||||
"""
|
||||
import re
|
||||
|
||||
transcript = context._app_transcript
|
||||
assert transcript, "Expected at least one transcript entry after submit"
|
||||
entry = transcript[-1] # the most recently appended entry
|
||||
|
||||
# The stored entry must NOT contain unescaped Rich markup tags.
|
||||
assert not re.search(r"(?<!\\)\[/?[a-zA-Z][^\]]*\]", entry), (
|
||||
f"Bug: transcript entry was stored WITHOUT escaping markup: {entry!r}"
|
||||
)
|
||||
# The text content must still be present after escaping.
|
||||
assert "attack" in entry, (
|
||||
f"Message content 'attack' must survive markup escaping: {entry!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("_format_worker_outcome should re-raise it")
|
||||
def step_format_worker_outcome_reraises(context: Any) -> None:
|
||||
"""Assert _format_worker_outcome re-raises BaseException subclasses."""
|
||||
raised = False
|
||||
err = context._worker_error
|
||||
exc_type = type(err)
|
||||
try:
|
||||
_format_worker_outcome(result=None, error=err)
|
||||
except exc_type:
|
||||
raised = True
|
||||
assert raised, f"_format_worker_outcome must re-raise {exc_type.__name__}"
|
||||
|
||||
|
||||
@then("the formatted outcome should be None")
|
||||
def step_formatted_outcome_none(context: Any) -> None:
|
||||
"""Assert _format_worker_outcome returns None when both inputs are None."""
|
||||
assert context._worker_outcome is None, (
|
||||
f"Expected None from _format_worker_outcome(None, None), "
|
||||
f"got: {context._worker_outcome!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the returned facade should be None")
|
||||
def step_returned_facade_none(context: Any) -> None:
|
||||
"""Assert _build_tui_facade returns None when construction fails."""
|
||||
assert context._facade_result is None, (
|
||||
f"Expected _build_tui_facade to return None on failure, "
|
||||
f"got: {context._facade_result!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the session should have been created with actor "{actor}"')
|
||||
def step_session_created_with_actor(context: Any, actor: str) -> None:
|
||||
"""Assert service.create() was called with the persona's actor."""
|
||||
context._mock_service.create.assert_called_once_with(actor_name=actor)
|
||||
@@ -218,6 +218,34 @@ Feature: LLM-powered Strategy Actor
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "gpt-4"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse strategy actor name with namespace/name format
|
||||
When I parse strategy actor name "local/my-strategist"
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "local/my-strategist"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse strategy actor name with org namespace/name format
|
||||
When I parse strategy actor name "cleverthis/my-executor"
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "cleverthis/my-executor"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: Parse strategy actor name with unknown-provider-like segment
|
||||
When I parse strategy actor name "unknown/gpt-4"
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "unknown/gpt-4"
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: _is_known_provider recognises valid providers
|
||||
When I check if "openai" is a known strategy provider
|
||||
Then the strategy provider check should be true
|
||||
|
||||
@tdd_issue @tdd_issue_11254
|
||||
Scenario: _is_known_provider rejects namespace prefix
|
||||
When I check if "local" is a known strategy provider
|
||||
Then the strategy provider check should be false
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Cyclic dependency detection through LLM execute path (H8)
|
||||
# ---------------------------------------------------------------
|
||||
@@ -326,8 +354,8 @@ Feature: LLM-powered Strategy Actor
|
||||
|
||||
Scenario: Parse strategy actor name with multiple slashes
|
||||
When I parse strategy actor name "provider/model/version"
|
||||
Then the strategy parsed provider should be "provider"
|
||||
And the strategy parsed model should be "model/version"
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "provider/model/version"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Actor name with empty segments (M2 fix)
|
||||
@@ -345,8 +373,8 @@ Feature: LLM-powered Strategy Actor
|
||||
|
||||
Scenario: Parse strategy actor name with empty model uses default model
|
||||
When I parse strategy actor name "provider/"
|
||||
Then the strategy parsed provider should be "provider"
|
||||
And the strategy parsed model should be "gpt-4"
|
||||
Then the strategy parsed provider should be "openai"
|
||||
And the strategy parsed model should be "provider/"
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Whitespace-only definition_of_done (L5 fix)
|
||||
|
||||
@@ -1,62 +1,42 @@
|
||||
# TDD issue-capture test for bug #10455 — EntityStore persistence stubs.
|
||||
# Regression tests for EntityStore and MemoryService SQL persistence.
|
||||
#
|
||||
# EntityStore in MemoryService exposes a connection_string parameter that
|
||||
# implies SQL-backed entity persistence. However, both persistence methods
|
||||
# are unimplemented stubs:
|
||||
#
|
||||
# _load_from_persistence() — contains only `pass`, entities never loaded.
|
||||
# _persist_if_needed() — marks dirty=False without writing any data.
|
||||
#
|
||||
# This creates a silent data-loss bug: callers that supply a connection_string
|
||||
# expect entities to survive process restarts, but they do not.
|
||||
#
|
||||
# These scenarios prove the bug exists by simulating separate process
|
||||
# invocations (fresh EntityStore / MemoryService instances backed by the
|
||||
# same SQLite database) and asserting that entities added in one invocation
|
||||
# are visible in the next. They FAIL until the bug is fixed.
|
||||
# The @tdd_expected_fail tag inverts the result so CI passes.
|
||||
# EntityStore in MemoryService now implements real SQL-backed entity
|
||||
# persistence via _load_from_persistence() and _persist_if_needed().
|
||||
# These scenarios verify that entities tracked in one instance survive
|
||||
# in a fresh instance backed by the same SQLite database.
|
||||
#
|
||||
# Originally TDD issue-capture tests for bug #10455 (EntityStore stubs).
|
||||
# See: https://git.cleverthis.com/cleveragents/cleveragents-core/issues/10455
|
||||
|
||||
@tdd_issue @tdd_issue_10455 @mock_only
|
||||
Feature: TDD Issue #10455 — EntityStore entity data lost across process restarts
|
||||
@mock_only
|
||||
Feature: EntityStore and MemoryService SQL persistence
|
||||
As a developer using MemoryService with a connection_string
|
||||
I want entities tracked via track_entity() to survive process restarts
|
||||
So that cross-session entity recall works as documented
|
||||
|
||||
EntityStore._load_from_persistence() is a stub (pass) and
|
||||
_persist_if_needed() marks dirty=False without writing data.
|
||||
A fresh EntityStore instance backed by the same database should
|
||||
contain entities added by a previous instance.
|
||||
|
||||
@tdd_issue @tdd_issue_10455
|
||||
Scenario: Entity tracked in one EntityStore instance is visible in a fresh instance
|
||||
Given I create an EntityStore with a SQLite connection string and session "entity-persist-test"
|
||||
When I track a project entity "my-project" in the first EntityStore instance
|
||||
And I create a fresh EntityStore instance with the same connection string and session
|
||||
Then the fresh EntityStore instance should contain the entity "my-project"
|
||||
|
||||
@tdd_issue @tdd_issue_10455
|
||||
Scenario: Entity tracked via MemoryService survives simulated process restart
|
||||
Given I create a MemoryService with a SQLite connection string and session "memory-persist-test"
|
||||
When I track a plan entity "my-plan" via the MemoryService
|
||||
And I create a fresh MemoryService with the same connection string and session
|
||||
Then the fresh MemoryService should return the entity "my-plan" when queried
|
||||
|
||||
@tdd_issue @tdd_issue_10455
|
||||
Scenario: Persistence failure raises an exception rather than silently succeeding
|
||||
Given I create an EntityStore with an invalid connection string
|
||||
When I attempt to track an entity in the EntityStore with invalid connection
|
||||
Then an exception should be raised rather than silently failing
|
||||
|
||||
@tdd_issue @tdd_issue_10455
|
||||
Scenario: Multiple entities survive a simulated process restart
|
||||
Given I create an EntityStore with a SQLite connection string and session "multi-entity-persist"
|
||||
When I track multiple entities in the first EntityStore instance
|
||||
And I create a fresh EntityStore instance with the same connection string and session
|
||||
Then all tracked entities should be present in the fresh EntityStore instance
|
||||
|
||||
@tdd_issue @tdd_issue_10455
|
||||
Scenario: Entity metadata and mention count survive a simulated process restart
|
||||
Given I create an EntityStore with a SQLite connection string and session "entity-metadata-persist"
|
||||
When I track a project entity "project-delta" with metadata
|
||||
@@ -70,4 +50,4 @@ Feature: TDD Issue #10455 — EntityStore entity data lost across process restar
|
||||
| key | value |
|
||||
| owner | alice |
|
||||
| status | active |
|
||||
And the fresh EntityStore entity "project-delta" should have mention count 2
|
||||
And the fresh EntityStore entity "project-delta" should have mention count 2
|
||||
@@ -0,0 +1,30 @@
|
||||
@tdd_issue @tdd_issue_11249
|
||||
Feature: TDD Issue #11249 — TUI prompt input characters not visible until Enter
|
||||
|
||||
Textual's ``Input._watch_value`` sets ``self.virtual_size`` on every
|
||||
keystroke. ``virtual_size`` is defined as ``Reactive(layout=True)`` on
|
||||
``Widget``, so each change calls ``refresh(layout=True)`` which enqueues
|
||||
ANSI escape sequences in Textual's ``WriterThread``. The thread only
|
||||
flushes stdout when its queue is empty; with continuous layout writes the
|
||||
queue never drains between keystrokes, so typed characters are invisible
|
||||
until Enter produces a burst that drains the queue.
|
||||
|
||||
The fix overrides ``virtual_size`` on ``_PromptTextInput`` with
|
||||
``Reactive(layout=False)`` so changes to it no longer trigger full
|
||||
layout recalculations. The CSS also fixes ``height: auto`` → ``height: 1``
|
||||
on the Input to remove the ``auto_dimensions`` guard that adds a second
|
||||
``refresh(layout=True)`` per keystroke.
|
||||
|
||||
See CONTRIBUTING.md > Bug Fix Workflow > TDD Issue Test Tags.
|
||||
|
||||
Scenario: _PromptTextInput.virtual_size reactive has layout=False
|
||||
Given the _PromptTextInput class is available
|
||||
Then its virtual_size reactive should have layout=False
|
||||
|
||||
Scenario: _PromptTextInput is used as the inner input in _TextualPromptInput
|
||||
Given the _TextualPromptInput class is available
|
||||
Then its inner _input should be a _PromptTextInput instance
|
||||
|
||||
Scenario: CSS prompt Input height is fixed not auto
|
||||
Given the TUI CSS file is loaded
|
||||
Then the prompt Input rule should use a fixed height not auto
|
||||
@@ -0,0 +1,22 @@
|
||||
Feature: Transport selector chooses correct transport by mode
|
||||
As a CleverAgents developer
|
||||
I want TransportSelector to return the right transport based on server_url
|
||||
So that local and server modes use the correct communication channel
|
||||
|
||||
@coverage
|
||||
Scenario: Select stdio transport when no server URL is given
|
||||
Given no transport server URL is configured
|
||||
When I call TransportSelector.select with no server_url
|
||||
Then the returned transport should be an A2aStdioTransport instance
|
||||
|
||||
@coverage
|
||||
Scenario: Select HTTP transport when server URL is provided
|
||||
Given a server URL "http://localhost:8080" is configured
|
||||
When I call TransportSelector.select with server_url "http://localhost:8080"
|
||||
Then the returned transport should be an A2aHttpTransport instance
|
||||
|
||||
@coverage
|
||||
Scenario: Select stdio transport for empty string server URL
|
||||
Given an empty string server URL is configured
|
||||
When I call TransportSelector.select with server_url ""
|
||||
Then the returned transport should be an A2aStdioTransport instance
|
||||
@@ -82,7 +82,7 @@ Feature: TUI first-run experience with actor selection overlay
|
||||
Scenario: ActorSelectionOverlay show populates default actors
|
||||
Given a new ActorSelectionOverlay
|
||||
When I call show on the overlay
|
||||
Then the overlay actors list should contain "anthropic/claude-4-sonnet"
|
||||
Then the overlay actors list should contain "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
Scenario: ActorSelectionOverlay show accepts custom actor list
|
||||
Given a new ActorSelectionOverlay
|
||||
@@ -125,7 +125,7 @@ Feature: TUI first-run experience with actor selection overlay
|
||||
Given a new ActorSelectionOverlay
|
||||
When I call show on the overlay
|
||||
And I call confirm on the overlay
|
||||
Then the confirmed actor should be "anthropic/claude-4-sonnet"
|
||||
Then the confirmed actor should be "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
Scenario: ActorSelectionOverlay confirm hides the overlay
|
||||
Given a new ActorSelectionOverlay
|
||||
@@ -191,6 +191,6 @@ Feature: TUI first-run experience with actor selection overlay
|
||||
Scenario: _complete_first_run creates default persona and refreshes bar
|
||||
Given the TUI app is initialised with an empty persona registry
|
||||
When I call on_mount on the first-run app
|
||||
And I call _complete_first_run with actor "anthropic/claude-4-sonnet"
|
||||
And I call _complete_first_run with actor "anthropic/claude-sonnet-4-20250514"
|
||||
Then the registry should contain a persona named "default"
|
||||
And the persona bar should reflect the new actor
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
Feature: TUI normal text input dispatches to LLM via A2A facade
|
||||
As a user of the CleverAgents TUI
|
||||
I want my normal text messages to be sent to an LLM actor
|
||||
So that I can have a real conversation through the TUI interface
|
||||
|
||||
Background:
|
||||
Given the TUI app is initialised with a mock A2A facade and session
|
||||
|
||||
Scenario: Normal text input dispatches to LLM and renders response
|
||||
When I submit the text "hello there"
|
||||
Then the facade should have received a message/send request
|
||||
And the conversation should show the assistant response
|
||||
|
||||
Scenario: Dispatch passes active persona actor as override
|
||||
When I dispatch "hello there" overriding actor with "openai/gpt-4o"
|
||||
Then the facade should have received a message/send request with actor "openai/gpt-4o"
|
||||
|
||||
Scenario: Normal text with no facade falls back to preview mode
|
||||
Given the TUI app is initialised without a facade
|
||||
When I submit the text "hello there"
|
||||
Then the conversation should show the preview text "hello there"
|
||||
|
||||
Scenario: LLM dispatch handles SessionActorNotConfiguredError gracefully
|
||||
Given the facade raises SessionActorNotConfiguredError
|
||||
When I submit the text "hello there"
|
||||
Then the conversation should show the no-actor error message
|
||||
|
||||
Scenario: LLM dispatch handles generic facade error gracefully
|
||||
Given the facade returns an A2A error response
|
||||
When I submit the text "hello there"
|
||||
Then the conversation should show an error message
|
||||
|
||||
Scenario: _create_tui_session returns real session_id from service
|
||||
Given a mock session service that returns session_id "abc-123"
|
||||
When I call _create_tui_session with that service
|
||||
Then the returned session_id should be "abc-123"
|
||||
|
||||
Scenario: _create_tui_session falls back to default when service unavailable
|
||||
Given the session service raises an exception
|
||||
When I call _create_tui_session with that service
|
||||
Then the returned session_id should be "default"
|
||||
|
||||
Scenario: LLM dispatch handles SessionNotFoundError gracefully
|
||||
Given the facade raises SessionNotFoundError
|
||||
When I submit the text "hello"
|
||||
Then the conversation should show a session-not-found error message
|
||||
|
||||
Scenario: LLM dispatch handles DatabaseError gracefully
|
||||
Given the facade raises DatabaseError
|
||||
When I submit the text "hello"
|
||||
Then the conversation should show a database error message
|
||||
|
||||
Scenario: Worker error is surfaced in the conversation widget
|
||||
When the worker completes with error "network timeout"
|
||||
Then the formatted outcome should show an error message
|
||||
|
||||
Scenario: Worker success result is passed through unchanged
|
||||
When the worker completes with result "You: hi\n\nAssistant: hello"
|
||||
Then the formatted outcome should be "You: hi\n\nAssistant: hello"
|
||||
|
||||
Scenario: Empty assistant response shows fallback text
|
||||
Given the TUI app is initialised with a facade that returns empty response
|
||||
When I submit the text "hello"
|
||||
Then the conversation should show "(no response)"
|
||||
|
||||
Scenario: Rich markup in user input is escaped before storage in transcript
|
||||
Given the TUI app is initialised with a mock A2A facade and session
|
||||
When I submit the text "[bold]attack[/bold]"
|
||||
Then the transcript entry should have markup escaped
|
||||
|
||||
Scenario: _format_worker_outcome re-raises KeyboardInterrupt
|
||||
When the worker completes with a KeyboardInterrupt error
|
||||
Then _format_worker_outcome should re-raise it
|
||||
|
||||
Scenario: _format_worker_outcome re-raises SystemExit
|
||||
When the worker completes with a SystemExit error
|
||||
Then _format_worker_outcome should re-raise it
|
||||
|
||||
Scenario: _format_worker_outcome returns None when both result and error are None
|
||||
When the worker completes with neither result nor error
|
||||
Then the formatted outcome should be None
|
||||
|
||||
Scenario: _build_tui_facade returns None when facade construction fails
|
||||
When _build_tui_facade is called with a broken container
|
||||
Then the returned facade should be None
|
||||
|
||||
Scenario: _create_tui_session binds active persona actor at session creation
|
||||
Given a mock session service that returns session_id "abc-123"
|
||||
And a persona state with active actor "anthropic/claude-4-sonnet"
|
||||
When I call _create_tui_session with that service and persona state
|
||||
Then the session should have been created with actor "anthropic/claude-4-sonnet"
|
||||
@@ -563,6 +563,7 @@ def coverage_report(session: nox.Session):
|
||||
"*/.venv/*",
|
||||
"*/.nox/*",
|
||||
"src/cleveragents/discovery/*",
|
||||
"src/cleveragents/tui/materializer.py",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@@ -197,6 +197,7 @@ omit = [
|
||||
"*/.venv/*",
|
||||
"*/.nox/*",
|
||||
"src/cleveragents/discovery/*",
|
||||
"src/cleveragents/tui/materializer.py",
|
||||
]
|
||||
data_file = "build/.coverage"
|
||||
|
||||
|
||||
@@ -48,6 +48,8 @@ def _make_app(*, result: str) -> MagicMock:
|
||||
app_exec = MagicMock()
|
||||
app_exec.config = SimpleNamespace(global_context={})
|
||||
app_exec.run_single_shot = AsyncMock(return_value=result)
|
||||
# last_run_tool_calls is an int property; set to 0 so comparisons work in tests
|
||||
app_exec.last_run_tool_calls = 0
|
||||
return app_exec
|
||||
|
||||
|
||||
|
||||
@@ -86,7 +86,7 @@ def _make_service(
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle.complete_apply.return_value = plan
|
||||
lifecycle.constrain_apply.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
store = MagicMock()
|
||||
store.get.return_value = changeset
|
||||
return PlanApplyService(lifecycle_service=lifecycle, changeset_store=store)
|
||||
|
||||
@@ -61,7 +61,7 @@ def _make_service(
|
||||
) -> ErrorRecoveryService:
|
||||
lifecycle = MagicMock()
|
||||
lifecycle.get_plan.return_value = plan
|
||||
lifecycle._commit_plan = MagicMock()
|
||||
lifecycle.commit_plan = MagicMock()
|
||||
return ErrorRecoveryService(
|
||||
lifecycle_service=lifecycle,
|
||||
auto_retry_threshold=threshold,
|
||||
|
||||
@@ -160,7 +160,7 @@ def _estimation_lifecycle_integration() -> None:
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
p.processing_state = ProcessingState.PROCESSING
|
||||
service._commit_plan(p)
|
||||
service.commit_plan(p)
|
||||
service.complete_strategize(plan_id)
|
||||
|
||||
# Check the plan after complete_strategize — auto_progress might
|
||||
@@ -198,7 +198,7 @@ def _no_estimation_lifecycle() -> None:
|
||||
service.start_strategize(plan_id)
|
||||
p = service.get_plan(plan_id)
|
||||
p.processing_state = ProcessingState.PROCESSING
|
||||
service._commit_plan(p)
|
||||
service.commit_plan(p)
|
||||
service.complete_strategize(plan_id)
|
||||
|
||||
p = service.get_plan(plan_id)
|
||||
|
||||
@@ -102,7 +102,7 @@ def test_execute() -> None:
|
||||
plan = svc.get_plan(plan_id)
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
|
||||
svc.execute_plan(plan_id)
|
||||
plan = svc.get_plan(plan_id)
|
||||
@@ -124,13 +124,13 @@ def test_apply() -> None:
|
||||
# Advance to Execute/COMPLETE
|
||||
plan = svc.get_plan(plan_id)
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
plan.processing_state = ProcessingState.QUEUED
|
||||
plan.phase = PlanPhase.EXECUTE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
plan.processing_state = ProcessingState.PROCESSING
|
||||
plan.processing_state = ProcessingState.COMPLETE
|
||||
svc._commit_plan(plan)
|
||||
svc.commit_plan(plan)
|
||||
|
||||
svc.apply_plan(plan_id)
|
||||
plan = svc.get_plan(plan_id)
|
||||
|
||||
@@ -475,6 +475,7 @@ def llm_execute_acms_context() -> None:
|
||||
lifecycle = SimpleNamespace(
|
||||
get_plan=MagicMock(return_value=plan),
|
||||
get_action=MagicMock(return_value=action),
|
||||
resolve_actor_provider_model=MagicMock(return_value=None),
|
||||
)
|
||||
|
||||
actor = LLMExecuteActor(
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user