Compare commits

..

2 Commits

Author SHA1 Message Date
HAL9000 dfa701b354 fix(cli): wrap plan status --format json output in spec-required JSON envelope 2026-05-17 02:54:16 +00:00
HAL9000 f79c18a3ec fix(data-integrity): remove silent argument swap in ValidationAttachmentRepository.attach (#8177)
Fixed a critical data integrity bug where validation_name and resource_id
arguments were being silently swapped based on the fragile heuristic
('/' in resource_id and '/' not in validation_name). This caused silent
data corruption without any error or warning. The conditional swap block
has been removed, ensuring arguments flow directly from caller to the
persistence layer in their declared order.
2026-05-16 08:14:39 +00:00
255 changed files with 1376 additions and 25091 deletions
-3
View File
@@ -183,6 +183,3 @@ agents-test
# Generated test reports (CI artifacts) — build artifacts, not to be committed
test_reports/
# Auto-added by CleverAgents plan executor
plan-output/
-82
View File
@@ -6,67 +6,6 @@ Changed `wf10_batch.robot` to be less likely to create files, and
`plan_generation_graph.robot` to give more test answers.
## [Unreleased]
- **fix(cli): add Read-Only and Writes columns to tool list output** (#1476): Rewrote
`list_tools()` in `src/cleveragents/cli/commands/tool.py` to render exactly the 5
spec-required columns (Name, Type, Source, Read-Only, Writes), removed the legacy
Description and Timeout columns, computes read-only/writes status from `capability`
metadata (rendering `✓`/`—`), and added a Summary panel showing Total, Tools,
Validations, Read-Only, Writes, and Namespaces counts. Adds Behave BDD tests in
`features/tool_cli.feature` verifying correct column names, capability rendering, and
Summary panel presence.
- **Fix actor compiler to read LSP bindings from typed `lsp_binding` field** (#1488): Fixed
`_extract_lsp_bindings()` in `actor/compiler.py` to read from `node.lsp_binding` (the typed
`NodeLspBinding` Pydantic field on `NodeDefinition`) as the primary path, with
backward-compatible fallback to the legacy `lsp_bindings` config dict key. Per-node LSP
bindings specified via the `lsp_binding:` YAML key are no longer silently dropped. Includes
Behave BDD and Robot Framework regression tests (#1432).
- **TDD regression tests for automation_profile DI bypass** (#1031): Added BDD scenarios
and Robot Framework integration tests verifying that ``_get_service()`` in
``automation_profile.py`` resolves ``AutomationProfileService`` through the DI container
rather than directly calling ``create_engine`` or ``sessionmaker``. Bug #990 was fixed by
PR #1181 before this TDD test PR merged; these tests serve as permanent regression guards
confirming the fix remains in place.
- Added team collaboration features: multi-user connection handling with
user identity tracking (TeamMember with owner/admin/member/viewer roles),
role-based access control (TeamPermission with read/write/admin/manage_members),
concurrent session support (SessionRegistry with thread-safe locking),
and optimistic-locking conflict resolution (VersionStamp with last-writer-wins,
reject, and merge strategies). Includes TeamCollaborationService orchestrating
all collaboration operations. Behave BDD tests and Robot Framework integration
tests included. (#863)
- Added FastAPI-based ASGI server endpoint served by uvicorn for the
CleverAgents server mode. Includes health check endpoint (`/health`),
A2A Agent Card discovery (`/.well-known/agent.json`), A2A JSON-RPC 2.0
routing (`/a2a`), configurable host:port binding via Settings, graceful
shutdown on SIGTERM/SIGINT, and `agents server start` CLI command.
Behave BDD tests and Robot Framework integration tests included. (#862)
- **SubplanExecutionService lazy wiring in CLI** (#10268): Wired `subplan_service`
from the DI container into `_get_plan_executor()` so `PlanExecutor` can spawn child
plans during the Execute phase. When `SubplanService` is available but
`SubplanExecutionService` is not explicitly injected, `_execute_subplans()` now
lazily creates a `SubplanExecutionService` using the parent plan's `subplan_config`
and the `_execute_child_plan` callback. Added recursion guard and strategize result
validation to `_execute_child_plan` to prevent re-entrant or orphaned child plan
execution.
- **Actor namespace/name disambiguation** (#11254): When action YAML references
actors using `namespace/name` format (e.g. `strategy_actor: local/my-strategist`),
the `_parse_actor_name()` functions no longer mistake the namespace prefix for an
LLM provider name. A new `_is_known_provider()` utility checks whether the first
slash-separated segment matches a known `ProviderType` (e.g. `openai`, `anthropic`);
if not, the input is treated as namespace/name. `PlanLifecycleService` now provides
`resolve_actor_provider_model()` to resolve namespaced references to their underlying
`provider/model` via the actor registry. All affected call sites — `StrategyActor`,
`LLMStrategizeActor`, `LLMExecuteActor`, and `SessionWorkflow._resolve_llm()` —
pre-resolve actor names before passing them to the LLM provider, fixing the
`ValueError: Unknown provider type` crash when using namespaced actor references.
Data integrity fix: ValidationAttachmentRepository argument swap (#7492): Fixed
a critical data integrity issue in `ValidationAttachmentRepository.attach` where
`validation_name` and `resource_id` arguments were being silently swapped based on a
fragile heuristic (`"/" in resource_id`). Arguments are now passed in the correct order,
ensuring data is stored with proper parameter values.
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
requires whole-word closing keywords (avoids false positives like
@@ -183,17 +122,6 @@ ensuring data is stored with proper parameter values.
- **Plan Rollback Command** (#8557): Implemented `agents plan rollback <plan-id> [<checkpoint-id>]` for checkpoint-based plan state restoration in Epic #8493. The command restores a plan's sandbox to the state captured at a given checkpoint, discarding all decisions made after that checkpoint. The checkpoint can be specified as an optional positional second argument or via the `--to-checkpoint` named option. Supports `--yes/-y` flag to skip confirmation prompts and `--format/-f` for output format selection (rich/plain/json/yaml). Included with comprehensive BDD test coverage (>= 97%) and spec-aligned output formatting showing rollback summary, changes reverted, impact analysis, and post-rollback state panels.
### Fixed
- **ACMS execute-phase assembler respects project-level hot_max_tokens** (#11035): Fixed
``_resolve_hot_max_tokens()`` to read ``hot_max_tokens`` from
``context_policy_json["acms_config"]["hot_max_tokens"]`` — the correct sub-key written
by ``agents project context set --hot-max-tokens``. The previous implementation read
from the top-level key (``config_dict.get("hot_max_tokens")``), which was always
``None``, causing the assembler to silently fall back to the global 16K default even
when a project-level override was configured. Also adds two Behave regression scenarios
with ``@tdd_issue @tdd_issue_11035`` tags that exercise the DB query code path and
verify the project-level budget is applied to ``CoreContextBudget`` and
``ContextRequest``.
- **Actor add `--config` crashes with combined-format ``config.actor`` YAML** (#11189): Fixed a bug where `agents actor add --config test/actor.yaml` raised ``click.BadParameter: "provider is required"`` when the config file used the spec-compliant combined ``config.actor`` format (both the compact string form ``config:\n actor: "<provider>/<model>"`` and the nested-dict form ``config:\n actor:\n type: llm\n provider: gcp\n model: gemini``). Added ``_detect_nested_config_actor()``, ``_flatten_config_actor()``, and corresponding handling in ``ActorConfiguration.from_blob()`` to transparently flatten the nesting so downstream v3 detection, schema validation, and canonicalisation see a flat dictionary. Includes new BDD scenarios and Python unit tests.
- **Guard cleanup_stale against execute/processing and execute/complete plans** (#11121):
``_create_sandbox_for_plan()`` in ``src/cleveragents/cli/commands/plan.py`` now
skips ``GitWorktreeSandbox.cleanup_stale()`` when the plan is in
@@ -1041,16 +969,6 @@ iteration` and data corruption under concurrent plan execution. All public
response format from the OpenCode API `/session/status` endpoint instead of an array.
Workers now dispatch and verify correctly, preventing incorrect session deletion.
- **Actor compiler ignores `actor_ref` field on SUBGRAPH nodes** (#1429): Fixed the
actor compiler (`src/cleveragents/actor/compiler.py`) to read `actor_ref` from the
top-level `NodeDefinition.actor_ref` field instead of `node.config.get("actor_ref")`.
Before this fix, `CompiledActor.metadata.subgraph_refs` was always empty and
`NodeConfig.subgraph` on every SUBGRAPH node was always `None`, silently breaking all
hierarchical/nested actor graph compositions. Added Behave regression tests covering
subgraph compilation with `actor_ref` fields and Robot Framework integration tests
verifying that `subgraph_refs` is correctly populated after compilation.
---
### Fixed
-7
View File
@@ -48,10 +48,3 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
* HAL 9000 has contributed the `ActorSelectionOverlay._render``_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
* HAL 9000 has contributed the actor compiler LSP binding fix (PR #1488 / issue #1432): fixed
`_extract_lsp_bindings()` to read `node.lsp_binding` (the typed `NodeLspBinding` field on
`NodeDefinition`) as the primary extraction path instead of only checking the untyped config
dict, so per-node LSP bindings specified via `lsp_binding:` YAML key are no longer silently
dropped. Includes Behave BDD and Robot Framework regression tests.
* HAL 9000 has contributed the config-actor combined-format support fix (PR #11232 / issue #11189): added ``_detect_nested_config_actor()``, ``_flatten_config_actor()``, and handling in ``ActorConfiguration.from_blob()`` to transparently flatten the nested ``config.actor`` block from both compact-string and nested-dict forms so v3 detection, schema validation, and canonicalisation see flat data — eliminating the ``"provider is required"`` crash.
* HAL 9000 has contributed the actor compiler `actor_ref` field fix (issue #1429): corrected `_map_node()` and `compile_actor()` in `src/cleveragents/actor/compiler.py` to read `actor_ref` from the top-level `NodeDefinition.actor_ref` field instead of `node.config.get("actor_ref")`, resolving silent failures on all SUBGRAPH nodes where `subgraph_refs` was always empty and `NodeConfig.subgraph` was always `None`.
+1 -2
View File
@@ -142,8 +142,7 @@ class SubgraphResolutionSuite:
type=NodeType.SUBGRAPH,
name="Sub",
description="Subgraph ref",
config={},
actor_ref="bench/inner",
config={"actor_ref": "bench/inner"},
),
]
outer_edges = [EdgeDefinition(from_node="main", to_node="sub")]
+1 -1
View File
@@ -89,7 +89,7 @@ def _make_service(plan: MagicMock, changeset: SpecChangeSet) -> PlanApplyService
lifecycle = MagicMock()
lifecycle.get_plan.return_value = plan
lifecycle.complete_apply.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
store = MagicMock()
store.get.return_value = changeset
return PlanApplyService(lifecycle_service=lifecycle, changeset_store=store)
+1 -1
View File
@@ -62,7 +62,7 @@ def _make_plan() -> MagicMock:
def _make_service(plan: MagicMock) -> ErrorRecoveryService:
lifecycle = MagicMock()
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
return ErrorRecoveryService(
lifecycle_service=lifecycle,
auto_retry_threshold=0.0,
+1 -1
View File
@@ -98,7 +98,7 @@ def _create_plan_and_service() -> tuple[str, PlanApplyService, InMemoryChangeSet
plan = lifecycle.get_plan(plan_id)
plan.changeset_id = cs.changeset_id
plan.sandbox_refs = ["sandbox-bench-001"]
lifecycle.commit_plan(plan)
lifecycle._commit_plan(plan)
return plan_id, apply_svc, store
+2 -2
View File
@@ -66,7 +66,7 @@ def _make_errored_plan(lifecycle: PlanLifecycleService, action_name: str) -> str
lifecycle.start_strategize(pid)
p = lifecycle.get_plan(pid)
p.decision_root_id = str(ULID())
lifecycle.commit_plan(p)
lifecycle._commit_plan(p)
lifecycle.complete_strategize(pid)
lifecycle.execute_plan(pid)
lifecycle.start_execute(pid)
@@ -104,7 +104,7 @@ class TimeCheckpointRecording:
self.lifecycle.start_strategize(pid)
p = self.lifecycle.get_plan(pid)
p.decision_root_id = str(ULID())
self.lifecycle.commit_plan(p)
self.lifecycle._commit_plan(p)
self.lifecycle.complete_strategize(pid)
self.lifecycle.execute_plan(pid)
self.lifecycle.start_execute(pid)
+31 -36
View File
@@ -1,13 +1,8 @@
"""ASV benchmarks for skeleton compressor overhead.
Measures the time to compress context fragments at various budgets
Measures the time to compress context fragments at various ratios
and fragment counts. The benchmark covers the hot path that runs
during subplan context inheritance.
The caller is responsible for converting a skeleton_ratio to an
absolute skeleton_budget before invoking compress(). These benchmarks
use representative absolute token budgets derived from typical ratios
applied to the fragment set sizes.
"""
from __future__ import annotations
@@ -38,9 +33,9 @@ from cleveragents.domain.models.core.context_fragment import ( # noqa: E402
_SKEL_PROV = FragmentProvenance(resource_uri="bench-skeleton://default")
def _build_fragments(count: int, tokens_each: int = 100) -> tuple[ContextFragment, ...]:
def _build_fragments(count: int, tokens_each: int = 100) -> list[ContextFragment]:
"""Generate *count* fragments for benchmarking."""
return tuple(
return [
ContextFragment(
fragment_id=f"bench-{i:05d}",
uko_node=f"bench-skeleton://file/{i}",
@@ -53,60 +48,60 @@ def _build_fragments(count: int, tokens_each: int = 100) -> tuple[ContextFragmen
),
)
for i in range(count)
)
]
class SkeletonCompressorSmallSuite:
"""Benchmark compression of a small fragment set (10 fragments, 1000 tokens)."""
"""Benchmark compression of a small fragment set (10 fragments)."""
def setup(self) -> None:
self._svc = SkeletonCompressorService()
self._fragments = _build_fragments(10) # 1000 total tokens
self._fragments = _build_fragments(10)
def time_compress_budget_full(self) -> None:
"""No compression (pass-through) — budget equals total tokens."""
self._svc.compress(self._fragments, 1000)
def time_compress_ratio_0(self) -> None:
"""No compression (pass-through)."""
self._svc.compress(self._fragments, skeleton_ratio=0.0)
def time_compress_budget_70pct(self) -> None:
"""Default-ratio-equivalent compression (budget = 70% of total)."""
self._svc.compress(self._fragments, 700)
def time_compress_ratio_03(self) -> None:
"""Default ratio compression."""
self._svc.compress(self._fragments, skeleton_ratio=0.3)
def time_compress_budget_50pct(self) -> None:
"""Moderate compression (budget = 50% of total)."""
self._svc.compress(self._fragments, 500)
def time_compress_ratio_05(self) -> None:
"""Moderate compression."""
self._svc.compress(self._fragments, skeleton_ratio=0.5)
def time_compress_budget_zero(self) -> None:
"""Maximum compression — empty result."""
self._svc.compress(self._fragments, 0)
def time_compress_ratio_1(self) -> None:
"""Maximum compression."""
self._svc.compress(self._fragments, skeleton_ratio=1.0)
class SkeletonCompressorMediumSuite:
"""Benchmark compression of a medium fragment set (100 fragments, 10000 tokens)."""
"""Benchmark compression of a medium fragment set (100 fragments)."""
def setup(self) -> None:
self._svc = SkeletonCompressorService()
self._fragments = _build_fragments(100) # 10000 total tokens
self._fragments = _build_fragments(100)
def time_compress_budget_70pct(self) -> None:
"""Default-ratio-equivalent compression."""
self._svc.compress(self._fragments, 7000)
def time_compress_ratio_03(self) -> None:
"""Default ratio compression."""
self._svc.compress(self._fragments, skeleton_ratio=0.3)
def time_compress_budget_50pct(self) -> None:
def time_compress_ratio_05(self) -> None:
"""Moderate compression."""
self._svc.compress(self._fragments, 5000)
self._svc.compress(self._fragments, skeleton_ratio=0.5)
class SkeletonCompressorLargeSuite:
"""Benchmark compression of a large fragment set (1000 fragments, 100000 tokens)."""
"""Benchmark compression of a large fragment set (1000 fragments)."""
def setup(self) -> None:
self._svc = SkeletonCompressorService()
self._fragments = _build_fragments(1000) # 100000 total tokens
self._fragments = _build_fragments(1000)
def time_compress_budget_50pct(self) -> None:
def time_compress_ratio_05(self) -> None:
"""Moderate compression over 1000 fragments."""
self._svc.compress(self._fragments, 50000)
self._svc.compress(self._fragments, skeleton_ratio=0.5)
def time_compress_budget_20pct(self) -> None:
def time_compress_ratio_08(self) -> None:
"""Heavy compression over 1000 fragments."""
self._svc.compress(self._fragments, 20000)
self._svc.compress(self._fragments, skeleton_ratio=0.8)
-30
View File
@@ -1,30 +0,0 @@
# docker-compose.yml — Local PostgreSQL for CleverAgents server-mode development
#
# Usage:
# docker compose up -d # start PostgreSQL
# docker compose down -v # tear down and remove data volume
#
# Connect from CleverAgents:
# export CLEVERAGENTS_SERVER_MODE=true
# export CLEVERAGENTS_DATABASE_URL=postgresql://cleveragents:cleveragents@localhost:5432/cleveragents
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: cleveragents
# WARNING: Development credentials only — do not use in production
POSTGRES_PASSWORD: cleveragents
POSTGRES_DB: cleveragents
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U cleveragents"]
interval: 5s
timeout: 5s
retries: 5
volumes:
pgdata:
-21
View File
@@ -25,15 +25,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Added
- **SubplanExecutionService lazy wiring in CLI** (#10268): Wired `subplan_service`
from the DI container into `_get_plan_executor()` so `PlanExecutor` can spawn child
plans during the Execute phase. When `SubplanService` is available but
`SubplanExecutionService` is not explicitly injected, `_execute_subplans()` now
lazily creates a `SubplanExecutionService` using the parent plan's `subplan_config`
and the `_execute_child_plan` callback. Added recursion guard and strategize result
validation to `_execute_child_plan` to prevent re-entrant or orphaned child plan
execution.
- **TDD: MCPToolAdapter.infer_resource_slots() TypeError with null properties** (#10470):
Added a TDD issue-capture Behave scenario that reproduces the bug where
`MCPToolAdapter.infer_resource_slots()` raises `TypeError` when the input schema
@@ -59,18 +50,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Fixed
- **Actor namespace/name disambiguation** (#11254): When action YAML references
actors using `namespace/name` format (e.g. `strategy_actor: local/my-strategist`),
the `_parse_actor_name()` functions no longer mistake the namespace prefix for an
LLM provider name. A new `_is_known_provider()` utility checks whether the first
slash-separated segment matches a known `ProviderType` (e.g. `openai`, `anthropic`);
if not, the input is treated as namespace/name. `PlanLifecycleService` now provides
`resolve_actor_provider_model()` to resolve namespaced references to their underlying
`provider/model` via the actor registry. All affected call sites — `StrategyActor`,
`LLMStrategizeActor`, `LLMExecuteActor`, and `SessionWorkflow._resolve_llm()`
pre-resolve actor names before passing them to the LLM provider, fixing the
`ValueError: Unknown provider type` crash when using namespaced actor references.
- **Built-in actors v3 YAML format** (#10883): Fixed `agents actor run` failing for
built-in actors (e.g., `openai/gpt-4`, `anthropic/claude-3-opus`) due to missing
v3 `type` field in stored configuration. `ActorRegistry.ensure_built_in_actors()`
+33 -54
View File
@@ -88,8 +88,8 @@ via the `getExtendedAgentCard` operation.
## Transport Modes
| Mode | Transport | Class / SDK Component | Behaviour |
|--------|------------------------|------------------------------------|--------------------------------------------------------------------|
| Mode | Transport | Class / SDK Component | Behaviour |
|--------|------------------------|------------------------------------|--------------------------------------------------------|
| Local | A2A JSON-RPC over stdio | `A2aLocalFacade` + JSON-RPC stdio | Agent as subprocess; extensions resolved in-process |
| Server | A2A JSON-RPC over HTTP | A2A SDK HTTP transport | JSON-RPC 2.0 to CleverAgents server |
@@ -165,7 +165,7 @@ Pattern applies to all entity types (`actor`, `skill`, `tool`, `validation`,
### Context Operations
| Method | Service Method | Required Params |
|--------|----------------|-----------------|
|--------|---------------|-----------------|
| `_cleveragents/context/show` | `ContextService.show()` | `project_name` |
| `_cleveragents/context/inspect` | `ContextService.inspect()` | `project_name` |
| `_cleveragents/context/simulate` | `ContextService.simulate()` | `project_name`, simulation params |
@@ -174,7 +174,7 @@ Pattern applies to all entity types (`actor`, `skill`, `tool`, `validation`,
### Sync Operations
| Method | Service Method | Required Params |
|--------|----------------|-----------------|
|--------|---------------|-----------------|
| `_cleveragents/sync/pull` | `SyncService.pull()` | `namespace` |
| `_cleveragents/sync/push` | `SyncService.push()` | `namespace`, `entities` |
| `_cleveragents/sync/status` | `SyncService.status()` | `namespace` (optional) |
@@ -182,7 +182,7 @@ Pattern applies to all entity types (`actor`, `skill`, `tool`, `validation`,
### Namespace Operations
| Method | Service Method | Required Params |
|--------|----------------|-----------------|
|--------|---------------|-----------------|
| `_cleveragents/namespace/list` | `NamespaceService.list()` | -- |
| `_cleveragents/namespace/show` | `NamespaceService.show()` | `namespace` |
| `_cleveragents/namespace/members` | `NamespaceService.members()` | `namespace` |
@@ -198,20 +198,13 @@ Pattern applies to all entity types (`actor`, `skill`, `tool`, `validation`,
## Streaming Events
A2A streaming uses SSE via the `message/stream` operation (client → server
request), delivering typed events as the task progresses. The SSE data
payloads are JSON-RPC 2.0 **notifications** (no `id` field):
A2A streaming uses SSE via the `message/stream` operation, delivering typed
events as the task progresses:
| Event Type | JSON-RPC Method | Emitted When |
|------------|-----------------|--------------|
| `TaskStatusUpdateEvent` | `task/statusUpdate` | Task state changes (submitted -> working -> completed, etc.) |
| `TaskArtifactUpdateEvent` | `task/artifactUpdate` | Agent produces output (response chunks, plan entries, tool results) |
> **Note:** `message/stream` is the **request** method (client → server) that
> initiates streaming. The SSE data payloads use `task/statusUpdate` or
> `task/artifactUpdate` as the notification method name — not `message/stream`.
> Non-spec fields (`event_id`, `event_type`, `timestamp`) are carried in SSE
> envelope headers (`event:` and `id:` lines), not in the data payload.
| Event Type | Emitted When |
|-----------|-------------|
| `TaskStatusUpdateEvent` | Task state changes (submitted -> working -> completed, etc.) |
| `TaskArtifactUpdateEvent` | Agent produces output (response chunks, plan entries, tool results) |
### Task Lifecycle States
@@ -227,30 +220,16 @@ payloads are JSON-RPC 2.0 **notifications** (no `id` field):
### Example Streaming Event
SSE data payload for a task status update (JSON-RPC 2.0 notification):
```json
{
"jsonrpc": "2.0",
"method": "task/statusUpdate",
"method": "message/stream",
"params": {
"taskId": "task_01HXR...",
"status": { "state": "working" }
}
}
```
SSE data payload for an artifact update:
```json
{
"jsonrpc": "2.0",
"method": "task/artifactUpdate",
"params": {
"taskId": "task_01HXR...",
"artifact": {
"id": "task_01HXR...",
"status": { "state": "working" },
"artifacts": [{
"parts": [{ "text": "I'll start by extracting..." }]
}
}]
}
}
```
@@ -298,9 +277,9 @@ response = facade.dispatch(A2aRequest(
### Constructor
| Parameter | Type | Default | Description |
|------------|----------------------------------|---------|--------------------------------|
| `services` | `dict[str, Any] \| None` | `None` | Named services for routing |
| Parameter | Type | Default | Description |
|------------|----------------------------|---------|-------------------------------|
| `services` | `dict[str, Any] \| None` | `None` | Named services for routing |
### Methods
@@ -320,15 +299,15 @@ registered later with `register_service()`.
### Service Keys
| Key | Type | Wired Methods |
|----------------------------------|----------------------------------|--------------------------------------------|
| `session_service` | `SessionWorkflow` | Standard message operations |
| `plan_lifecycle_service` | `PlanLifecycleService` | `_cleveragents/plan/*` |
| `tool_registry` | `ToolRegistry` | `_cleveragents/registry/tool/*` |
| `resource_registry_service` | `ResourceRegistryService` | `_cleveragents/registry/resource/*` |
| `sync_service` | `SyncService` | `_cleveragents/sync/*` |
| `namespace_service` | `NamespaceService` | `_cleveragents/namespace/*` |
| `context_service` | `ContextService` | `_cleveragents/context/*` |
| Key | Type | Wired Methods |
|------------------------------|-----------------------------|-------------------------------------|
| `session_service` | `SessionWorkflow` | Standard message operations |
| `plan_lifecycle_service` | `PlanLifecycleService` | `_cleveragents/plan/*` |
| `tool_registry` | `ToolRegistry` | `_cleveragents/registry/tool/*` |
| `resource_registry_service` | `ResourceRegistryService` | `_cleveragents/registry/resource/*` |
| `sync_service` | `SyncService` | `_cleveragents/sync/*` |
| `namespace_service` | `NamespaceService` | `_cleveragents/namespace/*` |
| `context_service` | `ContextService` | `_cleveragents/context/*` |
---
@@ -423,8 +402,8 @@ CleverAgentsError
└── A2aOperationNotFoundError
```
| Exception | When Raised |
|------------------------------|--------------------------------------------------------------------|
| `A2aNotAvailableError` | Server-mode operation attempted without connection |
| `A2aVersionMismatchError` | Unsupported A2A version |
| `A2aOperationNotFoundError` | Unknown method dispatched |
| Exception | When Raised |
|-----------------------------|----------------------------------------------------|
| `A2aNotAvailableError` | Server-mode operation attempted without connection |
| `A2aVersionMismatchError` | Unsupported A2A version |
| `A2aOperationNotFoundError` | Unknown method dispatched |
-24
View File
@@ -1,24 +0,0 @@
# Example: Required validation for unit tests
# Mirrors the workflow example in docs/specification.md (Example 1, Step 1).
name: local/unit-tests
description: Unit tests
source: custom
mode: required
code: |
import subprocess
def run(input_data):
result = subprocess.run(["pytest", "tests/"], capture_output=True, text=True)
passed = result.returncode == 0
return {
"passed": passed,
"message": "Unit tests passed" if passed else "Unit tests failed",
"data": {
"stdout": result.stdout,
"stderr": result.stderr,
"returncode": result.returncode,
},
}
timeout: 300
+1 -1
View File
@@ -223,7 +223,7 @@ Feature: A2A local facade coverage — uncovered handler and edge-case paths
When I dispatch facade-cov operation "_cleveragents/sync/push" with params {}
Then the facade-cov response status should be "ok"
And the facade-cov response data key "stub" should be true
And the facade-cov response data key "status" should equal "no_sync_service"
And the facade-cov response data key "status" should equal "not_implemented"
Scenario: Sync status returns not_implemented stub
Given a facade-cov facade with no services
@@ -82,42 +82,3 @@ Feature: A2A local facade coverage boost — uncovered validation paths
Then the coverage-boost response status should be "error"
And the coverage-boost response error code should be "INTERNAL_ERROR"
And the coverage-boost response error message should contain "plan_id is required"
# ---------------------------------------------------------------
# message/send validation (lines 468-471)
# ---------------------------------------------------------------
Scenario: message/send with empty session_id returns validation error
Given a coverage-boost facade with a mock PlanLifecycleService
When I dispatch coverage-boost operation "message/send" with params {"session_id": "", "message": "hi"}
Then the coverage-boost response status should be "error"
And the coverage-boost response error code should be "INTERNAL_ERROR"
And the coverage-boost response error message should contain "session_id is required"
Scenario: message/send with empty message returns validation error
Given a coverage-boost facade with a mock PlanLifecycleService
When I dispatch coverage-boost operation "message/send" with params {"session_id": "s1", "message": ""}
Then the coverage-boost response status should be "error"
And the coverage-boost response error code should be "INTERNAL_ERROR"
And the coverage-boost response error message should contain "message is required"
# ---------------------------------------------------------------
# Static DI-fallback resolvers (lines 411-425, 434-452)
# ---------------------------------------------------------------
Scenario: _build_actor_resolver_for_session_workflow falls back when DI container raises
When I call the static resolver builder with a failing DI container
Then the static resolver result should be None
Scenario: _build_actor_options_resolver_for_session_workflow falls back when DI container raises
When I call the static options-resolver builder with a failing DI container
Then the static options-resolver result should be None
# ---------------------------------------------------------------
# Service accessor: _provider_registry property (line 160)
# ---------------------------------------------------------------
Scenario: _provider_registry property reads from services dict
Given a coverage-boost facade wired with a mock ProviderRegistry
When I access the _provider_registry property
Then the provider registry accessor should return the mock
@@ -1,53 +0,0 @@
Feature: Actor add CLI supports config-actor combined format (issue #11189)
As a CleverAgents user
I want `agents actor add --config` to accept the spec-compliant combined
``config.actor`` format so that nested config definitions work correctly.
The combined format lets users specify all actor metadata inside a single
``config actor`` block, either as a compact string (e.g. ``"aws/gpt-4"``)
or as a nested dict with individual fields.
Background:
Given an actor CLI runner
And a temporary directory for test actor configs
# ────────────────────────────────────────────────────────────
# Combined-Format String Actor Registration
# ────────────────────────────────────────────────────────────
Scenario: Register via combined-format string "provider/model"
Given a v3 LLM actor YAML file with combined config.actor format "openai/gpt-4" and name "local/combined-string-llm"
When I run the actor add command for "local/combined-string-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "llm"
Scenario: Fallback registration when combined-format lacks a top-level type
Given a YAML config file with nested config format and combined string without explicit type
When I run the actor add command for "local/nested-combined-llm" with the prepared config file
Then v3actor the command should succeed
# ─────────────────────────────────════════════════════════───
# Combined-Format Nested-Dict Registration
# ─────────────────────────────────════════════════════════───
Scenario: Register via combined-format nested dict with type, provider and model
Given a v3 LLM actor YAML file with nested config.actor format "anthropic/claude-haiku" and name "local/nested-dict-llm"
When I run the actor add command for "local/nested-dict-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "llm"
# ─────────────────────────────────════════════════════════───
# Top-level name extraction from combined format
# ─────────────────────────────────════════════════════════───
Scenario: Derive name from top-level 'name' field when config-actor is used
Given a YAML config file with nested config and top-level name
When I run the actor add command for "local/config-with-name" with the prepared config file
Then v3actor the command should succeed
Scenario: Reject when no name is available anywhere in combined format
Given a YAML config file with nested config but no name anywhere
When I run the actor add command without a name argument
Then v3actor the command should fail
And v3actor the error message should contain "Actor name is required"
-22
View File
@@ -1,22 +0,0 @@
Feature: Actor compiler reads LSP bindings from NodeDefinition.lsp_binding typed field
As a CleverAgents developer
I want the actor compiler to read LSP bindings from the typed lsp_binding field
So that per-node LSP bindings are not silently dropped when specified via lsp_binding: YAML key
Background:
Given the actor compiler is available
@tdd_issue @tdd_issue_1432
Scenario: Node with lsp_binding typed field populates CompilationMetadata.lsp_bindings
Given a GRAPH actor config with a node using the lsp_binding typed field
When I compile the actor config
Then the compilation should succeed
And the compilation metadata should have 1 LSP binding
And the LSP binding should reference server "local/pyright"
@tdd_issue @tdd_issue_1432
Scenario: Node without lsp_binding field produces empty lsp_bindings in metadata
Given a GRAPH actor config with a single node
When I compile the actor config
Then the compilation should succeed
And the compilation metadata should have 0 LSP bindings
-109
View File
@@ -1,109 +0,0 @@
Feature: Actor options propagation to LLM constructor
Tests that actor-level `options` (openai_api_base, openai_api_key, etc.)
are correctly forwarded to `ProviderRegistry.create_llm()` across all
call sites: Strategize phase, Execute phase, and SessionWorkflow.
Forgejo: #11256
# ------------------------------------------------------------------
# build_llm_kwargs_from_options shared utility
# ------------------------------------------------------------------
Scenario: build_llm_kwargs_from_options forwards openai_api_base
Given aop actor options with "openai_api_base" set to "http://localhost:8000/v1"
When I call build_llm_kwargs_from_options
Then the result should contain "openai_api_base" with value "http://localhost:8000/v1"
Scenario: build_llm_kwargs_from_options routes openai_api_key through sentinel
Given aop actor options with "openai_api_key" set to "none"
When I call build_llm_kwargs_from_options
Then the result should contain "__api_key_sentinel" with value "none"
And the result should not contain "openai_api_key"
Scenario: build_llm_kwargs_from_options forwards allowed temperature
Given aop actor options with "temperature" set to "0.7"
When I call build_llm_kwargs_from_options
Then the result should contain "temperature" with value 0.7
Scenario: build_llm_kwargs_from_options rejects reserved keys
Given aop actor options with "provider_type" set to "anthropic"
When I call build_llm_kwargs_from_options
Then the result should not contain "provider_type"
Scenario: build_llm_kwargs_from_options rejects unknown keys
Given aop actor options with "dangerous_param" set to "value"
When I call build_llm_kwargs_from_options
Then the result should not contain "dangerous_param"
Scenario: build_llm_kwargs_from_options with None returns empty dict
Given aop actor options is None
When I call build_llm_kwargs_from_options
Then the result should be an empty dict
Scenario: build_llm_kwargs_from_options with empty dict returns empty dict
Given aop actor options is an empty dict
When I call build_llm_kwargs_from_options
Then the result should be an empty dict
Scenario: build_llm_kwargs_from_options forwards multiple allowed keys
Given aop actor options with "openai_api_base" set to "http://local:8000/v1"
And aop actor options with "max_tokens" set to "2048"
And aop actor options with "timeout" set to "60"
When I call build_llm_kwargs_from_options
Then the result should contain "openai_api_base" with value "http://local:8000/v1"
And the result should contain "max_tokens" with value 2048
And the result should contain "timeout" with value 60
# ------------------------------------------------------------------
# StrategyActor forwards actor options to create_llm
# ------------------------------------------------------------------
Scenario: StrategyActor resolves actor options and forwards to create_llm
Given aop a mock lifecycle service that resolves actor "local/test-strategist"
And aop the resolved actor options contain "openai_api_base" set to "http://backend:9000/v1"
And aop the resolved actor options contain "openai_api_key" set to "test-key"
And aop a mock provider registry that captures create_llm kwargs with LLM response
When aop the StrategyActor executes with LLM for plan "01KSQ4ADB3AVXTWEK2DD0ZJDKN"
Then aop create_llm should have been called
And aop create_llm kwargs should contain "openai_api_base" with value "http://backend:9000/v1"
And aop create_llm kwargs should contain "__api_key_sentinel" with value "test-key"
Scenario: StrategyActor creates LLM without options when actor has no options
Given aop a mock lifecycle service that resolves actor "openai/gpt-4"
And aop the resolved actor options is None
And aop a mock provider registry that captures create_llm kwargs with LLM response
When aop the StrategyActor executes with LLM for plan "01KSQ4ADB3AVXTWEK2DD0ZJCKN"
Then aop create_llm should have been called
And aop create_llm kwargs should not contain "openai_api_base"
# ------------------------------------------------------------------
# LLMStrategizeActor forwards actor options to create_llm
# ------------------------------------------------------------------
Scenario: LLMStrategizeActor forwards actor options to create_llm
Given aop a valid LLMStrategizeActor with actor options
When aop I call strategize execute with plan_id "PLAN_OPT" and stream callback
Then aop the strategize create_llm should have received "openai_api_base" with value "http://custom-backend:7000/v1"
And aop the strategize create_llm should have received "__api_key_sentinel" with value "local-key"
Scenario: LLMStrategizeActor creates LLM without options when none exist
Given aop a valid LLMStrategizeActor without actor options
When aop I call strategize execute with plan_id "PLAN_NOOPT" and no stream callback
Then aop the strategize create_llm should not have received "openai_api_base"
And aop the strategize create_llm should not have received "__api_key_sentinel"
# ------------------------------------------------------------------
# LLMExecuteActor forwards actor options to create_llm
# ------------------------------------------------------------------
Scenario: LLMExecuteActor forwards actor options to create_llm
Given aop a valid LLMExecuteActor with actor options
When aop I call execute actor with plan_id "EXEC_OPT" and read_only False
Then aop the execute create_llm should have received "openai_api_base" with value "http://custom-executor:9000/v1"
And aop the execute create_llm should have received "__api_key_sentinel" with value "exec-key"
Scenario: LLMExecuteActor creates LLM without options when none exist
Given aop a valid LLMExecuteActor without actor options
When aop I call execute actor with plan_id "EXEC_NOOPT" and read_only False
Then aop the execute create_llm should not have received "openai_api_base"
And aop the execute create_llm should not have received "__api_key_sentinel"
-307
View File
@@ -1,307 +0,0 @@
@coverage @coverage_actor_run_tool_calling
Feature: Actor run tool-calling via ToolCallingRuntime
When a skill is attached to `agents actor run` via `--skill`, the actor
should perform real LLM tool calls through ToolCallingRuntime.
Background:
Given a minimal ToolCallingAgent fixture
# ---------- A: single tool call succeeds ----------
Scenario: Tool call succeeds — mock LLM returns read_file call
Given a mock LLM caller that makes one read_file tool call
When ToolCallingAgent.process is called with prompt "Review src/auth.py"
Then the tool_calls count is 1
And the final response is non-empty
# ---------- B: multi-turn tool loop ----------
Scenario: Multi-turn tool loop — mock LLM makes two consecutive tool calls
Given a mock LLM caller that makes 2 consecutive tool calls before finishing
When ToolCallingAgent.process is called with prompt "Analyse two files"
Then the tool_calls count is 2
And the loop ran for at least 3 iterations
# ---------- C: no-skill plain LLM regression ----------
Scenario: No-skill plain LLM — SimpleLLMAgent used, no tool schemas sent
Given a SimpleLLMAgent with a mock LLM
When SimpleLLMAgent.process is called with a simple prompt
Then SimpleLLMAgent returns the mocked response
And no tool schemas were sent to the LLM
# ---------- D: skill tools not silently dropped ----------
Scenario: Skill tools not silently dropped when actor has no base tools list
Given an actor config with no base tools list
And resolved skill tool entries for the actor
When _make_agent_instance is called for that actor
Then the result is a ToolCallingAgent instance
# ---------- E: ToolCallingLLMCaller first-call system prompt ----------
Scenario: ToolCallingLLMCaller sends system prompt on first call
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
When invoke is called for the first time with a user prompt
Then the accumulated messages include a SystemMessage
And the accumulated messages include a HumanMessage
# ---------- F: ToolCallingLLMCaller subsequent call appends tool results ----------
Scenario: ToolCallingLLMCaller appends tool results on subsequent call
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
And a prior first invoke has been made
When invoke is called again with tool_results
Then a ToolMessage is appended to the accumulated messages
# ---------- G: GraphExecutor passes context to ToolCallingAgent ----------
Scenario: GraphExecutor._invoke_agent passes context dict to ToolCallingAgent
Given a ToolCallingAgent spy that captures its process() arguments
When GraphExecutor._invoke_agent is called with that agent and a context dict
Then the context dict was forwarded to ToolCallingAgent.process
# ---------- H: last_run_tool_calls property reflects run results ----------
Scenario: ReactiveCleverAgentsApp.last_run_tool_calls reflects tool calls
Given a ReactiveCleverAgentsApp with a registered ToolCallingAgent that has last_result
When _tally_tool_calls is called on the app
Then last_run_tool_calls equals the number of tool call history entries
# ---------- I: ToolCallingLLMCaller _render_prompt ----------
Scenario: ToolCallingLLMCaller._render_prompt handles empty template
Given a fresh ToolCallingLLMCaller
When _render_prompt is called with an empty string template
Then _render_prompt returns an empty string
Scenario: ToolCallingLLMCaller._render_prompt renders Jinja2 template
Given a fresh ToolCallingLLMCaller
When _render_prompt is called with a Jinja2 template and context
Then _render_prompt returns the rendered string
Scenario: ToolCallingLLMCaller._render_prompt handles rendering exception
Given a fresh ToolCallingLLMCaller
When _render_prompt is called with a template that raises during rendering
Then _render_prompt returns the original template
# ---------- J: ToolCallingLLMCaller _resolve_llm with kwargs ----------
Scenario: ToolCallingLLMCaller._resolve_llm passes temperature max_tokens max_retries to provider
Given a ToolCallingLLMCaller with temperature max_tokens max_retries in actor_config
When _resolve_llm is called with tool_schemas
Then the provider registry create_llm was called with the config kwargs
Scenario: ToolCallingLLMCaller._resolve_llm skips bind_tools when no schemas
Given a ToolCallingLLMCaller with temperature max_tokens max_retries in actor_config
When _resolve_llm is called without tool_schemas
Then bind_tools was not called on the LLM
Scenario: ToolCallingLLMCaller._resolve_llm falls back when bind_tools raises
Given a ToolCallingLLMCaller with a provider that raises on bind_tools
When _resolve_llm is called with tool_schemas for a failing bind
Then the plain LLM is used without tool binding
# ---------- K: ToolCallingLLMCaller invoke no system prompt ----------
Scenario: ToolCallingLLMCaller.invoke without system prompt only adds HumanMessage
Given a ToolCallingLLMCaller with an actor config without system_prompt
When invoke is called for the first time with a user prompt
Then the accumulated messages include a HumanMessage
And no SystemMessage was added
# ---------- L: ToolCallingLLMCaller invoke list content ----------
Scenario: ToolCallingLLMCaller.invoke handles list content in response
Given a ToolCallingLLMCaller with an actor config without system_prompt
When invoke is called and the LLM returns list content
Then the response content joins the list parts
# ---------- M: ToolCallingLLMCaller invoke failed tool result ----------
Scenario: ToolCallingLLMCaller.invoke uses error field on failed tool result
Given a ToolCallingLLMCaller with an actor config containing a system_prompt
And a prior first invoke has been made
When invoke is called with a failed tool_result
Then a ToolMessage with the error text is appended
# ---------- N: ToolCallingAgent._build_tool_registry edge cases ----------
Scenario: ToolCallingAgent._build_tool_registry skips empty tool name
Given a ToolCallingAgent with an entry that has an empty name
When _build_tool_registry is called
Then the local registry is empty
Scenario: ToolCallingAgent._build_tool_registry skips duplicate names
Given a ToolCallingAgent with two entries for the same builtin tool
When _build_tool_registry is called
Then the local registry has exactly one tool registered
Scenario: ToolCallingAgent._build_tool_registry warns when tool not in builtin registry
Given a ToolCallingAgent with an entry not found in builtin registry
When _build_tool_registry is called
Then the local registry is empty
# ---------- O: ToolCallingAgent.process_message_sync ----------
Scenario: ToolCallingAgent.process_message_sync delegates to process
Given a ToolCallingAgent with no tools and a mock process method
When process_message_sync is called
Then the result is the same as calling process directly
# ---------- P: ToolCallingLLMCaller.invoke returns tool calls from response ----------
Scenario: ToolCallingLLMCaller.invoke extracts tool calls from LLM response
Given a ToolCallingLLMCaller with an actor config without system_prompt
When invoke is called and the LLM returns a response with tool call dicts
Then the LLMResponse contains the extracted tool calls
# ---------- Q: _resolve_provider_format maps provider to correct format ----------
Scenario: _resolve_provider_format returns ANTHROPIC for anthropic provider
Given an actor config with provider "anthropic"
When _resolve_provider_format is called with that actor config
Then _resolve_provider_format returns ProviderFormat.ANTHROPIC
Scenario: _resolve_provider_format returns OPENAI for openai provider
Given an actor config with provider "openai"
When _resolve_provider_format is called with that actor config
Then _resolve_provider_format returns ProviderFormat.OPENAI
Scenario: _resolve_provider_format returns OPENAI for unknown provider
Given an actor config with provider "google"
When _resolve_provider_format is called with that actor config
Then _resolve_provider_format returns ProviderFormat.OPENAI
Scenario: _resolve_provider_format returns OPENAI for empty actor config
Given an empty actor config
When _resolve_provider_format is called with that actor config
Then _resolve_provider_format returns ProviderFormat.OPENAI
Scenario: _resolve_provider_format returns OPENAI for None actor config
When _resolve_provider_format is called with actor_config None
Then _resolve_provider_format returns ProviderFormat.OPENAI
# ---------- R: ToolCallingAgent.process passes provider_format to runtime ----------
Scenario: ToolCallingAgent.process passes provider_format to ToolCallingRuntime
Given a ToolCallingAgent with an anthropic actor config and a file-read tool entry
When ToolCallingAgent.process is called with prompt "Read my file"
Then ToolCallingRuntime was constructed with ProviderFormat.ANTHROPIC
# ---------- S: Tool name encoding/decoding ----------
Scenario: _encode_tool_name replaces colon and slash with uppercase sentinels
When _encode_tool_name is called with "server:namespace/tool"
Then _encode_tool_name returns "server_C_namespace_S_tool"
Scenario: _encode_tool_name is idempotent on already-clean names
When _encode_tool_name is called with "my-clean_tool"
Then _encode_tool_name returns "my-clean_tool"
Scenario: _decode_tool_name restores colon and slash from sentinels
When _decode_tool_name is called with "server_C_namespace_S_tool"
Then _decode_tool_name returns "server:namespace/tool"
Scenario: _decode_tool_name is idempotent on clean names
When _decode_tool_name is called with "my-clean_tool"
Then _decode_tool_name returns "my-clean_tool"
Scenario: encode-decode round-trip preserves original name
Given a tool name "local/my__tool"
When the tool name is encoded then decoded
Then the round-trip name equals "local/my__tool"
Scenario: encode-decode round-trip for server-qualified name
Given a tool name "server.example:builtin/git-status"
When the tool name is encoded then decoded
Then the round-trip name equals "server.example:builtin/git-status"
# ---------- T: _resolve_llm encodes tool names in schemas ----------
Scenario: _resolve_llm encodes tool names before calling bind_tools
Given a ToolCallingLLMCaller with default actor config
When _resolve_llm is called with a schema containing a namespaced tool name
Then bind_tools was called with the encoded tool name
# ---------- U: invoke decodes tool names from LLM response ----------
Scenario: invoke decodes tool names when extracting tool calls from LLM response
Given a ToolCallingLLMCaller with an actor config without system_prompt
When invoke is called and the LLM returns a response with encoded tool call names
Then the LLMResponse contains the decoded tool calls
# ---------- S: Tool name encoding/decoding ----------
Scenario: _encode_tool_name replaces colon and slash with uppercase sentinels
When _encode_tool_name is called with "server:namespace/tool"
Then _encode_tool_name returns "server_C_namespace_S_tool"
Scenario: _encode_tool_name is idempotent on already-clean names
When _encode_tool_name is called with "my-clean_tool"
Then _encode_tool_name returns "my-clean_tool"
Scenario: _decode_tool_name restores colon and slash from sentinels
When _decode_tool_name is called with "server_C_namespace_S_tool"
Then _decode_tool_name returns "server:namespace/tool"
Scenario: _decode_tool_name is idempotent on clean names
When _decode_tool_name is called with "my-clean_tool"
Then _decode_tool_name returns "my-clean_tool"
Scenario: encode-decode round-trip preserves original name
Given a tool name "local/my__tool"
When the tool name is encoded then decoded
Then the round-trip name equals "local/my__tool"
Scenario: encode-decode round-trip for server-qualified name
Given a tool name "server.example:builtin/git-status"
When the tool name is encoded then decoded
Then the round-trip name equals "server.example:builtin/git-status"
# ---------- T: _resolve_llm encodes tool names in schemas ----------
Scenario: _resolve_llm encodes tool names before calling bind_tools
Given a ToolCallingLLMCaller with default actor config
When _resolve_llm is called with a schema containing a namespaced tool name
Then bind_tools was called with the encoded tool name
# ---------- U: invoke decodes tool names from LLM response ----------
Scenario: invoke decodes tool names when extracting tool calls from LLM response
Given a ToolCallingLLMCaller with an actor config without system_prompt
When invoke is called and the LLM returns a response with encoded tool call names
Then the LLMResponse contains the decoded tool calls
# ---------- V: ToolCallingLLMCaller options block forwarding (#11243) ----------
@tdd_issue @tdd_issue_11243
Scenario: ToolCallingLLMCaller forwards openai_api_base and openai_api_key from options block
Given a ToolCallingLLMCaller with actor config options containing openai_api_base and openai_api_key
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
Then create_llm was called with openai_api_base forwarded from options
And create_llm was called with __api_key_sentinel extracted from options openai_api_key
@tdd_issue @tdd_issue_11243
Scenario: ToolCallingLLMCaller forwards allowed options keys to create_llm
Given a ToolCallingLLMCaller with actor config options containing allowed extra keys
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
Then create_llm was called with the allowed options keys forwarded
@tdd_issue @tdd_issue_11243
Scenario: ToolCallingLLMCaller rejects reserved keys in options block with a warning
Given a ToolCallingLLMCaller with actor config options containing reserved key provider_type
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
Then create_llm was NOT called with the reserved key provider_type
@tdd_issue @tdd_issue_11243
Scenario: ToolCallingLLMCaller rejects unknown keys in options block with a warning
Given a ToolCallingLLMCaller with actor config options containing unknown key foo_bar
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
Then create_llm was NOT called with the unknown key foo_bar
@tdd_issue @tdd_issue_11243
Scenario: ToolCallingLLMCaller top-level config takes precedence over options block
Given a ToolCallingLLMCaller with actor config with top-level temperature and options temperature
When _resolve_llm is called on the ToolCallingLLMCaller with empty tool_schemas
Then create_llm was called with the top-level temperature value
-21
View File
@@ -211,24 +211,3 @@ Feature: v3 Actor YAML schema support in CLI and registry
Given a v3 graph actor config dict with skills and lsp as dict
When I parse the v3 config through ReactiveConfigParser
Then each graph node agent should have lsp dict propagated
# M5: options block forwarded to agent_config in _build_from_v3 (#11223).
@tdd_issue @tdd_issue_11223
Scenario: ReactiveConfigParser propagates options block to agent config
Given a v3 LLM actor config dict with options block
When I parse the v3 config through ReactiveConfigParser
Then the agent config should contain the options block
# M5: actor without options block is unaffected (#11223).
@tdd_issue @tdd_issue_11223
Scenario: ReactiveConfigParser handles v3 actor without options block
Given a v3 LLM actor config dict without options block
When I parse the v3 config through ReactiveConfigParser
Then the agent config should not contain an options key
# M5: empty options dict is preserved (#11223).
@tdd_issue @tdd_issue_11223
Scenario: ReactiveConfigParser preserves empty options block
Given a v3 LLM actor config dict with empty options block
When I parse the v3 config through ReactiveConfigParser
Then the agent config should contain an empty options block
+1 -1
View File
@@ -188,4 +188,4 @@ Feature: Automation Profile CLI commands
# Legacy flag removal tests
Scenario: Legacy --automation-level flag is rejected on plan use
When I invoke plan use with --automation-level "manual"
Then the plan output should contain "--automation-level"
Then the plan output should contain "No such option: --automation-level"
+2 -21
View File
@@ -92,6 +92,7 @@ Feature: CLI output formats parity
Then json_keys and yaml_keys should match
# Direct formatting module tests
@tdd_issue @tdd_issue_4364 @tdd_expected_fail
Scenario: Format output handles all format types for dict
When I call format_output with a dict and format json
Then the format result should be valid JSON dict
@@ -101,33 +102,13 @@ Feature: CLI output formats parity
Then the format result should contain plain key-value pairs
When I call format_output with a dict and format table
Then the format result should contain table output
# fix(cli): format_output() rich format must produce styled output, not JSON
Scenario: Format output rich format produces styled terminal output not JSON
When I call format_output with a dict and format rich
Then the format result should contain rich styled output
And the format result should not be valid JSON
# fix(cli): format_output() color format must produce ANSI-colored output not plain text
Scenario: Format output color format produces ANSI-colored output not plain text
When I call format_output with a dict and format color
Then the format result should contain color styled output
And the format result should not be plain text only
Then the format result should be valid JSON dict
Scenario: Format output handles list data
When I call format_output with a list and format plain
Then the format result should contain separator lines
Scenario: Format output rich format handles list data
When I call format_output with a list and format rich
Then the format result should contain rich styled list output
And the format result should not be valid JSON
Scenario: Format output color format handles list data
When I call format_output with a list and format color
Then the format result should contain color styled list output
And the format result should not be plain text only
Scenario: Serialize value handles enums and nested dicts
When I call serialize_value with enum and nested data
Then the serialized result should have string enum values
-21
View File
@@ -1379,24 +1379,3 @@ Feature: Consolidated Routing
When the stream router tool agent processes "keep_me" through operation
Then the stream router tool agent operation result should be "keep_me"
# M5: SimpleLLMAgent._resolve_llm forwards options to registry.create_llm (#11223).
@tdd_issue @tdd_issue_11223
Scenario: SimpleLLMAgent forwards options block to LLM constructor
Given a stream router llm agent with options block containing custom base url
When the stream router llm agent resolves the llm
Then the llm constructor should have received the custom base url
# M5: actor without options block is unaffected (#11223).
@tdd_issue @tdd_issue_11223
Scenario: SimpleLLMAgent without options block calls LLM constructor without extra kwargs
Given a stream router llm agent without options block
When the stream router llm agent resolves the llm
Then the llm constructor should not have received extra kwargs
# M5: top-level keys take precedence over duplicates in options (#11223).
@tdd_issue @tdd_issue_11223
Scenario: SimpleLLMAgent top-level key takes precedence over options duplicate
Given a stream router llm agent with top-level temperature and options block
When the stream router llm agent resolves the llm
Then the llm constructor should have received the top-level temperature
-277
View File
@@ -1,277 +0,0 @@
@container-handler
Feature: Container infrastructure resource handler module
Verifies that the container handler module exists and implements all
five handler classes referenced in _resource_registry_container.py:
ContainerRuntimeHandler, ContainerImageHandler, ContainerChildHandler,
ContainerVolumeHandler, ContainerNetworkHandler.
All handlers must conform to the ResourceHandler protocol and be
importable from cleveragents.resource.handlers.container.
# ── Module importability ─────────────────────────────────
Scenario: Container handler module is importable
When I import the container handler module
Then the container handler module import should succeed without errors
Scenario Outline: Handler class is importable from container module
When I import "<class_name>" from the container handler module
Then the class should be importable
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
# ── Protocol conformance ─────────────────────────────────
Scenario Outline: Handler class conforms to ResourceHandler protocol
Given a "<class_name>" instance
Then the instance should satisfy the ResourceHandler protocol
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
# ── Handler resolution via resolver ─────────────────────
Scenario Outline: Handler resolves via resolve_handler for container types
When I resolve the handler "<handler_ref>"
Then the handler should resolve without HandlerResolutionError
Examples:
| handler_ref |
| cleveragents.resource.handlers.container:ContainerRuntimeHandler |
| cleveragents.resource.handlers.container:ContainerImageHandler |
| cleveragents.resource.handlers.container:ContainerChildHandler |
| cleveragents.resource.handlers.container:ContainerVolumeHandler |
| cleveragents.resource.handlers.container:ContainerNetworkHandler |
# ── ContainerRuntimeHandler ──────────────────────────────
Scenario: ContainerRuntimeHandler has correct type label
Given a "ContainerRuntimeHandler" instance
Then the handler type_label should be "container-runtime"
Scenario: ContainerRuntimeHandler resolve raises NotImplementedError
Given a "ContainerRuntimeHandler" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call resolve on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerRuntimeHandler read raises NotImplementedError
Given a "ContainerRuntimeHandler" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call read on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerRuntimeHandler content_hash returns EMPTY_CONTENT_HASH for no location
Given a "ContainerRuntimeHandler" instance
And a container resource of type "container-runtime" with no location
When I call content_hash on the container handler
Then the content_hash result should be the EMPTY_CONTENT_HASH sentinel
Scenario: ContainerRuntimeHandler content_hash returns identity hash for location
Given a "ContainerRuntimeHandler" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call content_hash on the container handler
Then the content_hash result should be a non-empty hex string
# ── ContainerImageHandler ────────────────────────────────
Scenario: ContainerImageHandler has correct type label
Given a "ContainerImageHandler" instance
Then the handler type_label should be "container-image"
Scenario: ContainerImageHandler resolve raises NotImplementedError
Given a "ContainerImageHandler" instance
And a container resource of type "container-image" with location "ubuntu:22.04"
When I call resolve on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerImageHandler content_hash returns identity hash for image ref
Given a "ContainerImageHandler" instance
And a container resource of type "container-image" with location "ubuntu:22.04"
When I call content_hash on the container handler
Then the content_hash result should be a non-empty hex string
# ── ContainerChildHandler ────────────────────────────────
Scenario: ContainerChildHandler has correct type label
Given a "ContainerChildHandler" instance
Then the handler type_label should be "container-child"
Scenario: ContainerChildHandler resolve raises NotImplementedError
Given a "ContainerChildHandler" instance
And a container resource of type "container-mount" with location "/mnt/data"
When I call resolve on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerChildHandler read raises NotImplementedError
Given a "ContainerChildHandler" instance
And a container resource of type "container-mount" with location "/mnt/data"
When I call read on the container handler
Then a container handler NotImplementedError should be raised
# ── ContainerVolumeHandler ───────────────────────────────
Scenario: ContainerVolumeHandler has correct type label
Given a "ContainerVolumeHandler" instance
Then the handler type_label should be "container-volume"
Scenario: ContainerVolumeHandler resolve raises NotImplementedError
Given a "ContainerVolumeHandler" instance
And a container resource of type "container-volume" with location "my-volume"
When I call resolve on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerVolumeHandler content_hash returns identity hash for volume name
Given a "ContainerVolumeHandler" instance
And a container resource of type "container-volume" with location "my-volume"
When I call content_hash on the container handler
Then the content_hash result should be a non-empty hex string
# ── ContainerNetworkHandler ──────────────────────────────
Scenario: ContainerNetworkHandler has correct type label
Given a "ContainerNetworkHandler" instance
Then the handler type_label should be "container-network"
Scenario: ContainerNetworkHandler resolve raises NotImplementedError
Given a "ContainerNetworkHandler" instance
And a container resource of type "container-network" with location "bridge"
When I call resolve on the container handler
Then a container handler NotImplementedError should be raised
Scenario: ContainerNetworkHandler content_hash returns identity hash for network name
Given a "ContainerNetworkHandler" instance
And a container resource of type "container-network" with location "bridge"
When I call content_hash on the container handler
Then the content_hash result should be a non-empty hex string
# ── CRUD stubs ───────────────────────────────────────────
Scenario Outline: Handler write raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call write on the container handler with data b"test"
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
Scenario Outline: Handler delete raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call delete on the container handler
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
Scenario Outline: Handler list_children raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call list_children on the container handler
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
Scenario Outline: Handler diff raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call diff on the container handler with other_location "/other"
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
Scenario Outline: Handler discover_children raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call discover_children on the container handler
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
# ── Lifecycle stubs ──────────────────────────────────────
Scenario Outline: Handler create_checkpoint raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call create_checkpoint on the container handler
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
Scenario Outline: Handler rollback_to raises NotImplementedError
Given a "<class_name>" instance
And a container resource of type "container-runtime" with location "/var/run/docker.sock"
When I call rollback_to on the container handler
Then a container handler NotImplementedError should be raised
Examples:
| class_name |
| ContainerRuntimeHandler |
| ContainerImageHandler |
| ContainerChildHandler |
| ContainerVolumeHandler |
| ContainerNetworkHandler |
# ── Registry integration ─────────────────────────────────
Scenario: container-runtime handler resolves without error after module creation
When I resolve the handler for resource type "container-runtime"
Then the handler should resolve without HandlerResolutionError
Scenario: container-image handler resolves without error after module creation
When I resolve the handler for resource type "container-image"
Then the handler should resolve without HandlerResolutionError
Scenario: container-volume handler resolves without error after module creation
When I resolve the handler for resource type "container-volume"
Then the handler should resolve without HandlerResolutionError
Scenario: container-network handler resolves without error after module creation
When I resolve the handler for resource type "container-network"
Then the handler should resolve without HandlerResolutionError
-53
View File
@@ -1,53 +0,0 @@
Feature: Database URL sanitisation — credentials must never be exposed
As a CleverAgents operator
I want database URLs in CLI output to have their credentials masked
So that running ``agents info`` never leaks passwords or API tokens
@tdd_bug_8395
Scenario Outline: PostgreSQL URL with user and password — all are masked
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| postgresql://user:secret@localhost/mydb | postgresql://***:***@localhost/mydb |
| postgresql://admin:p%40ssw0rd@db.example.com:5432/agents | postgresql://***:***@db.example.com:5432/agents |
| postgresql://readonly:r0ad_only@pg.cluster.internal:5433/production | postgresql://***:***@pg.cluster.internal:5433/production |
Scenario Outline: MySQL URL with credentials — all are masked
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| mysql://app:s3cret@mysql.internal:3306/agents | mysql://***:***@mysql.internal:3306/agents |
| mysql+pymysql://root:toor@localhost/testdb | mysql+pymysql://***:***@localhost/testdb |
Scenario Outline: SQLite URLs — remain unchanged (no credentials)
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| sqlite:///data/cleveragents.db | sqlite:///data/cleveragents.db |
| sqlite:////absolute/path/to/db.sqlite | sqlite:////absolute/path/to/db.sqlite |
| memory | memory |
Scenario: Username-only URL — password is still masked
Given the database url is "postgres://deploy@db.example.com/prod"
When I run sanitise_db_url
Then the sanitised database url should be "postgres://***:***@db.example.com/prod"
Scenario: SQLite URL without credentials remains unchanged
Given the database url is "sqlite:///test.db"
When I run sanitise_db_url
Then the sanitised database url should be "sqlite:///test.db"
Scenario: build_info_data returns sanitised database URL
Given a mock settings object with database url "postgresql://admin:supersecret@host.example.com:5432/mydb"
When I call build_info_data
Then the database field in info data should be "postgresql://***:***@host.example.com:5432/mydb"
@@ -1,38 +0,0 @@
Feature: Decomposition respects max_child_depth limit
As a plan orchestrator
I want the decomposition service to enforce plan.max-child-depth
So that child plans cannot nest deeper than the configured limit
Background:
Given a decomposition service
Scenario: Decomposition stops recursing when max_child_depth limit is reached
Given a project with 500 files across 8 directory levels
When I decompose with max_child_depth 2 and max_depth 10
Then the decomposition result should have max_depth_reached <= 2
And the decomposition warning log should contain "max-child-depth limit"
Scenario: Decomposition with default max_child_depth allows standard depth
Given a project with 2000 files across 5 directory levels
When I decompose with default max_child_depth
Then the decomposition result should have max_depth_reached >= 1
And the decomposition result should have max_depth_reached <= 5
Scenario: max_child_depth guard triggers before max_depth when more restrictive
Given a project with 500 files across 8 directory levels
When I decompose with max_child_depth 1 and max_depth 10
Then the decomposition result should have max_depth_reached <= 1
And the decomposition warning log should contain "max-child-depth limit"
Scenario: max_depth can be more restrictive than max_child_depth
Given a project with 500 files across 8 directory levels
When I decompose with max_child_depth 10 and max_depth 2
Then the decomposition result should have max_depth_reached <= 2
Scenario: Default config includes max_child_depth with expected value
Given default decomposition config
Then max_child_depth should be 5
Scenario: Invalid max_child_depth value is rejected
When I create a config with max_child_depth 0
Then a decomposition ValueError should be raised
-452
View File
@@ -1,452 +0,0 @@
@mock_only
Feature: Entity sync via _cleveragents/sync/*
As a CleverAgents user with multiple devices
I want to sync entities between my client and the server
So that I can access the same actors, skills, actions, plans, and sessions everywhere
# ------------------------------------------------------------------
# Sync models
# ------------------------------------------------------------------
Scenario: Vector clock increment produces new clock
Given a vector clock with entries {"client1": 1, "client2": 3}
When I increment the clock for node "client1"
Then the clock entry for "client1" should be 2
And the clock entry for "client2" should be 3
Scenario: Vector clock merge takes element-wise max
Given a vector clock with entries {"a": 2, "b": 1}
And another vector clock with entries {"a": 1, "b": 3, "c": 1}
When I merge the two clocks
Then the merged clock should have entries {"a": 2, "b": 3, "c": 1}
Scenario: Vector clock happens-before detection
Given a vector clock with entries {"a": 1, "b": 1}
And another vector clock with entries {"a": 2, "b": 2}
Then the first clock should happen before the second
Scenario: Vector clock concurrency detection
Given a vector clock with entries {"a": 2, "b": 1}
And another vector clock with entries {"a": 1, "b": 2}
Then the clocks should be concurrent
Scenario: Vector clock increment rejects empty node_id
Given an empty vector clock
When I try to increment the clock with an empty node_id
Then a sync ValueError should be raised
Scenario: Vector clock merge rejects non-VectorClock
Given an empty vector clock
When I try to merge with a non-VectorClock value
Then a sync TypeError should be raised
Scenario: SyncEntitySnapshot rejects empty entity_id
When I try to create a snapshot with an empty entity_id
Then a sync validation error should be raised
Scenario: SyncPullRequest rejects local namespace
When I try to create a pull request with namespace "local"
Then a sync validation error should be raised
Scenario: SyncPushRequest rejects local namespace
When I try to create a push request with namespace "local"
Then a sync validation error should be raised
Scenario: SyncStatusRequest rejects local namespace
When I try to create a status request with namespace "local"
Then a sync validation error should be raised
Scenario: SyncConflict requires non-empty conflict_id
When I try to create a conflict with an empty conflict_id
Then a sync validation error should be raised
Scenario: SyncQueueEntry requires non-empty queue_id
When I try to create a queue entry with an empty queue_id
Then a sync validation error should be raised
Scenario: SyncState requires non-empty namespace
When I try to create a sync state with an empty namespace
Then a sync validation error should be raised
# ------------------------------------------------------------------
# SyncService — pull
# ------------------------------------------------------------------
Scenario: Pull entities from server to local cache
Given a sync service with node_id "test-client"
And server entities in namespace "team" with 3 actors
When I pull all entities from namespace "team"
Then the pull response should contain 3 entities
And the pull response should have 0 conflicts
Scenario: Pull detects concurrent modifications as conflicts
Given a sync service with node_id "test-client"
And a local actor "my-actor" in namespace "team" with clock {"client": 2, "server": 1}
And a server actor "my-actor" in namespace "team" with clock {"client": 1, "server": 2}
When I pull all entities from namespace "team"
Then the pull response should have 1 conflicts
Scenario: Pull with incremental sync uses since parameter
Given a sync service with node_id "test-client"
And server entities in namespace "team" with timestamps
When I pull entities from namespace "team" since "2025-01-01T00:00:00+00:00"
Then only entities updated after the since timestamp are pulled
Scenario: Pull with force overwrites local entities
Given a sync service with node_id "test-client"
And a local actor "my-actor" in namespace "team" with clock {"client": 2}
And a server actor "my-actor" in namespace "team" with clock {"server": 1}
When I force pull all entities from namespace "team"
Then the pull response should contain 1 entities
And the pull response should have 0 conflicts
Scenario: Pull rejects local namespace
Given a sync service with node_id "test-client"
When I try to pull from namespace "local"
Then a sync ValidationError should be raised with message containing "local"
Scenario: Pull type-checks the request
Given a sync service with node_id "test-client"
When I try to pull with a non-SyncPullRequest argument
Then a sync TypeError should be raised
# ------------------------------------------------------------------
# SyncService — push
# ------------------------------------------------------------------
Scenario: Push entities from local to server
Given a sync service with node_id "test-client"
And 2 local actors in namespace "shared"
When I push all local entities to namespace "shared"
Then the push response should have 2 accepted
And the push response should have 0 rejected
Scenario: Push detects and resolves conflicts with last-writer-wins
Given a sync service with node_id "test-client"
And a local actor "conflict-actor" in namespace "team" clocked {"client": 2, "server": 1} at "2025-06-01T12:00:00+00:00"
And a server actor "conflict-actor" in namespace "team" clocked {"client": 1, "server": 2} at "2025-05-01T12:00:00+00:00"
When I push the local actor to namespace "team" with last-writer-wins resolution
Then the push response should have 1 accepted
And the push response should have 1 conflicts
Scenario: Push with force bypasses conflict detection
Given a sync service with node_id "test-client"
And a server actor "existing" in namespace "team" with clock {"server": 5}
When I force push a local actor "existing" to namespace "team"
Then the push response should have 1 accepted
And the push response should have 0 conflicts
Scenario: Push rejects local namespace
Given a sync service with node_id "test-client"
When I try to push to namespace "local"
Then a sync ValidationError should be raised with message containing "local"
Scenario: Push type-checks the request
Given a sync service with node_id "test-client"
When I try to push with a non-SyncPushRequest argument
Then a sync TypeError should be raised
# ------------------------------------------------------------------
# SyncService — status
# ------------------------------------------------------------------
Scenario: Status detects local-ahead drift
Given a sync service with node_id "test-client"
And a local actor "new-actor" in namespace "team" not on server
When I request sync status for namespace "team"
Then the status should show drift detected
And the status should have 1 local-ahead entities
Scenario: Status detects server-ahead drift
Given a sync service with node_id "test-client"
And a server actor "server-only" in namespace "team" not on local
When I request sync status for namespace "team"
Then the status should show drift detected
And the status should have 1 server-ahead entities
Scenario: Status detects no drift when in sync
Given a sync service with node_id "test-client"
And a synced actor "synced-actor" in namespace "team" on both sides
When I request sync status for namespace "team"
Then the status should show no drift
Scenario: Status rejects local namespace
Given a sync service with node_id "test-client"
When I try requesting sync status for namespace "local"
Then a sync ValidationError should be raised with message containing "local"
Scenario: Status type-checks the request
Given a sync service with node_id "test-client"
When I try to check status with a non-SyncStatusRequest argument
Then a sync TypeError should be raised
Scenario: Status filters by entity type
Given a sync service with node_id "test-client"
And a local actor "my-actor" in namespace "team" not on server
And a local skill "my-skill" in namespace "team" not on server
When I request filtered sync status for namespace "team" entity type "actor"
Then the status should have 1 local-ahead entities
# ------------------------------------------------------------------
# Conflict resolution
# ------------------------------------------------------------------
Scenario: Resolve conflict manually with local winner
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I resolve the conflict manually choosing "local"
Then the conflict should be marked as resolved
And the conflict winner should be "local"
Scenario: Resolve conflict manually with server winner
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I resolve the conflict manually choosing "server"
Then the conflict should be marked as resolved
And the conflict winner should be "server"
Scenario: Manual resolution without winner raises BusinessRuleViolation
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I try to resolve manually without specifying a winner
Then a sync BusinessRuleViolation should be raised
Scenario: Resolve non-existent conflict raises ValueError
Given a sync service with node_id "test-client"
When I try to resolve a conflict with id "nonexistent"
Then a sync ValueError should be raised
Scenario: Resolve conflict with empty conflict_id raises ValueError
Given a sync service with node_id "test-client"
When I try to resolve a conflict with an empty id
Then a sync ValueError should be raised
# ------------------------------------------------------------------
# Offline queue
# ------------------------------------------------------------------
Scenario: Enqueue offline push operation
Given a sync service with node_id "test-client"
And a local actor snapshot for offline queuing
When I enqueue an offline push to namespace "team"
Then the offline queue should have 1 entry
And the queued entry should have status "queued"
Scenario: Process offline queue retries push operations
Given a sync service with node_id "test-client"
And a queued offline push for actor "offline-actor" in namespace "team"
When I process the offline queue
Then the processed entries should include the offline actor
And the offline queue should be empty
Scenario: Offline queue respects max retries
Given a sync service with node_id "test-client"
And a queued offline push at max retries
When I process the offline queue
Then the failed entry should have status "failed"
Scenario: Enqueue rejects local namespace
Given a sync service with node_id "test-client"
And a local actor snapshot for offline queuing
When I try to enqueue to namespace "local"
Then a sync ValueError should be raised
Scenario: Enqueue rejects empty namespace
Given a sync service with node_id "test-client"
And a local actor snapshot for offline queuing
When I try to enqueue to an empty namespace
Then a sync ValueError should be raised
# ------------------------------------------------------------------
# Facade integration
# ------------------------------------------------------------------
Scenario: Facade sync/pull without service returns stub
Given a facade without sync service
When I dispatch "_cleveragents/sync/pull" with sync params {"namespace": "team"}
Then the facade response should be ok
And the facade response data should have "stub" set to true
Scenario: Facade sync/push without service returns stub
Given a facade without sync service
When I dispatch "_cleveragents/sync/push" with sync params {"namespace": "team", "entities": []}
Then the facade response should be ok
And the facade response data should have "stub" set to true
Scenario: Facade sync/status without service returns stub
Given a facade without sync service
When I dispatch "_cleveragents/sync/status" with sync params {"namespace": "team"}
Then the facade response should be ok
And the facade response data should have "stub" set to true
Scenario: Facade sync/pull with service returns entities
Given a facade with sync service containing server entities
When I dispatch "_cleveragents/sync/pull" with sync params {"namespace": "team"}
Then the facade response should be ok
And the facade response data should contain key "entities"
And the facade response data should contain key "total_pulled"
Scenario: Facade sync/push with service accepts entities
Given a facade with sync service
When I dispatch "_cleveragents/sync/push" with sync push params
Then the facade response should be ok
And the facade response data should contain key "accepted"
Scenario: Facade sync/status with service returns drift info
Given a facade with sync service containing local-only entities
When I dispatch "_cleveragents/sync/status" with sync params {"namespace": "team"}
Then the facade response should be ok
And the facade response data should contain key "drift_detected"
# ------------------------------------------------------------------
# SyncService constructor validation
# ------------------------------------------------------------------
Scenario: SyncService rejects non-string node_id
When I try to create a SyncService with a non-string node_id
Then a sync TypeError should be raised
Scenario: SyncService with None node_id generates ULID
When I create a SyncService with None node_id
Then the service should have a non-empty node_id
Scenario: SyncService register_local_entity rejects non-snapshot
Given a sync service with node_id "test-client"
When I try to register a non-snapshot as local entity
Then a sync TypeError should be raised
Scenario: SyncService register_server_entity rejects non-snapshot
Given a sync service with node_id "test-client"
When I try to register a non-snapshot as server entity
Then a sync TypeError should be raised
Scenario: Enqueue offline rejects invalid direction type
Given a sync service with node_id "test-client"
And a local actor snapshot for offline queuing
When I try to enqueue with an invalid direction type
Then a sync TypeError should be raised
Scenario: Enqueue offline rejects invalid entity type
Given a sync service with node_id "test-client"
When I try to enqueue with an invalid entity type
Then a sync TypeError should be raised
Scenario: Resolve conflict rejects invalid resolution type
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I try to resolve with an invalid resolution type
Then a sync TypeError should be raised
# ------------------------------------------------------------------
# Edge cases
# ------------------------------------------------------------------
Scenario: Pull from empty server returns empty response
Given a sync service with node_id "test-client"
When I pull all entities from namespace "empty-ns"
Then the pull response should contain 0 entities
Scenario: Push empty entity list returns empty response
Given a sync service with node_id "test-client"
When I push an empty entity list to namespace "team"
Then the push response should have 0 accepted
Scenario: Vector clock equal clocks are not concurrent
Given a vector clock with entries {"a": 1}
And another vector clock with entries {"a": 1}
Then the clocks should not be concurrent
Scenario: Status with entity type filter on skills
Given a sync service with node_id "test-client"
And a local skill "my-skill" in namespace "team" not on server
When I request filtered sync status for namespace "team" entity type "skill"
Then the status should have 1 local-ahead entities
Scenario: Push with server_wins resolution resolves to server entity
Given a sync service with node_id "test-client"
And a local actor "conflict-actor" in namespace "team" with clock {"client": 2, "server": 1}
And a server actor "conflict-actor" in namespace "team" with clock {"client": 1, "server": 2}
When I push the local actor to namespace "team" with server-wins resolution
Then the push response should have 1 accepted
And the push conflict winner should be "server"
Scenario: Push with manual resolution rejects conflicting entities
Given a sync service with node_id "test-client"
And a local actor "conflict-actor" in namespace "team" with clock {"client": 2, "server": 1}
And a server actor "conflict-actor" in namespace "team" with clock {"client": 1, "server": 2}
When I push the local actor to namespace "team" with manual resolution
Then the push response should have 0 accepted
And the push response should have 1 rejected
Scenario: Resolve conflict with client_wins
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I resolve the conflict with client_wins strategy
Then the conflict should be marked as resolved
And the conflict winner should be "local"
Scenario: Resolve conflict with server_wins
Given a sync service with node_id "test-client"
And an unresolved conflict between local and server versions
When I resolve the conflict with server_wins strategy
Then the conflict should be marked as resolved
And the conflict winner should be "server"
Scenario: Resolve conflict with last_writer_wins picks newer timestamp
Given a sync service with node_id "test-client"
And an unresolved conflict where local is newer
When I resolve the conflict with last_writer_wins strategy
Then the conflict should be marked as resolved
And the conflict winner should be "local"
Scenario: Pull updates sync state
Given a sync service with node_id "test-client"
And server entities in namespace "team" with 1 actors
When I pull all entities from namespace "team"
Then the pull response should have a sync state with last_pull_at set
Scenario: Push updates sync state
Given a sync service with node_id "test-client"
And 1 local actors in namespace "shared"
When I push all local entities to namespace "shared"
Then the push response should have a sync state with last_push_at set
Scenario: Offline queue enqueue pull operation
Given a sync service with node_id "test-client"
And a local actor snapshot for offline queuing
When I enqueue an offline pull from namespace "team"
Then the offline queue should have 1 entry
Scenario: Vector clock happens_before rejects non-VectorClock
Given an empty vector clock
When I try to call happens_before with a non-VectorClock value
Then a sync TypeError should be raised
Scenario: Vector clock is_concurrent rejects non-VectorClock
Given an empty vector clock
When I try to call is_concurrent with a non-VectorClock value
Then a sync TypeError should be raised
Scenario: Process offline queue processes pull operation
Given a sync service with node_id "test-client"
And server entities in namespace "team" with 1 actors
And a local actor snapshot for offline queuing
When I enqueue an offline pull from namespace "team"
And I process the offline queue
Then the processed entries should include the offline actor
And the offline queue should be empty
Scenario: Resolve conflict with last_writer_wins picks server when newer
Given a sync service with node_id "test-client"
And an unresolved conflict where server is newer
When I resolve the conflict with last_writer_wins strategy
Then the conflict should be marked as resolved
And the conflict winner should be "server"
Scenario: Push with client_wins resolution accepts local entity
Given a sync service with node_id "test-client"
And a local actor "conflict-actor" in namespace "team" with clock {"client": 2, "server": 1}
And a server actor "conflict-actor" in namespace "team" with clock {"client": 1, "server": 2}
When I push the local actor to namespace "team" with client-wins resolution
Then the push response should have 1 accepted
And the push conflict winner should be "local"
@@ -235,21 +235,3 @@ Feature: Execute-phase context assembler coverage
When epcov I call assemble on the plan
Then epcov the assembled result should be an AssembledContext
And epcov the assembled result should have fragments and metadata
# ---- _resolve_hot_max_tokens: reads context_policy_json from DB ----
@tdd_issue @tdd_issue_11035
Scenario: epcov assemble uses project-level hot_max_tokens from context_policy_json
Given epcov an assembler with context_policy_json hot_max_tokens 32000
And epcov a plan with project links
And epcov scoped fragments that pass all filters
When epcov I call assemble on the plan
Then epcov the pipeline received budget max_tokens of 32000
@tdd_issue @tdd_issue_11035
Scenario: epcov assemble falls back to global hot_max_tokens when context_policy_json has no override
Given epcov an assembler with no hot_max_tokens in context_policy_json
And epcov a plan with project links
And epcov scoped fragments that pass all filters
When epcov I call assemble on the plan
Then epcov the pipeline received budget max_tokens of 4096
@@ -114,7 +114,6 @@ Feature: Large-project hierarchical decomposition
Scenario: Default config has expected values
Given default decomposition config
Then max_depth should be 4
And max_child_depth should be 5
And max_files_per_subplan should be 500
And max_tokens_per_subplan should be 100000
And min_files_per_subplan should be 10
@@ -204,17 +204,9 @@ Feature: Legacy Data Migrator Coverage
Given I have a legacy project with read-only JSON files
When I run the legacy data migration
Then the migration should handle backup errors gracefully
Scenario: get_all_for_project is called only once for multiple plans
Given I have a legacy project with multiple plans to migrate efficiently
When I run the legacy data migration with query tracking
Then get_all_for_project should be called exactly once
And the migration should return true
And all plans should be migrated correctly
Scenario: Map existing plan IDs correctly during migration
Given I have a legacy project with plans already in database
When I run the legacy data migration
Then existing plans should not be duplicated
And the migration should return true
And the existing plan id should be mapped without type suppression
-29
View File
@@ -26,24 +26,6 @@ Feature: LLM Actors Coverage
Then the parsed provider should be "anthropic"
And the parsed model should be "claude-3"
@tdd_issue @tdd_issue_11254
Scenario: Parse actor name with namespace/name format
When I parse actor name "local/my-strategist"
Then the parsed provider should be "openai"
And the parsed model should be "local/my-strategist"
@tdd_issue @tdd_issue_11254
Scenario: Parse actor name with org namespace/name format
When I parse actor name "cleverthis/my-executor"
Then the parsed provider should be "openai"
And the parsed model should be "cleverthis/my-executor"
@tdd_issue @tdd_issue_11254
Scenario: Parse actor name with unknown-provider-like segment
When I parse actor name "unknown/gpt-4"
Then the parsed provider should be "openai"
And the parsed model should be "unknown/gpt-4"
# ---------------------------------------------------------------
# LLMStrategizeActor.__init__ validation
# ---------------------------------------------------------------
@@ -219,17 +201,6 @@ Feature: LLM Actors Coverage
When I parse file blocks from empty LLM output
Then I should get 0 changeset entries
# M7 fix: <CAFS>/<CAFE> short-form and >>>>>>>> non-conflicting formats
# had zero test coverage. These scenarios exercise both new patterns.
Scenario: Parse file blocks using CAFS short delimiter format
When I parse file blocks using the CAFS short delimiter format
Then I should get 1 changeset entry with path "src/cafs_example.py"
Scenario: Parse file blocks using the non-conflicting arrow delimiter format
When I parse file blocks using the new arrow delimiter format
Then I should get 1 changeset entry with path "src/arrow_example.py"
# ---------------------------------------------------------------
# LLMExecuteActor._write_to_sandbox
# ---------------------------------------------------------------
-83
View File
@@ -1,83 +0,0 @@
@tdd @tdd_issue @tdd_issue_10878
Feature: Delimiter parsing does not trip on triple-backticks in LLM output (#10878)
Bug #10878 caused architecture reviews to be truncated because the old regex
delimiter (triple-backtick markdown code fences ```) would stop at the first
triple-backtick appearing *inside* the LLM-generated file content, cutting off
all subsequent sections.
This test suite re-proves that the new delimiter scheme using unique sentinel
markers is not vulnerable to markdown collision.
Forgejo: #10878
# ---------------------------------------------------------------
# The old backtick-based parser was broken (reproduction of bug)
# ---------------------------------------------------------------
Scenario: Old backtick delimiter parser stops at first triple-backtick in content
Given a string containing ````
When I run the command "echo 'backtick-test'"
And I create file with LLM output containing code fences in body
When I parse the LLM output as if it were old backtick-delimited file blocks
Then the parser should return only 2 entries truncated at first triple-backtick
Scenario: New CLEVERAGENTS_FILE_START delimiter parser handles code fences inside content
Given a string containing ````
When I run the command "echo 'backtick-test'"
And I create file with LLM output using unique delimiters and code fences embedded in body
When I parse the LLM output with new CLEVERAGENTS_FILE_START/CLEVERAGENTS_FILE_END delimiters
Then the parser should return 6 entries not truncated at any triple-backtick
Scenario: Old backtick-delimiter parser fails when markdown report contains triple backticks
Given a string containing ````
When I run the command "echo 'old-parser-test'"
And I create LLM output where file content contains embedded markdown code blocks
When I parse it with the old backtick-style delimiter pattern
Then the parser should return only 1 entry instead of the expected 4
Scenario: New delimited parser handles markdown code blocks inside file content
Given a string containing ````
When I run the command "echo 'new-parser-test'"
And I create LLM output using unique sentinels with embedded ```python code blocks in body
When I parse it with CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters
Then the parser should return a result count of 4 entries
# ---------------------------------------------------------------
# Verify new delimiter handles the real-world bug pattern: report + 3+ file blocks each
# containing inline code examples that use triple-backticks
# ---------------------------------------------------------------
Scenario: Delimiter survives LLM output with architecture-report header and nested code
Given a string containing ````
When I run the command "echo 'full-regression-test'"
And I create full regression file block output that includes report section plus multiple files each with inline ```python examples
When I parse it with new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters
Then the parser should return the full entry count of all embedded blocks
# ---------------------------------------------------------------
# Unit tests on LLMExecuteActor._parse_file_blocks directly (unit-test level)
# ---------------------------------------------------------------
@mock_only
Scenario: _parse_file_blocks handles file content with triple-backtick code fences inside body
Given a mock plan_id "01HQXXXXX"
And LLM output containing two FILE blocks each with embedded ```python sections in the content
When I call _parse_file_blocks on that LLM output
Then it should return 2 changeset entries not 1
@mock_only
Scenario: _parse_file_blocks handles file content with a trailing triple-backtick line
Given a mock plan_id "01HYYYYYY"
And an llm_output where the file body ends on a line that is exactly ``` after content
When I call _parse_file_blocks on that output
Then it should return 1 entry
Scenario: Backwards compatibility — backtick-delimited file blocks still parse correctly (sanitisation)
Given a string ````
When I run the command "echo 'backtick-sanitise'"
And LLM input with backtick-delimited FILE blocks containing code without triple-backticks inside body
When I call _parse_file_blocks on that output
Then it should return 1 entry
@@ -1,86 +0,0 @@
@tdd @tdd_issue @tdd_issue_10878
Feature: Delimiter _parse_file_blocks handles embedded markdown code fences (#10878)
Bug #10878 caused architecture reviews to be truncated because the old
backtick delimiter pattern stopped at the first ``` appearing *inside*
LLM-generated file content. All subsequent sections were lost - for
example a report with 9 TOC entries produced only 3 before truncation.
The fix introduces unique sentinel markers:
<<<<<<< CLEVERAGENTS_FILE_START >>>>>>> / <<<<<<< CLEVERAGENTS_FILE_END >>>>>>>
This regression test proves the new sentinel delimiters do NOT collide
when file bodies contain triple-backtick markdown code fences.
Forgejo: #10878
# -------------------------------------------------------------------
# Main regression - embedded ```python code fences inside FILE blocks
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: New delimiter parses all FILE blocks when each body contains embedded ```python code fences
Given an LLM response with two FILE blocks, each having ```python sections in the body
And a mock plan_id "01HQREGTEST"
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
Then it should return 2 changeset entries
# -------------------------------------------------------------------
# Real-world architecture-review pattern (the original bug)
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: Full regression - report header + three FILE blocks each with inline markdown code fences
Given an LLM response mimicking an architecture review with report prose and three FILE blocks each containing markdown code example triple-backticks in their body
And a mock plan_id "01HQARCHTEST"
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
Then it should return the full entry count of all embedded blocks
# -------------------------------------------------------------------
# Edge case - trailing ``` at end of file body before END marker
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: A FILE block whose last line before the END sentinel is exactly ``` parses correctly
Given an LLM response where a single FILE block body ends on a line that is exactly ``` after code content
And a mock plan_id "01HQTRAILT"
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
Then it should return 1 changeset entries
# -------------------------------------------------------------------
# Edge case - sentinel text mentioned inline without full markers
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: LLM body mentions 'CLEVERAGENTS_FILE_END' in prose but does not use the full sentinel marker
Given an LLM response where a FILE block body contains the word 'CLEVERAGENTS_FILE_END' referenced in prose within content and another file follows
And a mock plan_id "01HQSENTINEL"
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
Then it should return 2 changeset entries (second entry has content in body)
# -------------------------------------------------------------------
# Edge case - git merge-conflict markers alongside new delimiters
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: FILE block bodies contain git merge-conflict-style markers but only the CLEVERAGENTS sentinels match
Given an LLM response where FILE block bodies include text like '>>>>>>> some_branch' and '<<<<<<< base' but the actual delimiters are CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END markers
And a mock plan_id "01HQGITTEST"
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
Then it should return 2 changeset entries
# -------------------------------------------------------------------
# Escape support for delimiter markers in file content (#262743)
# -------------------------------------------------------------------
@tdd_issue @tdd_issue_10878
Scenario: Escaped marker occurrences inside file body are not treated as boundaries
Given an LLM response with a single FILE block using legacy markers where the body contains a backslash-escaped <<<<<<< CLEVERAGENTS_FILE_START >>>>>>> sequence that should be preserved as literal text
And a mock plan_id "01HQESCTEST"
When I parse file blocks using the new delimiter pattern
Then it should return 1 changeset entries
-23
View File
@@ -5,35 +5,12 @@ Feature: LSP StdioTransport coverage
# ── start() error paths ─────────────────────────────────────────
@tdd_issue @tdd_issue_7044
Scenario: ltcov start cleans up subprocess when post-Popen init fails
Given ltcov I create a StdioTransport for command "cat"
And ltcov Popen is mocked to succeed with a running process
And ltcov logger.info after spawn raises RuntimeError
When ltcov I try to start the transport
Then ltcov an error occurred during start()
And ltcov the internal process should be None
And ltcov the mock process should have been terminated
And ltcov the mock process should have been waited on
Scenario: ltcov start raises RuntimeError when already alive
Given ltcov I create a StdioTransport for command "cat"
And ltcov the transport has a running mock process
When ltcov I try to start the transport
Then ltcov the error should be a RuntimeError with message "already started"
@tdd_issue @tdd_issue_7044
Scenario: ltcov start is alive after successful spawn
Given ltcov I create a StdioTransport for command "cat"
When ltcov I try to start the transport
Then ltcov the transport should be alive
And ltcov the internal process should not be None
Scenario: ltcov is_alive returns False when no process
Given ltcov I create a StdioTransport for command "cat"
When ltcov I check if the transport is alive
Then ltcov the is_alive result should be false
Scenario: ltcov start raises LspError on FileNotFoundError
Given ltcov I create a StdioTransport for command "nonexistent_binary_xyz"
And ltcov Popen is mocked to raise FileNotFoundError
-37
View File
@@ -1,37 +0,0 @@
Feature: CLI main() error handling paths
As a developer
I want to cover the error-handling branches in cleveragents.cli.main
So that convert_exit_code(), subcommand import failure, and config validation are exercised
@coverage
Scenario: invalid --config-path that is a directory is rejected
When I run CLI with arguments ["--config-path", "/tmp", "version"]
Then the main cli exit code should be 1
And the main cli output contains "is not a file"
@coverage
Scenario: unknown command returns exit code 2
When I run CLI with arguments ["this_is_not_a_command"]
Then the main cli exit code should be 2
And the main cli output contains "Invalid command 'this_is_not_a_command'"
# These tests call convert_exit_code and _print_basic_help directly
@coverage
Scenario: convert_exit_code(None) returns 0
When the main exit-code converter is called with the value "None"
Then the main convert_exit_code result is "0"
@coverage
Scenario: convert_exit_code(int -1) returns -1 for negative codes
When the main exit-code converter is called with the value "-1"
Then the main convert_exit_code result is "-1"
@coverage
Scenario: convert_exit_code(string "abc") returns 1 for unparseable
When the main exit-code converter is called with the value "abc"
Then the main convert_exit_code result is "1"
@coverage
Scenario: _print_basic_help prints without error
When I call the main _print_basic_help
Then the main _print_basic_help completes ok
+1 -7
View File
@@ -309,19 +309,13 @@ def make_mock_registry(response_content: str) -> SimpleNamespace:
def make_mock_lifecycle(
strategy_actor: str | None = "openai/gpt-4",
execution_actor: str | None = None,
) -> SimpleNamespace:
"""Create a mock lifecycle service."""
plan = SimpleNamespace(action_name="test-action")
action = SimpleNamespace(
strategy_actor=strategy_actor,
execution_actor=execution_actor or strategy_actor,
)
action = SimpleNamespace(strategy_actor=strategy_actor)
return SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value=None),
resolve_actor_options=MagicMock(return_value=None),
)
@@ -133,18 +133,3 @@ Feature: AuditService EventBus Wiring
Given an event bus with an audit subscriber
When a plan_applied event is emitted
Then the audit entry user_identity should be None
Scenario: Event timestamp is preserved in audit entry via subscriber
Given an event bus with an audit subscriber
When a plan_applied event with a known timestamp is emitted
Then the audit entry created_at should match the event timestamp
Scenario: AuditService.record with explicit timestamp
Given a direct audit service instance
When record is called with an explicit timestamp
Then the audit entry created_at should equal the explicit timestamp
Scenario: AuditService.record without timestamp auto-generates created_at
Given a direct audit service instance
When record is called without a timestamp
Then the audit entry created_at should be close to now
-58
View File
@@ -1,58 +0,0 @@
Feature: Plan apply correction diff rendering functions
As a developer maintaining plan_apply_service.py
I want to cover _build_correction_diff_dict, _render_correction_diff_plain, and _render_correction_diff_rich
So that all output rendering branches are exercised
@coverage
Scenario: Build correction diff dict with full data including artifacts path
Given I have a CorrectionAttemptRecord with mode="revert", original_decision_id="dec1", new_decision_id="dec2", guidance="Test fix", and archived_artifacts_path="/artifacts/zip"
When I call _build_correction_diff_dict on the record
Then the result should contain "correction_diff", "comparison", and "patch_preview" keys
And the comparison should have 1 reverted decision and 1 added decision
And the patch preview should include the artifacts path
@coverage
Scenario: Build correction diff dict without new_decision_id or archived artifacts
Given I have a CorrectionAttemptRecord with mode="append", original_decision_id="dec3", no new_decision, guidance="Review needed", and no archived artifacts
When I call _build_correction_diff_dict on the record
Then the result should contain "correction_diff", "comparison", and "patch_preview" keys
And the comparison counts are "1" reverted decisions and "0" new ones
And the patch preview should indicate correction execution is pending
@coverage
Scenario: Render correction diff dict as plain text with artifacts
Given I have a correction diff dict with mode="revert", original_decision_id="dec-100", guidance="Fix logic", completed_at present, and one added decision "dec-200"
When I call _render_correction_diff_plain on the dict
Then the output should contain "Correction Diff" heading
And the output should contain the mode value "revert"
And the output should contain "[OK] Correction diff generated" trailer
And the output should contain "Completed:" timestamp
And the output should contain the added decision line starting with "+ dec-200"
@coverage
Scenario: Render correction diff dict as plain text without completed_at or guidance
Given I have a correction diff dict with mode="append", original_decision_id="dec-300", no guidance text, and patch preview note about pending execution
When I call _render_correction_diff_plain on the dict
Then the output should contain "Correction Diff" heading
And the output should NOT contain "Completed:" line
And the output should NOT contain a "Guidance:" line
And the output should contain "[OK] Correction diff generated" trailer
And the output should contain the pending-execution note text
@coverage
Scenario: Render correction diff dict as rich text with artifacts
Given I have a correction diff dict with mode="revert", original_decision_id="dec-rc1", guidance="Patch rollback", completed_at present, and one added decision "dec-rc2"
When I call _render_correction_diff_rich on the dict
Then the output should contain "[bold]Correction Diff[/bold]" heading
And the output should contain "[cyan]revert[/cyan]" mode color markup
And the output should contain the rich green checkmark marker at end
And the output should contain a red-colored reverted decision ID
And the output should contain a green-colored added decision ID
@coverage
Scenario: Render correction diff dict as rich text without guidance or completion
Given I have a correction diff dict with mode="append", original_decision_id="dec-rc3", no guidance, and patch preview note about pending execution
When I call _render_correction_diff_rich on the dict
Then the output should contain "[bold]Correction Diff[/bold]" heading
And the output should NOT contain a "Guidance" line
And the output should contain the pending-execution note in dim markup
@@ -13,7 +13,7 @@ Feature: PlanApplyService branch coverage for handle_merge_failure logger path
When pas_branch I call handle_merge_failure and capture any exception
Then pas_branch a RuntimeError should have been raised
And pas_branch the plan error_details should still contain merge_conflict
And pas_branch lifecycle commit_plan should have been invoked before the error
And pas_branch lifecycle _commit_plan should have been invoked before the error
And pas_branch lifecycle fail_apply should have been invoked before the error
Scenario: handle_merge_failure returns plan when logger.error succeeds
@@ -1,68 +0,0 @@
@coverage @context
Feature: PlanExecutionContext coverage boost
As a developer
I want complete coverage of PlanExecutionContext and RuntimeExecuteActor
So that all uncovered lines in plan_execution_context.py are exercised
Scenario: PlanExecutionContext rejects empty plan_id
Given a pec fresh context
When I pec construct PlanExecutionContext with empty plan_id
Then a pec ValidationError should be raised with "plan_id must not be empty"
Scenario: PlanExecutionContext properties return configured values
Given a pec PlanExecutionContext with all optional fields set
Then pec the automation_profile property should return the configured value
And pec the plan_env property should return the configured value
And pec the project_env property should return the configured value
And pec the project_resources property should return the configured dict
And pec the resource_bindings property should return the configured dict
And pec the changeset_store property should return the configured store
And pec the active_changeset_ids property should return a list
Scenario: record_change raises PlanError when no changeset is active
Given a pec PlanExecutionContext with a valid plan_id
When I pec call record_change without starting a changeset
Then a pec PlanError should be raised about no active changeset
Scenario: get_changeset returns None for unknown changeset
Given a pec PlanExecutionContext with a valid plan_id
When I pec call get_changeset with a nonexistent changeset_id
Then pec the result should be None
Scenario: summarize returns metadata for execution context
Given a pec PlanExecutionContext with a valid plan_id
When I pec call summarize on the execution context
Then pec the summary should contain plan_id and decision_root_id
And pec the summary should contain counts for resources and bindings
Scenario: RuntimeExecuteActor rejects None tool_runner
Given a pec valid PlanExecutionContext
When I pec construct RuntimeExecuteActor with None tool_runner
Then a pec ValidationError should be raised with "tool_runner must not be None"
Scenario: RuntimeExecuteActor rejects None execution_context
Given a pec mock ToolRunner
When I pec construct RuntimeExecuteActor with None execution_context
Then a pec ValidationError should be raised with "execution_context must not be None"
Scenario: RuntimeExecuteActor properties return configured values
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context
Then pec the tool_runner property should return the configured ToolRunner
And pec the execution_context property should return the configured PlanExecutionContext
Scenario: RuntimeExecuteActor execute with stream callback produces events
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context with sandbox_root
And a pec stream event collector
When I pec execute with decisions and stream callback
Then pec the stream events should include "runtime_execute_started"
And pec the stream events should include "runtime_execute_complete"
Scenario: RuntimeExecuteActor execute without stream callback still works
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context
When I pec execute with decisions and no stream callback
Then pec the result should be a RuntimeExecuteResult with a valid changeset_id
Scenario: RuntimeExecuteActor execute with sandbox_root populates sandbox_refs
Given a pec RuntimeExecuteActor with valid tool_runner and execution_context with sandbox_root
When I pec execute with decisions and no stream callback
Then pec the result sandbox_refs should include the sandbox_root
@@ -1,28 +0,0 @@
@mock_only @subplan @plan_executor @tdd_issue @tdd_issue_10268
Feature: PlanExecutor hierarchical execution wiring
As a system executing hierarchical plans via the production CLI
I want SubplanExecutionService to be available when SubplanService is wired
So that child plan execution works without a pre-configured execution service
# ------------------------------------------------------------------
# SubplanExecutionService lazy creation (Forgejo #10268)
# ------------------------------------------------------------------
Scenario: PlanExecutor lazily creates SubplanExecutionService when SubplanService is present
Given a PlanExecutor with SubplanService but no SubplanExecutionService
And a parent plan in Execute phase with a subplan_spawn decision
When I call run_execute on the parent plan
Then SubplanService.get_spawn_decisions should have been called
And SubplanService.spawn should have been called with the spawn entries
And SubplanExecutionService.execute_all should have been called via lazy creation
# ------------------------------------------------------------------
# _execute_child_plan callback (Forgejo #10268)
# ------------------------------------------------------------------
Scenario: _execute_child_plan callback invokes strategize for a child plan
Given a PlanExecutor with SubplanService but no SubplanExecutionService
And a parent plan in Execute phase with a subplan_spawn decision
And a child plan ready for execution in the lifecycle service
When I call run_execute on the parent plan
Then SubplanExecutionService.execute_all should have been called via lazy creation
@@ -1,85 +0,0 @@
@subplan @plan_executor @integration @phase4 @tdd_issue @tdd_issue_10270
Feature: Plan hierarchy 4-phase lifecycle execution
As a system executing hierarchical plans
I want child plans to independently complete all 4 phases Strategize, Decompose, Execute, Validate
So that hierarchical decomposition produces fully executed, verifiable, and aggregated results
Background:
Given a plan lifecycle service with in-memory storage
And a DecisionService for recording spawn decisions
And a SubplanService backed by the DecisionService
# ------------------------------------------------------------------
# Happy path — 2-level hierarchy (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: Parent plan with 2-level hierarchy executes all phases for parent and children
Given a parent plan "parent-happy" in Strategize phase with a spawn decision recorded
When I create two child plans for "parent-happy" and pre-register them in the lifecycle
And I strategize and execute the parent plan "parent-happy"
Then the parent plan Strategize phase should be complete
And the parent plan Execute phase should be complete
And each child plan should have completed Strategize phase
And each child plan should have completed Execute phase
And the parent plan should have 1 subplan statuses
And all child subplan statuses should indicate success
# ------------------------------------------------------------------
# Checkpoint triggers — on_subplan_spawn (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: on_subplan_spawn checkpoint is created before first child execution
Given a parent plan "parent-checkpoint" in Strategize phase with a spawn decision recorded
And I create one child plan for "parent-checkpoint" and pre-register it in the lifecycle
And a CheckpointService is configured for the parent plan "parent-checkpoint"
When I strategize and execute the parent plan "parent-checkpoint"
Then at least one on_subplan_spawn checkpoint should have been created
# ------------------------------------------------------------------
# Max-child-depth enforcement (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: Decomposition respects max-child-depth during execution
Given a decomposition service with max-child-depth 1
And a project with 100 files across 5 directory levels
When I decompose the project files with stored config
Then the decomposition result should have max_depth_reached <= 1
Scenario: Decomposition with default max-child-depth allows moderate nesting
Given a decomposition service with default config
And a project with 200 files across 5 directory levels
When I decompose the project files with stored config
Then the decomposition result should have max_depth_reached >= 1
# ------------------------------------------------------------------
# Child success aggregation (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: Parent plan aggregates child subplan results after execution
Given a parent plan "parent-aggregate" in Strategize phase with two spawn decisions recorded
When I create two child plans for "parent-aggregate" and pre-register them in the lifecycle
And I strategize and execute the parent plan "parent-aggregate"
Then the parent plan should have subplan statuses with execution results
And the parent plan error_details should not contain failed_subplan_ids
# ------------------------------------------------------------------
# Child failure handling (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: Parent plan captures child plan failure in error_details
Given a parent plan "parent-fail" in Strategize phase with a spawn decision recorded without action registration
When I strategize and execute the parent plan "parent-fail"
Then the parent plan error_details should have failed subplan ids
# ------------------------------------------------------------------
# Nested execution — 3-level hierarchy (Forgejo #10270)
# ------------------------------------------------------------------
Scenario: Grandchild plan in 3-level hierarchy executes and cascades results
Given a parent plan "parent-nested" in Strategize phase with a spawn decision recorded
And I create child plan "child-nested" for "parent-nested" with its own spawn decision recorded
And I create a grandchild plan for "child-nested" and pre-register it in the lifecycle
When I strategize and execute the parent plan "parent-nested"
Then the parent plan should have 1 subplan statuses
And the child plan should complete both Strategize and Execute phases
And the grandchild plan should complete both Strategize and Execute phases
@@ -1,124 +0,0 @@
@plan @executor @subplan @coverage
Feature: PlanExecutor child plan execution coverage
As a developer
I want comprehensive tests for _execute_child_plan, _execute_subplans fallback,
_apply_subplan_results_to_plan exec_result=None, and checkpoint PlanError handling
So that uncovered lines in plan_executor.py are fully exercised
# ------------------------------------------------------------------
# _execute_child_plan — success path (lines 606, 616-649, 662)
# ------------------------------------------------------------------
Scenario: Execute child plan succeeds in stub mode
Given a child3 PlanExecutor for child plan execution
And child3 a SubplanStatus for a valid child plan
When I child3 call _execute_child_plan on the executor
Then child3 the SubplanExecutionOutput should indicate success
And child3 the SubplanExecutionOutput should have a changeset_summary
And child3 the running_plan_ids should be empty
# ------------------------------------------------------------------
# _execute_child_plan — circular detection (lines 607-615)
# ------------------------------------------------------------------
Scenario: Execute child plan detects circular re-entrant execution
Given a child3 PlanExecutor for child plan execution
And child3 a SubplanStatus for a valid child plan
And I child3 pre-add the subplan_id to running_plan_ids
When I child3 call _execute_child_plan on the executor
Then child3 the SubplanExecutionOutput should indicate failure
And child3 the SubplanExecutionOutput error should mention circular
# ------------------------------------------------------------------
# _execute_child_plan — plan not found (lines 620-624)
# ------------------------------------------------------------------
Scenario: Execute child plan returns failure when child plan not found
Given a child3 PlanExecutor with lifecycle returning None for child plan
And child3 a SubplanStatus for a valid child plan
When I child3 call _execute_child_plan on the executor
Then child3 the SubplanExecutionOutput should indicate failure
And child3 the SubplanExecutionOutput error should mention "not found"
# ------------------------------------------------------------------
# _execute_child_plan — strategize produces no decisions (lines 626-634)
# ------------------------------------------------------------------
Scenario: Execute child plan returns failure when strategize produces no decisions
Given a child3 PlanExecutor for child plan with empty definition
And child3 a SubplanStatus for a valid child plan
When I child3 call _execute_child_plan on the executor
Then child3 the SubplanExecutionOutput should indicate failure
And child3 the SubplanExecutionOutput error should mention "no decisions"
# ------------------------------------------------------------------
# _execute_child_plan — exception during execution (lines 650-660)
# ------------------------------------------------------------------
Scenario: Execute child plan catches exception and returns failure output
Given a child3 PlanExecutor whose lifecycle start_strategize raises RuntimeError
And child3 a SubplanStatus for a valid child plan
When I child3 call _execute_child_plan on the executor
Then child3 the SubplanExecutionOutput should indicate failure
And child3 the SubplanExecutionOutput error should contain the exception message
# ------------------------------------------------------------------
# _execute_subplans — fallback when subplan_execution_service is None
# but subplan_service is available (lines 521-527)
# ------------------------------------------------------------------
Scenario: _execute_subplans creates SubplanExecutionService when not injected
Given a child3 PlanExecutor with subplan_service but no subplan_execution_service
And child3 a SpawnResult with one spawned status
When I child3 call _execute_subplans on the executor
Then child3 the SubplanExecutionResult should not be None
# ------------------------------------------------------------------
# _execute_subplans — returns None when no subplan service (line 521)
# ------------------------------------------------------------------
Scenario: _execute_subplans returns None when both services are missing
Given a child3 PlanExecutor with no subplan services at all
And child3 a SpawnResult with one spawned status
When I child3 call _execute_subplans on the executor
Then child3 the SubplanExecutionResult should be None
# ------------------------------------------------------------------
# _execute_subplans — returns None when statuses is empty (line 532)
# ------------------------------------------------------------------
Scenario: _execute_subplans returns None when spawned_statuses is empty
Given a child3 PlanExecutor with subplan_service and success executor
And child3 a SpawnResult with no spawned statuses
When I child3 call _execute_subplans on the executor
Then child3 the SubplanExecutionResult should be None
# ------------------------------------------------------------------
# _apply_subplan_results_to_plan — exec_result is None (lines 569-570)
# ------------------------------------------------------------------
Scenario: _apply_subplan_results keeps spawned statuses when exec_result is None
Given a child3 PlanExecutor for apply subplan results
And child3 a SpawnResult with one spawned status
When I child3 call _apply_subplan_results_to_plan with exec_result None
Then child3 the plan subplan_statuses should match the spawn result statuses
# ------------------------------------------------------------------
# Property accessors for optional services (lines 433, 438, 443)
# ------------------------------------------------------------------
Scenario: Optional service properties return configured values
Given a child3 PlanExecutor with all optional services configured
Then child3 the fix_revalidate_orchestrator property should return the configured instance
And child3 the subplan_service property should return the configured instance
And child3 the subplan_execution_service property should return the configured instance
# ------------------------------------------------------------------
# _try_create_checkpoint — PlanError re-raise (lines 731-741, 745)
# ------------------------------------------------------------------
Scenario: _try_create_checkpoint re-raises PlanError from plan update failure
Given a child3 PlanExecutor with checkpoint manager and lifecycle that fails on commit_plan
And child3 a sandbox that returns a valid checkpoint
When I child3 attempt _try_create_checkpoint expecting PlanError
Then a child3 PlanError should be raised about persisting checkpoint metadata
+1 -1
View File
@@ -225,7 +225,7 @@ Feature: PlanExecutor comprehensive coverage
Then the cov2 strategize run result should be a StrategizeResult
And the cov2 lifecycle should have called start_strategize
And the cov2 lifecycle should have called complete_strategize
And the cov2 lifecycle should have called commit_plan
And the cov2 lifecycle should have called _commit_plan
Scenario: run_strategize sets decision_root_id on execution context
Given a cov2 mock lifecycle service
@@ -1,64 +0,0 @@
@tdd @tdd_issue @tdd_issue_10938 @tier-hydration
Feature: Tier hydrator integration in PlanExecutor run_strategize (#10938)
Verifies that PlanExecutor.run_strategize correctly:
- Calls hydrate_tiers_for_plan when tier_service, project_repository, and
resource_registry are all wired
- Skips hydration if this plan_id has already been hydrated by this executor
instance (plan-id-scoped cache, prevents cross-plan contamination)
- Catches hydration failures without aborting the strategize phase
Forgejo: #10938
# -------------------------------------------------------------------
# Success — hydrate_tiers_for_plan is called when all dependencies exist
# -------------------------------------------------------------------
Scenario: Hydration is invoked when tier_service, project_repository and resource_registry are present
Given a mock PlanExecutor with tier_service, project_repository and resource_registry wired in
And a plan in the Strategize phase
When I call run_strategize on that PlanExecutor
Then hydrate_tiers_for_plan should have been called once
And the plan executor strategize result should be a valid run_strategize response
# -------------------------------------------------------------------
# Cache — skip already-hydrated tiers (new block from PR #10938)
# -------------------------------------------------------------------
Scenario: Hydration is skipped when plan_id has already been hydrated by this executor
Given a mock PlanExecutor with tier_service and plan_id already marked as hydrated
And a plan in the Strategize phase
When I call run_strategize on that PlanExecutor
Then hydrate_tiers_for_plan should not have been called because the plan was already hydrated
And the plan executor strategize result should be a valid run_strategize response
Scenario: Hydration runs when hot tier is empty
Given a mock PlanExecutor with tier_service but no pre-existing hot fragments
And a plan in the Strategize phase
When I call run_strategize on that PlanExecutor
Then hydrate_tiers_for_plan should have been called once
# -------------------------------------------------------------------
# Failure — hydration failure does not block strategize (new try/except from PR #10938)
# -------------------------------------------------------------------
Scenario: Hydration OSError is caught non-fatally and strategize continues
Given a mock PlanExecutor with tier_service, project_repository and resource_registry where hydrate fails with OSError
And a plan in the Strategize phase
When I call run_strategize on that PlanExecutor
Then hydrate_tiers_for_plan should have been called once
Then the strategy result should contain decisions (strategize succeeded despite hydration failure)
# -------------------------------------------------------------------
# No-op — tier_service not wired does not crash
# -------------------------------------------------------------------
Scenario: Strategize proceeds when tier_service is None (no hydration attempted)
Given a mock PlanExecutor without tier_service
And a plan in the Strategize phase
When I call run_strategize on that PlanExecutor
Then hydrate_tiers_for_plan should not have been called
And the plan executor strategize result should be a valid run_strategize response
@@ -160,45 +160,4 @@ Scenario: _handle_correction_applied skips event with no plan_id
Given the plan lifecycle service is configured for r4
And get_plan will fail for the plan for r4
When I handle a CORRECTION_APPLIED event when get_plan fails for r4
Then no exception should be raised for r4
# ---------------------------------------------------------------
# resolve_actor_provider_model — namespace/name → provider/model
# Lines 731-764 (factor resolution for #11254)
# ---------------------------------------------------------------
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model returns provider/model for known provider
Given the plan lifecycle service has a unit of work for actor resolution r4
When I resolve actor provider model for "openai/gpt-4"
Then the resolved actor provider model should be "openai/gpt-4"
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model resolves namespace/name via actor lookup
Given the plan lifecycle service has a unit of work with a known actor "local/strategist" using "openai/gpt-4" for r4
When I resolve actor provider model for "local/strategist"
Then the resolved actor provider model should be "openai/gpt-4"
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model returns None when actor is not found
Given the plan lifecycle service has a unit of work without actor "local/missing-actor" for r4
When I resolve actor provider model for "local/missing-actor"
Then the resolved actor provider model should be None
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model catches exception from actor lookup
Given the plan lifecycle service has a unit of work that raises on actor lookup for r4
When I resolve actor provider model for "local/crash-actor"
Then the resolved actor provider model should be None
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model returns None when unit_of_work is absent
Given the plan lifecycle service has no unit of work for r4
When I resolve actor provider model for "local/strategist"
Then the resolved actor provider model should be None
@tdd_issue @tdd_issue_11254
Scenario: resolve_actor_provider_model returns None for empty actor name
Given the plan lifecycle service has a unit of work for actor resolution r4
When I resolve actor provider model for ""
Then the resolved actor provider model should be None
Then no exception should be raised for r4
-111
View File
@@ -1,111 +0,0 @@
@phase1 @database @postgresql @server_mode
Feature: PostgreSQL Storage Backend for Server Mode
As a platform operator
I want CleverAgents to support PostgreSQL as its storage backend
So that server mode can handle multi-user collaborative access
Background:
Given I have imported the database infrastructure modules
# ── Settings & URL Selection ──────────────────────────────────────
@settings @database_url
Scenario: Default database URL is SQLite in local mode
Given I create settings with server_mode disabled
Then the resolved database URL should start with "sqlite"
@settings @database_url
Scenario: Server mode with default URL raises ValueError
Given I create settings with server_mode enabled
And the database_url is the default SQLite value
Then resolving the database URL should raise a ValueError
@settings @database_url
Scenario: Server mode with explicit PostgreSQL URL uses that URL
Given I create settings with server_mode enabled
And the database_url is "postgresql://myhost/mydb"
Then the resolved database URL should be "postgresql://myhost/mydb"
@settings @database_url
Scenario: is_postgresql returns True for PostgreSQL URLs
Given I create settings with server_mode enabled
And the database_url is "postgresql://localhost/cleveragents"
Then is_postgresql should return True
@settings @database_url
Scenario: is_postgresql returns False for SQLite URLs
Given I create settings with server_mode disabled
Then is_postgresql should return False
@settings @database_url
Scenario: is_postgresql returns False when server mode raises ValueError for SQLite URL
Given I create settings with server_mode enabled
And the database_url is the default SQLite value
Then is_postgresql should return False
# ── Pool Configuration ────────────────────────────────────────────
@settings @pool
Scenario: Default pool settings are appropriate for multi-user
Given I create settings with server_mode enabled
Then the db_pool_size should be 5
And the db_max_overflow should be 10
And the db_pool_recycle should be 1800
@settings @pool
Scenario: Pool settings can be overridden via environment
Given I create settings with custom pool configuration
| setting | value |
| db_pool_size | 20 |
| db_max_overflow | 30 |
| db_pool_recycle | 900 |
Then the db_pool_size should be 20
And the db_max_overflow should be 30
And the db_pool_recycle should be 900
# ── UnitOfWork Engine Creation ────────────────────────────────────
@engine @sqlite
Scenario: UnitOfWork creates SQLite engine for local mode
Given I create a UnitOfWork with URL "sqlite:///:memory:"
When I access the engine
Then the engine dialect should be "sqlite"
@engine @pool
Scenario: UnitOfWork passes pool parameters to non-SQLite engines
Given I create a UnitOfWork with URL "sqlite:///:memory:" and pool_size 8
Then the UnitOfWork pool_size should be 8
And the UnitOfWork max_overflow should be 10
And the UnitOfWork pool_recycle should be 1800
# ── ORM Model Dialect Compatibility ───────────────────────────────
@models @dialect
Scenario: ORM models use only dialect-agnostic column types
Given I inspect all ORM models from the Base metadata
Then no column should use a SQLite-specific type
And all columns should use standard SQLAlchemy types
@models @dialect
Scenario: ORM models define no raw SQL or sqlite3 imports
Given I read the models module source code
Then the source should not contain "sqlite3"
And the source should not contain "PRAGMA"
And the source should not contain "raw_sql"
@models @dialect
Scenario: All table names are valid PostgreSQL identifiers
Given I inspect all ORM models from the Base metadata
Then all table names should be valid SQL identifiers
# ── Migration Runner ──────────────────────────────────────────────
@migration
Scenario: Migration runner handles SQLite URLs
Given I create a MigrationRunner with URL "sqlite:///:memory:"
Then the migration runner database_url should start with "sqlite"
@migration
Scenario: Migration runner handles PostgreSQL URLs
Given I create a MigrationRunner with URL "postgresql://localhost/test"
Then the migration runner database_url should start with "postgresql"
@@ -87,10 +87,10 @@ Feature: Reactive application coverage boost
Then a CleverAgentsException should be raised for resolution failure
@coverage
Scenario: Reactive app skill injection upgrades LLM agents to ToolCallingAgent
Scenario: Reactive app skill injection skips LLM agents without tools
Given a reactive app with skill tools and a config with an LLM agent
When agents are registered from config with skill tools present
Then the LLM agent should be a ToolCallingAgent not a SimpleLLMAgent
Then the LLM agent should remain a SimpleLLMAgent not a SimpleToolAgent
@coverage
Scenario: Reactive app raises error for skill resolution ValueError
-166
View File
@@ -1,166 +0,0 @@
@phase2 @a2a @server
Feature: ASGI Server Lifecycle
As a platform operator
I want to start, health-check, and shut down the CleverAgents ASGI server
So that the A2A JSON-RPC 2.0 endpoint is available for client communication
# -----------------------------------------------------------------------
# ASGI application creation
# -----------------------------------------------------------------------
Scenario: create_asgi_app returns a FastAPI application
When I create an ASGI application with default settings
Then the ASGI app should be a FastAPI instance
Scenario: create_asgi_app accepts a custom facade
Given an A2aLocalFacade with no services
When I create an ASGI application with that facade
Then the ASGI app should be a FastAPI instance
Scenario: create_asgi_app rejects invalid facade type
When I try to create an ASGI application with facade "not-a-facade"
Then a TypeError should be raised for invalid facade
Scenario: create_asgi_app rejects invalid port
When I try to create an ASGI application with port 0
Then a ValueError should be raised for invalid port
Scenario: create_asgi_app rejects invalid host
When I try to create an ASGI application with an empty host
Then a ValueError should be raised for invalid host
# -----------------------------------------------------------------------
# Health check endpoint
# -----------------------------------------------------------------------
Scenario: Health endpoint returns healthy status
Given a running ASGI test client
When I request GET /health
Then the response status code should be 200
And the response body should contain "healthy"
# -----------------------------------------------------------------------
# Agent Card discovery endpoint
# -----------------------------------------------------------------------
Scenario: Agent Card endpoint returns valid agent card
Given a running ASGI test client
When I request GET /.well-known/agent.json
Then the response status code should be 200
And the response body should contain "CleverAgents"
And the response body should contain "url"
# -----------------------------------------------------------------------
# A2A JSON-RPC endpoint
# -----------------------------------------------------------------------
Scenario: A2A endpoint dispatches health check operation
Given a running ASGI test client
When I POST a JSON-RPC request to /a2a with operation "_cleveragents/health/check"
Then the response status code should be 200
And the A2A response status should be "healthy"
Scenario: A2A endpoint returns error for unknown operation
Given a running ASGI test client
When I POST a JSON-RPC request to /a2a with operation "nonexistent.operation"
Then the response status code should be 404
Scenario: A2A endpoint returns 400 for malformed request
Given a running ASGI test client
When I POST a malformed JSON body to /a2a
Then the response status code should be 400
# -----------------------------------------------------------------------
# ServerLifecycle construction
# -----------------------------------------------------------------------
Scenario: ServerLifecycle accepts valid configuration
When I create a ServerLifecycle with host "127.0.0.1" and port 9000
Then the lifecycle host should be "127.0.0.1"
And the lifecycle port should be 9000
And the lifecycle should not be started
And the lifecycle should not be stopped
Scenario: ServerLifecycle rejects empty host
When I try to create a ServerLifecycle with an empty host
Then a ValueError should be raised for invalid host
Scenario: ServerLifecycle rejects invalid port 0
When I try to create a ServerLifecycle with port 0
Then a ValueError should be raised for invalid port
Scenario: ServerLifecycle rejects port above 65535
When I try to create a ServerLifecycle with port 70000
Then a ValueError should be raised for invalid port
Scenario: ServerLifecycle rejects empty log_level
When I try to create a ServerLifecycle with empty log_level
Then a ValueError should be raised for invalid log_level
# -----------------------------------------------------------------------
# Server configuration from Settings
# -----------------------------------------------------------------------
Scenario: Settings provides default server host
Then the default server host from Settings should be "0.0.0.0"
Scenario: Settings provides default server port
Then the default server port from Settings should be 8080
# -----------------------------------------------------------------------
# Graceful shutdown
# -----------------------------------------------------------------------
Scenario: ServerLifecycle request_shutdown sets flag
Given a ServerLifecycle with a mock uvicorn server
When I call request_shutdown on the lifecycle
Then the mock server should_exit flag should be true
Scenario: ServerLifecycle cannot be started twice
Given a ServerLifecycle that has already been started
When I try to start the lifecycle again
Then a RuntimeError should be raised for double start
# -----------------------------------------------------------------------
# Full start lifecycle (mocked uvicorn)
# -----------------------------------------------------------------------
Scenario: ServerLifecycle start runs uvicorn and marks stopped on exit
Given a ServerLifecycle with mocked uvicorn server run
When I call start on the lifecycle
Then the lifecycle should be started
And the lifecycle should be stopped
And the mocked uvicorn server run should have been called
Scenario: ServerLifecycle start installs signal handlers on main thread
Given a ServerLifecycle with mocked uvicorn and signal handlers
When I call start on the lifecycle
Then SIGTERM and SIGINT handlers should have been installed
Scenario: Signal handler calls request_shutdown with signal name
Given a ServerLifecycle with mocked uvicorn server run
When I call start on the lifecycle
And I invoke the installed SIGTERM handler
Then the mocked server should_exit should be true
# -----------------------------------------------------------------------
# run_server convenience function
# -----------------------------------------------------------------------
Scenario: run_server resolves defaults from Settings
When I call run_server with mocked ServerLifecycle
Then the lifecycle should have been created with Settings defaults
And start should have been called on the lifecycle
Scenario: run_server passes explicit host and port overrides
When I call run_server with host "10.0.0.1" and port 3000 using mocked lifecycle
Then the lifecycle should have been created with host "10.0.0.1" and port 3000
# -----------------------------------------------------------------------
# Signal handler edge case — non-main thread
# -----------------------------------------------------------------------
Scenario: Signal handlers are skipped when not on main thread
Given a ServerLifecycle with mocked uvicorn and signal tracking
When I call _install_signal_handlers from a non-main thread
Then no signal handlers should have been installed
@@ -429,6 +429,11 @@ Feature: Services __init__ lazy-import coverage (round 3)
Then svcov3 the attribute "PersistentSessionService" should be resolved
# ---------- skeleton_compressor ----------
Scenario: svcov3 lazy-load CompressionResult
Given svcov3 a fresh services module
When svcov3 I access lazy attribute "CompressionResult"
Then svcov3 the attribute "CompressionResult" should be resolved
Scenario: svcov3 lazy-load SkeletonCompressorService
Given svcov3 a fresh services module
When svcov3 I access lazy attribute "SkeletonCompressorService"
+67 -39
View File
@@ -6,100 +6,134 @@ Feature: Skeleton compressor
Background:
Given a skeleton compressor service
# --- budget validation -------------------------------------------------
# --- ratio validation -------------------------------------------------
Scenario: Reject negative skeleton_budget
Scenario: Reject ratio below 0.0
Given context fragments with total tokens 1000
When I compress with skeleton_budget -1
Then the compressor should raise a ValueError for invalid budget
When I compress with skeleton_ratio -0.1
Then the compressor should raise a ValueError for invalid ratio
Scenario: Accept budget 0 returns empty
Scenario: Reject ratio above 1.0
Given context fragments with total tokens 1000
When I compress with skeleton_budget 0
Then the result should contain zero fragments
When I compress with skeleton_ratio 1.5
Then the compressor should raise a ValueError for invalid ratio
Scenario: Accept large budget returns all fragments
Scenario: Accept ratio 0.0
Given context fragments with total tokens 1000
When I compress with skeleton_budget 1000
When I compress with skeleton_ratio 0.0
Then all fragments should be returned unchanged
Scenario: Accept budget at boundary 500
Scenario: Accept ratio 1.0
Given context fragments with total tokens 1000
When I compress with skeleton_budget 500
When I compress with skeleton_ratio 1.0
Then only the highest-relevance fragment should be returned
Scenario: Accept ratio at boundary 0.5
Given context fragments with total tokens 1000
When I compress with skeleton_ratio 0.5
Then compressed tokens should be at most 500
# --- default handling -------------------------------------------------
Scenario: Default ratio applied when None
Given context fragments with total tokens 1000
When I compress with skeleton_ratio not specified
Then the metadata ratio should equal the default 0.15
# --- stable ordering --------------------------------------------------
Scenario: Fragments with equal relevance are ordered by id
Given three fragments with equal relevance 0.5
When I compress with skeleton_budget 10000
When I compress with skeleton_ratio 0.0
Then fragments should be ordered by fragment_id ascending
Scenario: Fragments are ordered by relevance descending
Given fragments with relevances 0.9, 0.3, and 0.7
When I compress with skeleton_budget 10000
When I compress with skeleton_ratio 0.0
Then the first fragment should have relevance 0.9
And the last fragment should have relevance 0.3
# --- metadata ----------------------------------------------------------
Scenario: Metadata records correct token counts
Given context fragments with total tokens 1000
When I compress with skeleton_ratio 0.5
Then metadata original_tokens should be 1000
And metadata compressed_tokens should be at most 500
Scenario: Metadata records source decision IDs
Given fragments with known decision IDs
When I compress with skeleton_ratio 0.0
Then metadata should contain all source decision IDs
Scenario: Metadata ratio matches input
Given context fragments with total tokens 1000
When I compress with skeleton_ratio 0.7
Then metadata ratio should be 0.7
# --- edge cases -------------------------------------------------------
Scenario: Empty fragment list compresses to empty
Given an empty fragment list
When I compress with skeleton_budget 500
When I compress with skeleton_ratio 0.5
Then the result should contain zero fragments
And metadata original_tokens should be 0
And metadata compressed_tokens should equal 0
Scenario: Single fragment fits within budget
Scenario: Single fragment at ratio 0.5
Given a single fragment with 100 tokens
When I compress with skeleton_budget 200
Then the result should contain one fragment
Scenario: Single fragment kept even when it exceeds budget
Given a single fragment with 100 tokens
When I compress with skeleton_budget 50
When I compress with skeleton_ratio 0.5
Then the result should contain one fragment
# --- argument validation ----------------------------------------------
Scenario: Reject non-tuple fragments argument
When I compress with a non-tuple fragments argument
Scenario: Reject non-list fragments argument
When I compress with a non-list fragments argument
Then the compressor should raise a TypeError
Scenario: Reject fragment with negative token count
Given a fragment with negative token count
When I compress with skeleton_budget 500
When I compress with skeleton_ratio 0.5
Then the compressor should raise a ValueError for invalid fragment
Scenario: Reject fragment with empty id
Given a fragment with empty fragment_id
When I compress with skeleton_budget 500
When I compress with skeleton_ratio 0.5
Then the compressor should raise a ValueError for invalid fragment
Scenario: Reject fragment with relevance out of range
Given a fragment with relevance 1.5
When I compress with skeleton_budget 500
When I compress with skeleton_ratio 0.5
Then the compressor should raise a ValueError for invalid fragment
Scenario: Reject non-ContextFragment item in tuple
When I compress with a tuple containing a non-fragment item
Scenario: Reject non-ContextFragment item in list
When I compress with a list containing a non-fragment item
Then the compressor should raise a TypeError for invalid item
Scenario: Reject non-integer skeleton_budget
Scenario: Reject non-numeric skeleton_ratio
Given context fragments with total tokens 1000
When I compress with a non-integer skeleton_budget
Then the compressor should raise a TypeError for invalid budget type
When I compress with a non-numeric skeleton_ratio
Then the compressor should raise a TypeError for invalid ratio type
Scenario: Reject skeleton metadata with compressed exceeding original
When I create skeleton metadata with compressed exceeding original
Then a validation error should be raised for compressed exceeding original
# --- skeleton_budget integration with plan model -----------------------
# --- compression summary (original vs compressed) ----------------------
Scenario: Compression summary stored in plan metadata
Given context fragments with total tokens 1000
When I compress with skeleton_ratio 0.6
Then compressed_tokens should be less than original_tokens
# --- skeleton_ratio integration with plan model ------------------------
Scenario: Plan model accepts skeleton_metadata
Given a skeleton metadata with ratio 0.5 and 1000 original tokens and 500 compressed
When I attach skeleton_metadata to a plan
Then the plan should expose skeleton metadata in cli dict
# --- _validate_fragments edge-cases ------------------------------------
# --- _validate_fragments edge-cases (lines 153-163) --------------------
Scenario: Reject fragment with negative token_count
Given a context fragment constructed with negative token_count
@@ -115,9 +149,3 @@ Feature: Skeleton compressor
Given a context fragment constructed with empty fragment_id
When I validate the invalid fragments
Then the compressor should raise a ValueError mentioning "fragment_id must be non-empty"
# --- structural subtype assertion -------------------------------------
Scenario: SkeletonCompressorService satisfies SkeletonCompressor protocol
When I check that SkeletonCompressorService satisfies the SkeletonCompressor protocol
Then the structural subtype assertion should pass
-60
View File
@@ -310,63 +310,3 @@ Feature: Skill YAML schema validation
And the skill config model_dump should contain key "mcp_servers"
And the skill config model_dump should contain key "agent_skill_folders"
# ────────────────────────────────────────────────────────────
# skill: wrapper key scenarios
# ────────────────────────────────────────────────────────────
@tdd_issue
@tdd_issue_1472
Scenario: Load spec-compliant YAML with skill: wrapper key
Given a spec-compliant skill YAML with skill: wrapper key
When I validate the skill schema
Then the skill schema validation should succeed
And the skill config name should be "local/wrapped-skill"
And the skill config should have 1 tools
@tdd_issue
@tdd_issue_1472
Scenario: Load spec-compliant YAML with cleveragents: header and skill: wrapper
Given a spec-compliant skill YAML with cleveragents: header and skill: wrapper
When I validate the skill schema
Then the skill schema validation should succeed
And the skill config name should be "local/wrapped-with-meta"
@tdd_issue
@tdd_issue_1472
Scenario: skill: wrapper key with None value raises ValueError
Given a skill YAML with skill: wrapper key with None value
When I validate the skill schema expecting failure
Then the skill schema validation should fail
And the skill schema error should mention "empty"
@tdd_issue
@tdd_issue_1472
Scenario: skill: wrapper key with non-dict value raises ValueError
Given a skill YAML with skill: wrapper key and string value
When I validate the skill schema expecting failure
Then the skill schema validation should fail
And the skill schema error should mention "mapping"
@tdd_issue
@tdd_issue_1472
Scenario: skill: wrapper key with list value raises ValueError
Given a skill YAML with skill: wrapper key and list value
When I validate the skill schema expecting failure
Then the skill schema validation should fail
And the skill schema error should mention "mapping"
@tdd_issue
@tdd_issue_1472
Scenario: Flat YAML without wrapper key still works (backward compatibility)
Given a skill YAML string with only the name field
When I validate the skill schema
Then the skill schema validation should succeed
And the skill config name should be "local/empty-skill"
@tdd_issue
@tdd_issue_1472
Scenario: cleveragents: header alone (flat format with metadata, no skill: wrapper)
Given a skill YAML with cleveragents: header and flat format
When I validate the skill schema
Then the skill schema validation should succeed
And the skill config name should be "local/flat-with-meta"
-68
View File
@@ -1,68 +0,0 @@
"""Shared helper functions for LSP transport BDD step definitions.
Extracted common mock-building utilities so individual step files stay
under the 500-line limit (see CONTRIBUTING.md file-size guideline).
"""
from __future__ import annotations
import json
import subprocess
from typing import Any
from unittest.mock import MagicMock
def make_mock_process(
*,
poll_return: int | None = None,
returncode: int = 0,
stdin: object | None = "auto",
stdout: object | None = "auto",
stderr: object | None = "auto",
pid: int = 12345,
) -> MagicMock:
"""Create a ``MagicMock`` resembling ``subprocess.Popen``.
Args:
poll_return: Value for :attr:`~MagicMock.poll` return value.
``None`` means the process is still running.
returncode: The subprocess exit code when ``poll()`` returns.
stdin: Stdin mock object (or ``"auto"`` for auto-generated).
stdout: Stdout mock object (or ``"auto"`` for auto-generated).
stderr: Stderr mock object (or ``"auto"`` for auto-generated).
pid: Process ID to report.
Returns:
A fully configured ``MagicMock`` with ``subprocess.Popen`` spec.
"""
proc = MagicMock(spec=subprocess.Popen)
proc.pid = pid
proc.poll.return_value = poll_return
proc.returncode = returncode
if stdin == "auto":
proc.stdin = MagicMock()
else:
proc.stdin = stdin
if stdout == "auto":
proc.stdout = MagicMock()
else:
proc.stdout = stdout
if stderr == "auto":
proc.stderr = MagicMock()
else:
proc.stderr = stderr
return proc
def build_lsp_frame(body_dict: dict[str, Any]) -> bytes:
"""Encode a dict as a Content-Length framed LSP message.
Args:
body_dict: The JSON-serialisable message body.
Returns:
Raw bytes including the ``Content-Length`` header and empty line prefix.
"""
payload = json.dumps(body_dict, separators=(",", ":")).encode("utf-8")
header = f"Content-Length: {len(payload)}\r\n\r\n".encode("ascii")
return header + payload
@@ -137,75 +137,5 @@ def step_cb_error_message_contains(context: Context, fragment: str) -> None:
)
# ---------------------------------------------------------------------------
# Static resolver fallback paths (lines 411-425, 434-452 in facade.py)
# ---------------------------------------------------------------------------
@when(r"I call the static resolver builder with a failing DI container")
def step_cb_static_actor_resolver_di_fail(context: Context) -> None:
from unittest.mock import patch
with patch(
"cleveragents.application.container.get_container",
side_effect=RuntimeError("DI not initialized"),
):
context.cb_static_resolver_result = (
A2aLocalFacade._build_actor_resolver_for_session_workflow()
)
@then(r"the static resolver result should be None")
def step_cb_static_resolver_is_none(context: Context) -> None:
assert context.cb_static_resolver_result is None, (
f"Expected None, got {context.cb_static_resolver_result!r}"
)
@when(r"I call the static options-resolver builder with a failing DI container")
def step_cb_static_options_resolver_di_fail(context: Context) -> None:
from unittest.mock import patch
with patch(
"cleveragents.application.container.get_container",
side_effect=RuntimeError("DI not initialized"),
):
context.cb_static_options_resolver_result = (
A2aLocalFacade._build_actor_options_resolver_for_session_workflow()
)
@then(r"the static options-resolver result should be None")
def step_cb_static_options_resolver_is_none(context: Context) -> None:
assert context.cb_static_options_resolver_result is None, (
f"Expected None, got {context.cb_static_options_resolver_result!r}"
)
# ---------------------------------------------------------------------------
# _provider_registry property (line 160 in facade.py)
# ---------------------------------------------------------------------------
@given(r"a coverage-boost facade wired with a mock ProviderRegistry")
def step_cb_facade_provider_registry(context: Context) -> None:
context.cb_mock_provider_registry = MagicMock()
context.cb_facade = A2aLocalFacade(
services={"provider_registry": context.cb_mock_provider_registry}
)
@when(r"I access the _provider_registry property")
def step_cb_access_provider_registry(context: Context) -> None:
context.cb_provider_registry_result = context.cb_facade._provider_registry
@then(r"the provider registry accessor should return the mock")
def step_cb_provider_registry_returns_mock(context: Context) -> None:
assert context.cb_provider_registry_result is context.cb_mock_provider_registry, (
f"Expected mock, got {context.cb_provider_registry_result!r}"
)
# Reset step matcher to parse (default) so subsequent step files are not affected
use_step_matcher("parse")
@@ -1,341 +0,0 @@
"""Step definitions for actor add combined config-actor format.
Tests for features/actor_add_combined_config_format.feature validates that
the ``agents actor add --config`` command correctly handles the spec-compliant
combined ``config.actor`` format (issue #11189).
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.actor.schema import (
ActorConfigSchema,
_detect_nested_config_actor,
_flatten_config_actor,
)
from cleveragents.cli.commands.actor import app as actor_app
from cleveragents.core.exceptions import NotFoundError
from cleveragents.domain.models.core.actor import Actor
# ──── Given / When / Then steps ------------------------------------------------
@given(
'a v3 LLM actor YAML file with combined config.actor format "{actor_string}"'
' and name "{name}"'
)
def step_llm_combined_string(context: Context, actor_string: str, name: str) -> None:
"""Create a valid v3 LLM actor YAML using combined-format string.
The YAML places all actor metadata inside ``config actor`` as a compact
``"<provider>/<model>"`` string (issue #11189).
"""
yaml_content = f"""\
name: {name}
type: llm
config:
actor: "{actor_string}"
description: Combined-format LLM actor
"""
context.config_file = _write_yaml_file(
context, name.replace("/", "_"), yaml_content
)
@given(
'a v3 LLM actor YAML file with nested config.actor format "{actor_string}"'
' and name "{name}"'
)
def step_llm_nested_dict_format(context: Context, actor_string: str, name: str) -> None:
"""Create a v3 LLM actor YAML using the nested-dict config-actor structure.
The YAML nests ``type``, ``provider``, and ``model`` inside ``config::actor``
as a dictionary (issue #11189).
"""
parts = actor_string.split("/", 1)
provider = parts[0] if len(parts) == 2 else "custom"
model = parts[1] if len(parts) == 2 else parts[0]
yaml_content = f"""\
name: {name}
type: llm
config:
actor:
type: llm
provider: {provider}
model: {model}
description: Nested-dict config-actor LLM
"""
context.config_file = _write_yaml_file(
context, name.replace("/", "_"), yaml_content
)
@given(
"a YAML config file with nested config format and combined string "
"without explicit type"
)
def step_nested_combined_no_type(context: Context) -> None:
"""Create a YAML where ``config.actor`` is a compact string but there's no v3 ``type`` at top level."""
yaml_content = """\
name: local/nested-combined-llm
description: Nested config with combined actor string
config:
actor: "gcp/gemini"
"""
context.config_file = _write_yaml_file(context, "nested-combined", yaml_content)
@given("a YAML config file with nested config and top-level name")
def step_nested_with_top_level_name(context: Context) -> None:
"""Create a YAML where ``config.actor`` holds metadata but the top-level ``name`` field is used."""
yaml_content = """\
name: local/config-with-name
description: Actor with nested config and explicit name
config:
actor: "aws/bedrock"
"""
context.config_file = _write_yaml_file(context, "config-with-name", yaml_content)
@given("a YAML config file with nested config but no name anywhere")
def step_nested_no_name(context: Context) -> None:
"""Create a YAML that uses nested config format but has NO ``name`` field at all."""
yaml_content = """\
description: Actor missing a name
config:
actor: "azure/vllm"
"""
context.config_file = _write_yaml_file(context, "nested-no-name", yaml_content)
@given("I run the actor add command")
def step_run_actor_add_generic(context: Context) -> None:
"""Run the actor-add CLI with a config file stored in context."""
if context.config_file is None:
raise ValueError("No config file set")
runner = getattr(context, "runner", CliRunner())
name = getattr(context, "actor_add_name", "local/nested-no-name")
mock_actor = Actor(
id=1,
name=name,
provider="custom",
model="default",
config_blob={"name": name},
config_hash=Actor.compute_hash({"name": name}),
graph_descriptor=None,
unsafe=False,
is_built_in=False,
is_default=False,
)
mock_registry = MagicMock()
mock_registry.upsert_actor.return_value = mock_actor
mock_registry.get_actor.side_effect = NotFoundError("not found")
mock_service = MagicMock()
with (
patch("cleveragents.cli.commands.actor._get_services") as mock_get_services,
patch.object(
ActorConfigSchema,
"model_validate",
wraps=ActorConfigSchema.model_validate,
) as schema_spy,
):
mock_get_services.return_value = (mock_service, mock_registry)
result = runner.invoke(
actor_app,
["add", name, "--config", str(context.config_file)],
catch_exceptions=True,
)
context.schema_validate_spy = schema_spy
context.cli_result = result
success = result.exit_code == 0
context.command_result = {
"success": success,
"exit_code": result.exit_code,
"output": result.output,
}
if success:
assert result.exception is None, f"Unexpected exception: {result.exception}"
context.command_error = None
context.actor_registered = True
# Capture metadata sent to upsert_actor so the shared assertion
# step ("the actor should be registered with type ...") defined in
# actor_add_v3_schema_validation_steps.py can read it.
if mock_registry.upsert_actor.called:
call_kwargs = mock_registry.upsert_actor.call_args
blob = call_kwargs.kwargs.get("config_blob") if call_kwargs.kwargs else None
if isinstance(blob, dict):
context.registered_actor_type = blob.get("type")
context.registered_actor_provider = blob.get("provider")
context.registered_actor_model = blob.get("model")
else:
context.registered_actor_type = None
context.registered_actor_provider = None
context.registered_actor_model = None
else:
context.registered_actor_type = None
context.registered_actor_provider = None
context.registered_actor_model = None
else:
assert isinstance(result.exception, SystemExit), (
f"Expected SystemExit, got: {result.exception}"
)
context.command_error = result.output
context.actor_registered = False
context.registered_actor_type = None
context.registered_actor_provider = None
context.registered_actor_model = None
@when("I run the actor add command without a name argument")
def step_run_actor_add_no_name_arg(context: Context) -> None:
"""Run the actor add CLI WITHOUT a name positional argument.
Exercises the CLI's "Actor name is required" validation path —
distinct from the v3-schema step which always supplies a name.
"""
if context.config_file is None:
raise ValueError("No config file set")
runner = getattr(context, "runner", CliRunner())
mock_registry = MagicMock()
mock_registry.get_actor.side_effect = NotFoundError("not found")
mock_service = MagicMock()
with patch("cleveragents.cli.commands.actor._get_services") as mock_get_services:
mock_get_services.return_value = (mock_service, mock_registry)
result = runner.invoke(
actor_app,
["add", "--config", str(context.config_file)],
catch_exceptions=True,
)
context.cli_result = result
success = result.exit_code == 0
context.command_result = {
"success": success,
"exit_code": result.exit_code,
"output": result.output,
}
context.command_error = None if success else result.output
context.actor_registered = success
# ──── Then steps ---------------------------------------------------------------
# Note: ``v3actor the command should succeed/fail`` and ``v3actor the error
# message should contain`` are defined in actor_add_v3_schema_validation_steps.py
# and shared across the v3-actor feature suite. Don't redefine them here —
# behave's step registry is global and duplicate registrations raise
# AmbiguousStep at suite-load time.
@then("the name should be available at top-level after flattening")
def step_name_available_flat(context: Context) -> None:
"""Assert that _flatten_config_actor produced a flat dict with a 'name' field."""
raw_blob = {"config": {"actor": {"name": "local/flat-actor"}}}
flat = _flatten_config_actor(raw_blob)
assert "name" in flat, f"Name should be top-level after flattening; blob: {flat}"
assert flat["name"] == "local/flat-actor", f"Unexpected name: {flat['name']}"
@then("the config block should be removed after flattening")
def step_config_removed_flat(context: Context) -> None:
"""Assert that _flatten_config_actor removes the top-level ``config`` key."""
raw_blob = {"config": {"actor": "aws/mistral"}, "name": "local/test"}
flat = _flatten_config_actor(raw_blob)
assert "config" not in flat, f"'config' should be removed; flat: {flat}"
# ──── is_v3_yaml / detection unit steps ----------------------------------------
@given('a config blob with combined-format string "{actor_string}"')
def step_config_blob_combined_string(context: Context, actor_string: str) -> None:
"""Store a config blob with combined-format ``config.actor`` string."""
context.config_blob_under_test = {
"name": "local/combined",
"config": {"actor": actor_string},
}
@when("I detect nested config-actor on the config blob")
def step_detect_nested_actor(context: Context) -> None:
"""Call _detect_nested_config_actor()."""
context.detect_result = _detect_nested_config_actor(context.config_blob_under_test)
@then("is_v3_yaml should return True for combined-format string")
def step_is_v3_combined_true(context: Context) -> None:
"""Assert is_v3_yaml() returned True for a combined-format blob."""
# Use the helper directly.
assert _detect_nested_config_actor(context.config_blob_under_test) is True
@given('a config blob with nested-dict actor "{actor_type}"')
def step_config_blob_nested_dict(context: Context, actor_type: str) -> None:
"""Store a config blob with a ``config.actor`` dict."""
context.config_blob_under_test = {
"name": "local/nested",
"type": actor_type,
"config": {"actor": {"provider": "aws", "model": "gpt-4"}},
}
@then("is_v3_yaml should return True for nested-dict config.actor")
def step_is_v3_nested_true(context: Context) -> None:
"""Assert is_v3_yaml() returns True for a nested-dict config-actor blob."""
assert _detect_nested_config_actor(context.config_blob_under_test) is True, (
"Should detect nested config-actor structure"
)
@given("a config blob without any combined-format actor")
def step_config_blob_no_combined(context: Context) -> None:
"""Store a plain flat config blob (no ``config.actor``)."""
context.config_blob_under_test = {
"name": "local/flat-config",
"type": "llm",
"provider": "openai",
"model": "gpt-4",
}
@then("is_v3_yaml should return False when there is no combined-format actor to detect")
def step_detect_nested_false(context: Context) -> None:
"""Assert _detect_nested_config_actor() returns False for a plain blob."""
assert not _detect_nested_config_actor(context.config_blob_under_test), (
"Should NOT detect config-actor in a flat blob"
)
# ──── Helpers ------------------------------------------------------------------
def _write_yaml_file(context: Context, name: str, content: str) -> Path:
"""Write a YAML file to the temporary directory."""
if not hasattr(context, "temp_dir") or context.temp_dir is None:
context.temp_dir = tempfile.TemporaryDirectory()
result_context_add_cleanup(context, context.temp_dir.cleanup)
safe_name = name.replace("/", "_")
file_path = Path(context.temp_dir.name) / f"{safe_name}.yaml"
file_path.write_text(content)
return file_path
def result_context_add_cleanup(ctx, fn):
"""Add a cleanup handler to the context."""
if not hasattr(ctx, "_cleanup_handlers"):
ctx._cleanup_handlers = []
ctx._cleanup_handlers.append(fn)
-2
View File
@@ -33,8 +33,6 @@ def _make_app(
app_exec = MagicMock()
app_exec.config = SimpleNamespace(global_context=dict(config_global_context or {}))
app_exec.run_single_shot = AsyncMock(return_value=result)
# last_run_tool_calls is an int property; set to 0 so comparisons work in tests
app_exec.last_run_tool_calls = 0
if run_side_effect is not None:
app_exec.run_single_shot.side_effect = run_side_effect
return app_exec
-22
View File
@@ -23,7 +23,6 @@ from cleveragents.actor.schema import (
ActorType,
EdgeDefinition,
NodeDefinition,
NodeLspBinding,
NodeType,
RouteDefinition,
)
@@ -134,27 +133,6 @@ def step_given_lsp_bindings(context: Context) -> None:
)
@given("a GRAPH actor config with a node using the lsp_binding typed field")
def step_given_lsp_typed_field(context: Context) -> None:
nodes = [
NodeDefinition(
id="coder",
type=NodeType.AGENT,
name="Coder",
description="Writes code with LSP support",
config={"agent": "coder_agent"},
lsp_binding=NodeLspBinding(
server="local/pyright",
languages=["python"],
auto=False,
),
),
]
context.actor_config = _build_graph_config(
"workflows/lsp_typed", nodes, [], "coder", ["coder"]
)
@given("a GRAPH actor config with a conditional routing node")
def step_given_conditional(context: Context) -> None:
nodes = [
@@ -1,386 +0,0 @@
"""Step definitions for actor_options_propagation.feature.
Tests that actor-level ``options`` (openai_api_base, openai_api_key,
etc.) are correctly forwarded to ``ProviderRegistry.create_llm()`` across
all call sites: Strategize phase, Execute phase, and SessionWorkflow.
All step text uses the ``aop`` prefix to avoid collisions with other
step files that manipulate ``context.mock_lifecycle`` etc.
Forgejo: #11256
"""
from types import SimpleNamespace
from unittest.mock import MagicMock
from behave import given, then, when
from cleveragents.actor.config import build_llm_kwargs_from_options
from cleveragents.application.services.llm_actors import (
LLMExecuteActor,
LLMStrategizeActor,
)
from cleveragents.application.services.strategy_actor import StrategyActor
from cleveragents.core.exceptions import ValidationError
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _aop_make_plan(action_name="test-action"):
return SimpleNamespace(action_name=action_name)
def _aop_make_action(strategy_actor=None, execution_actor=None):
return SimpleNamespace(
strategy_actor=strategy_actor,
execution_actor=execution_actor,
)
def _aop_llm_response(content):
return SimpleNamespace(content=content)
def _aop_capturing_registry():
"""Create a MagicMock registry that captures create_llm kwargs."""
llm = MagicMock()
llm.invoke.return_value = SimpleNamespace(content="[]")
captured = {}
def _fake_create(provider_type=None, model_id=None, **kw):
captured["provider_type"] = provider_type
captured["model_id"] = model_id
captured["kwargs"] = kw
return llm
reg = MagicMock()
reg.create_llm.side_effect = _fake_create
return reg, captured
# ---------------------------------------------------------------------------
# build_llm_kwargs_from_options shared utility
# ---------------------------------------------------------------------------
@given('aop actor options with "{key}" set to "{value}"')
def step_aop_given_options_with_key(context, key, value):
if not hasattr(context, "aop_options"):
context.aop_options = {}
try:
context.aop_options[key] = int(value)
except ValueError:
try:
context.aop_options[key] = float(value)
except ValueError:
context.aop_options[key] = value
@given("aop actor options is None")
def step_aop_given_options_none(context):
context.aop_options = None
@given("aop actor options is an empty dict")
def step_aop_given_options_empty(context):
context.aop_options = {}
@when("I call build_llm_kwargs_from_options")
def step_aop_when_build_llm_kwargs(context):
context.aop_result = build_llm_kwargs_from_options(context.aop_options)
@then('the result should contain "{key}" with value "{value}"')
def step_aop_then_contains_str(context, key, value):
kwargs = context.aop_result
assert key in kwargs, f"Expected '{key}' in result, got {kwargs}"
assert kwargs[key] == value, f"Expected {key}={value}, got {kwargs[key]}"
@then('the result should contain "{key}" with value {value}')
def step_aop_then_contains_int(context, key, value):
kwargs = context.aop_result
assert key in kwargs, f"Expected '{key}' in result, got {kwargs}"
try:
expected_val = int(value)
except ValueError:
expected_val = float(value)
assert kwargs[key] == expected_val, (
f"Expected {key}={expected_val}, got {kwargs[key]}"
)
@then('the result should not contain "{key}"')
def step_aop_then_not_contains(context, key):
assert key not in context.aop_result, (
f"Expected '{key}' NOT in result, got {context.aop_result}"
)
@then("the result should be an empty dict")
def step_aop_then_empty_dict(context):
assert context.aop_result == {}, f"Expected empty dict, got {context.aop_result}"
# ---------------------------------------------------------------------------
# StrategyActor options propagation
# ---------------------------------------------------------------------------
@given('aop a mock lifecycle service that resolves actor "{actor_name}"')
def step_aop_given_lifecycle(context, actor_name):
plan = _aop_make_plan()
action = _aop_make_action(strategy_actor=actor_name)
context.aop_lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value="openai/gpt-4"),
resolve_actor_options=MagicMock(return_value=None),
)
@given('aop the resolved actor options contain "{key}" set to "{value}"')
def step_aop_given_options_contain(context, key, value):
cur = context.aop_lifecycle.resolve_actor_options.return_value or {}
cur[key] = value
context.aop_lifecycle.resolve_actor_options.return_value = cur
@given("aop the resolved actor options is None")
def step_aop_given_options_is_none(context):
context.aop_lifecycle.resolve_actor_options.return_value = None
@given("aop a mock provider registry that captures create_llm kwargs with LLM response")
def step_aop_given_capturing_registry(context):
reg, captured = _aop_capturing_registry()
context.aop_registry = reg
context.aop_captured_kwargs = captured
@when('aop the StrategyActor executes with LLM for plan "{plan_id}"')
def step_aop_when_strategy_actor_executes(context, plan_id):
actor = StrategyActor(
provider_registry=context.aop_registry,
lifecycle_service=context.aop_lifecycle,
)
import contextlib
with contextlib.suppress(
ValidationError, AttributeError, KeyError, TypeError, ValueError
):
actor.execute(
plan_id=plan_id,
definition_of_done="test",
resources=None,
project_context=None,
invariants=None,
)
@then("aop create_llm should have been called")
def step_aop_then_create_llm_called(context):
assert context.aop_registry.create_llm.called, (
"Expected create_llm to have been called"
)
@then('aop create_llm kwargs should contain "{key}" with value "{value}"')
def step_aop_then_llm_kwargs_contain(context, key, value):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key in kwargs, f"Expected '{key}' in create_llm kwargs, got {kwargs}"
assert kwargs[key] == value, f"Expected {key}={value}, got {kwargs[key]}"
@then('aop create_llm kwargs should not contain "{key}"')
def step_aop_then_llm_kwargs_not_contain(context, key):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key not in kwargs, f"Expected '{key}' NOT in create_llm kwargs, got {kwargs}"
# ---------------------------------------------------------------------------
# LLMStrategizeActor options propagation
# ---------------------------------------------------------------------------
@given("aop a valid LLMStrategizeActor with actor options")
def step_aop_given_strategize_with_options(context):
reg, captured = _aop_capturing_registry()
context.aop_registry = reg
context.aop_captured_kwargs = captured
plan = _aop_make_plan()
action = _aop_make_action(strategy_actor="local/test-backend")
context.aop_lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value="openai/gpt-4"),
resolve_actor_options=MagicMock(
return_value={
"openai_api_base": "http://custom-backend:7000/v1",
"openai_api_key": "local-key",
}
),
)
context.aop_strategize_actor = LLMStrategizeActor(
provider_registry=context.aop_registry,
lifecycle_service=context.aop_lifecycle,
)
@given("aop a valid LLMStrategizeActor without actor options")
def step_aop_given_strategize_without_options(context):
reg, captured = _aop_capturing_registry()
context.aop_registry = reg
context.aop_captured_kwargs = captured
plan = _aop_make_plan()
action = _aop_make_action(strategy_actor="openai/gpt-4")
context.aop_lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value="openai/gpt-4"),
resolve_actor_options=MagicMock(return_value=None),
)
context.aop_strategize_actor = LLMStrategizeActor(
provider_registry=context.aop_registry,
lifecycle_service=context.aop_lifecycle,
)
@when("aop I call strategize execute with plan_id ")
@when('aop I call strategize execute with plan_id "{plan_id}" and stream callback')
def step_aop_when_strategize_execute(context, plan_id="PLAN_OPT"):
events = []
def callback(event_type, data):
events.append({"type": event_type, "data": data})
context.aop_strategize_result = context.aop_strategize_actor.execute(
plan_id=plan_id,
definition_of_done="Build a REST API",
stream_callback=callback,
)
@when('aop I call strategize execute with plan_id "{plan_id}" and no stream callback')
def step_aop_when_strategize_execute_no_cb(context, plan_id):
context.aop_strategize_result = context.aop_strategize_actor.execute(
plan_id=plan_id,
definition_of_done="Build a REST API",
stream_callback=None,
)
@then('aop the strategize create_llm should have received "{key}" with value "{value}"')
def step_aop_then_strategize_received(context, key, value):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key in kwargs, (
f"Expected '{key}' in strategize create_llm kwargs, got {kwargs}"
)
assert kwargs[key] == value, f"Expected {key}={value}, got {kwargs[key]}"
@then('aop the strategize create_llm should not have received "{key}"')
def step_aop_then_strategize_not_received(context, key):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key not in kwargs, (
f"Expected '{key}' NOT in strategize create_llm kwargs, got {kwargs}"
)
# ---------------------------------------------------------------------------
# LLMExecuteActor options propagation
# ---------------------------------------------------------------------------
def _aop_executing_registry(content):
"""Create a MagicMock registry + lifecycle for LLMExecuteActor tests."""
llm_resp = _aop_llm_response(content)
llm = MagicMock()
llm.invoke.return_value = llm_resp
captured = {}
def _fake_create(provider_type=None, model_id=None, **kw):
captured["provider_type"] = provider_type
captured["model_id"] = model_id
captured["kwargs"] = kw
return llm
reg = MagicMock()
reg.create_llm.side_effect = _fake_create
return reg, captured
@given("aop a valid LLMExecuteActor with actor options")
def step_aop_given_execute_with_options(context):
reg, captured = _aop_executing_registry("mock content")
context.aop_registry = reg
context.aop_captured_kwargs = captured
plan = _aop_make_plan()
action = _aop_make_action(execution_actor="local/test-executor")
context.aop_lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value="openai/gpt-4"),
resolve_actor_options=MagicMock(
return_value={
"openai_api_base": "http://custom-executor:9000/v1",
"openai_api_key": "exec-key",
}
),
)
context.aop_execute_actor = LLMExecuteActor(
provider_registry=context.aop_registry,
lifecycle_service=context.aop_lifecycle,
)
@given("aop a valid LLMExecuteActor without actor options")
def step_aop_given_execute_without_options(context):
reg, captured = _aop_executing_registry("mock content")
context.aop_registry = reg
context.aop_captured_kwargs = captured
plan = _aop_make_plan()
action = _aop_make_action(execution_actor="openai/gpt-4")
context.aop_lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value="openai/gpt-4"),
resolve_actor_options=MagicMock(return_value=None),
)
context.aop_execute_actor = LLMExecuteActor(
provider_registry=context.aop_registry,
lifecycle_service=context.aop_lifecycle,
)
@when('aop I call execute actor with plan_id "{plan_id}" and read_only {read_only}')
def step_aop_when_execute_actor(context, plan_id, read_only):
ro = read_only in ("True", "true")
try:
context.aop_execute_result = context.aop_execute_actor.execute(
plan_id=plan_id,
decisions=[],
sandbox_root=None,
stream_callback=None,
read_only=ro,
)
except (ValidationError, ValueError, RuntimeError):
context.aop_execute_result = None
@then('aop the execute create_llm should have received "{key}" with value "{value}"')
def step_aop_then_execute_received(context, key, value):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key in kwargs, f"Expected '{key}' in execute create_llm kwargs, got {kwargs}"
assert kwargs[key] == value, f"Expected {key}={value}, got {kwargs[key]}"
@then('aop the execute create_llm should not have received "{key}"')
def step_aop_then_execute_not_received(context, key):
kwargs = context.aop_captured_kwargs.get("kwargs", {})
assert key not in kwargs, (
f"Expected '{key}' NOT in execute create_llm kwargs, got {kwargs}"
)
@@ -41,8 +41,6 @@ def _make_app(
app_exec = MagicMock()
app_exec.config = SimpleNamespace(global_context=dict(config_global_context or {}))
app_exec.run_single_shot = AsyncMock(return_value=result)
# last_run_tool_calls is an int property; set to 0 so comparisons work in tests
app_exec.last_run_tool_calls = 0
return app_exec
@@ -167,7 +165,7 @@ def step_resolve_with_no_config_data(context: Any) -> None:
try:
resolve_config_files("local/empty-actor", [])
context.resolve_exit_code = 0
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
except (SystemExit, click.exceptions.Exit) as exc:
context.resolve_exit_code = getattr(
exc, "exit_code", getattr(exc, "code", 1)
)
@@ -211,7 +209,7 @@ def step_resolve_unknown_actor_directly(context: Any) -> None:
try:
resolve_config_files("nonexistent/actor", [])
context.resolve_exit_code = 0
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
except (SystemExit, click.exceptions.Exit) as exc:
context.resolve_exit_code = getattr(
exc, "exit_code", getattr(exc, "code", 1)
)
@@ -262,7 +260,7 @@ def step_resolve_with_empty_config_blob(context: Any) -> None:
try:
resolve_config_files("local/empty-blob-actor", [])
context.resolve_exit_code = 0
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
except (SystemExit, click.exceptions.Exit) as exc:
context.resolve_exit_code = getattr(
exc, "exit_code", getattr(exc, "code", 1)
)
@@ -358,7 +356,7 @@ def step_resolve_with_unserializable_config_blob(context: Any) -> None:
try:
resolve_config_files("local/bad-blob-actor", [])
context.resolve_exit_code = 0
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
except (SystemExit, click.exceptions.Exit) as exc:
context.resolve_exit_code = getattr(
exc, "exit_code", getattr(exc, "code", 1)
)
@@ -49,7 +49,7 @@ def step_resolve_with_control_character_name(context: Any) -> None:
try:
resolve_config_files(actor_name, [])
context.resolve_exit_code = 0
except (SystemExit, click.exceptions.Exit, typer.Exit) as exc:
except (SystemExit, click.exceptions.Exit) as exc:
context.resolve_exit_code = getattr(
exc, "exit_code", getattr(exc, "code", 1)
)
File diff suppressed because it is too large Load Diff
@@ -14,7 +14,6 @@ from unittest.mock import patch
import yaml
from behave import given, then, when
from behave.runner import Context
from cleveragents.actor.compiler import ActorCompilationError
from cleveragents.actor.config import ActorConfiguration
@@ -782,77 +781,3 @@ def step_graph_nodes_have_lsp_dict(context: Any) -> None:
assert lsp.get("auto") is True, (
f"Node '{node_id}' expected lsp.auto=True, got {lsp}"
)
# ── M5: options block forwarding (#11223) ─────────────────────────────────
@given("a v3 LLM actor config dict with options block")
def step_v3_llm_config_with_options(context: Context) -> None:
context.v3_config = {
"name": "local/my-llama-actor",
"type": "llm",
"description": "Local llama.cpp actor",
"provider": "openai",
"model": "my-local-model",
"options": {
"openai_api_base": "http://localhost:8080/v1",
"openai_api_key": "none",
"temperature": 1.0,
},
"system_prompt": "You are a helpful assistant.",
}
@given("a v3 LLM actor config dict without options block")
def step_v3_llm_config_without_options(context: Context) -> None:
context.v3_config = {
"name": "local/standard-actor",
"type": "llm",
"description": "Standard actor without options",
"provider": "openai",
"model": "gpt-4",
"system_prompt": "You are a helpful assistant.",
}
@then("the agent config should contain the options block")
def step_agent_config_has_options(context: Context) -> None:
agents = context.reactive_config.agents
agent = next(iter(agents.values()))
options = agent.config.get("options")
assert options == {
"openai_api_base": "http://localhost:8080/v1",
"openai_api_key": "none",
"temperature": 1.0,
}, f"Expected exact options dict, got {options}"
@then("the agent config should not contain an options key")
def step_agent_config_no_options(context: Context) -> None:
agents = context.reactive_config.agents
agent = next(iter(agents.values()))
assert "options" not in agent.config, (
f"Expected no 'options' key in agent config, got {agent.config}"
)
@given("a v3 LLM actor config dict with empty options block")
def step_v3_llm_config_with_empty_options(context: Context) -> None:
context.v3_config = {
"name": "local/empty-options-actor",
"type": "llm",
"description": "Actor with empty options",
"provider": "openai",
"model": "gpt-4",
"options": {},
"system_prompt": "You are a helpful assistant.",
}
@then("the agent config should contain an empty options block")
def step_agent_config_has_empty_options(context: Context) -> None:
agents = context.reactive_config.agents
agent = next(iter(agents.values()))
options = agent.config.get("options")
assert options == {}, f"Expected empty options dict, got {options}"
+1 -1
View File
@@ -89,7 +89,7 @@ def _make_service(
lifecycle.get_plan.return_value = plan
lifecycle.complete_apply.return_value = plan
lifecycle.constrain_apply.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
store = MagicMock()
if changeset is not None:
@@ -13,7 +13,6 @@ Split from ``audit_service_wiring_steps.py`` to stay within the
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import cast
from behave import given, then, when
@@ -222,76 +221,3 @@ def step_audit_entry_user_identity_none(context: Context) -> None:
assert entries[0].user_identity is None, (
f"Expected user_identity=None, got {entries[0].user_identity!r}"
)
# ── #719: DomainEvent.timestamp preservation ─────────────────────
@when("a plan_applied event with a known timestamp is emitted")
def step_emit_plan_applied_with_known_timestamp(context: Context) -> None:
context.known_timestamp = datetime(2024, 6, 15, 10, 30, 0, tzinfo=UTC)
context.event_bus.emit(
DomainEvent(
event_type=EventType.PLAN_APPLIED,
plan_id="PLAN-TS",
timestamp=context.known_timestamp,
)
)
@then("the audit entry created_at should match the event timestamp")
def step_audit_entry_matches_event_timestamp(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="plan_applied")
assert len(entries) >= 1, "No plan_applied entries found"
expected = context.known_timestamp.strftime("%Y-%m-%dT%H:%M:%S.%f")
assert entries[0].created_at == expected, (
f"Expected created_at={expected!r}, got {entries[0].created_at!r}"
)
@given("a direct audit service instance")
def step_direct_audit_service(context: Context) -> None:
context.audit_service = _fresh_audit_service()
@when("record is called with an explicit timestamp")
def step_record_with_explicit_timestamp(context: Context) -> None:
context.explicit_timestamp = datetime(2023, 1, 1, 12, 0, 0, tzinfo=UTC)
context.audit_service.record(
event_type="plan_applied",
plan_id="PLAN-EXPLICIT-TS",
timestamp=context.explicit_timestamp,
)
@then("the audit entry created_at should equal the explicit timestamp")
def step_audit_entry_equals_explicit_timestamp(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="plan_applied")
assert len(entries) >= 1, "No plan_applied entries found"
expected = context.explicit_timestamp.strftime("%Y-%m-%dT%H:%M:%S.%f")
assert entries[0].created_at == expected, (
f"Expected created_at={expected!r}, got {entries[0].created_at!r}"
)
@when("record is called without a timestamp")
def step_record_without_timestamp(context: Context) -> None:
context.before_record = datetime.now(tz=UTC)
context.audit_service.record(
event_type="plan_applied",
plan_id="PLAN-AUTO-TS",
)
context.after_record = datetime.now(tz=UTC)
@then("the audit entry created_at should be close to now")
def step_audit_entry_close_to_now(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="plan_applied")
assert len(entries) >= 1, "No plan_applied entries found"
recorded = datetime.fromisoformat(entries[0].created_at).replace(tzinfo=UTC)
tolerance = timedelta(seconds=60)
lower = context.before_record - tolerance
upper = context.after_record + tolerance
assert lower <= recorded <= upper, (
f"created_at={recorded!r} not within 60s of call time"
)
+1 -1
View File
@@ -921,7 +921,7 @@ def _make_guardrail_mock_lifecycle(plan: MagicMock) -> MagicMock:
lcs.start_execute = MagicMock()
lcs.complete_execute = MagicMock()
lcs.fail_execute = MagicMock()
lcs.commit_plan = MagicMock()
lcs._commit_plan = MagicMock()
return lcs
@@ -64,7 +64,7 @@ def _make_lifecycle_mock(plan: Any) -> MagicMock:
lcs.start_execute = MagicMock()
lcs.complete_execute = MagicMock()
lcs.fail_execute = MagicMock()
lcs.commit_plan = MagicMock()
lcs._commit_plan = MagicMock()
return lcs
@@ -1,151 +0,0 @@
"""Step definitions for CLI output format validation (rich / color formats)."""
from __future__ import annotations
import json
import yaml
from behave import then
from behave.runner import Context
@then("the format result should be valid JSON dict")
def step_format_result_json_dict(context: Context) -> None:
parsed = json.loads(context.format_result)
assert isinstance(parsed, dict)
# Verify spec-required envelope fields are present
for field in ("command", "status", "exit_code", "data", "timing", "messages"):
assert field in parsed, f"Envelope field '{field}' missing from JSON result"
# Verify data field contains the original dict
assert isinstance(parsed["data"], dict)
@then("the format result should be valid YAML dict")
def step_format_result_yaml_dict(context: Context) -> None:
parsed = yaml.safe_load(context.format_result)
assert isinstance(parsed, dict)
# Verify spec-required envelope fields are present
for field in ("command", "status", "exit_code", "data", "timing", "messages"):
assert field in parsed, f"Envelope field '{field}' missing from YAML result"
# Verify data field contains the original dict
assert isinstance(parsed["data"], dict)
@then("the format result should contain plain key-value pairs")
def step_format_result_plain(context: Context) -> None:
assert "name: test" in context.format_result
assert "nested:" in context.format_result
assert "items:" in context.format_result
@then("the format result should contain table output")
def step_format_result_table(context: Context) -> None:
assert "name" in context.format_result
assert "count" in context.format_result
@then("the format result should contain separator lines")
def step_format_result_separator(context: Context) -> None:
assert "---" in context.format_result
@then("the serialized result should have string enum values")
def step_serialized_enum_values(context: Context) -> None:
assert context.serialized["state"] == "available"
assert isinstance(context.serialized["nested"], dict)
assert context.serialized["items"][0] == "archived"
assert isinstance(context.serialized["ts"], str)
@then("the format result should contain rich styled output")
def step_format_result_rich_styled(context: Context) -> None:
"""Assert that rich format produces styled output (ANSI codes or key-value lines).
The rich format must NOT produce raw JSON it must produce styled terminal
output via the RichMaterializer (ANSI escape codes or structured key-value
lines with panel headers).
"""
result = context.format_result
# Rich output must contain the data keys
assert "name" in result, f"Rich output missing 'name': {result!r}"
assert "count" in result, f"Rich output missing 'count': {result!r}"
# Rich output must contain styled panel header or ANSI codes
has_ansi = "\033[" in result or "\x1b[" in result
has_panel_header = "Output" in result
assert has_ansi or has_panel_header, (
f"Rich output has no ANSI codes or panel header — got: {result!r}"
)
@then("the format result should not be valid JSON")
def step_format_result_not_json(context: Context) -> None:
"""Assert that the format result is NOT valid JSON (i.e. not the JSON fallback)."""
try:
json.loads(context.format_result)
raise AssertionError(
f"format_output() with 'rich' returned valid JSON — "
f"expected styled terminal output, got: {context.format_result!r}"
)
except (json.JSONDecodeError, ValueError):
pass # Expected: rich output is not JSON
@then("the format result should contain color styled output")
def step_format_result_color_styled(context: Context) -> None:
"""Assert that color format produces ANSI-colored output (not plain text).
The color format must NOT produce plain text it must produce ANSI-colored
terminal output via the ColorMaterializer.
"""
result = context.format_result
# Color output must contain the data keys
assert "name" in result, f"Color output missing 'name': {result!r}"
assert "count" in result, f"Color output missing 'count': {result!r}"
# Color output must contain ANSI escape codes or styled panel header
has_ansi = "\033[" in result or "\x1b[" in result
has_panel_header = "Output" in result
assert has_ansi or has_panel_header, (
f"Color output has no ANSI codes or panel header — got: {result!r}"
)
@then("the format result should not be plain text only")
def step_format_result_not_plain_text(context: Context) -> None:
"""Assert that the color format result is NOT plain text (i.e. not the plain fallback).
Plain text would look like 'name: test\\ncount: 42' with no ANSI codes.
Color output must have ANSI escape codes or styled panel headers.
"""
result = context.format_result
has_ansi = "\033[" in result or "\x1b[" in result
has_panel_header = "Output" in result
# If it's purely plain text (no ANSI, no panel header), that's the bug
is_plain_only = not has_ansi and not has_panel_header
assert not is_plain_only, (
f"format_output() with 'color' returned plain text — "
f"expected ANSI-colored output, got: {result!r}"
)
@then("the format result should contain rich styled list output")
def step_format_result_rich_list(context: Context) -> None:
"""Assert rich format produces styled output for list data (panel per item)."""
result = context.format_result
assert "name" in result, f"Rich list output missing 'name': {result!r}"
has_ansi = "\033[" in result or "\x1b[" in result
has_item_panel = "Item 1" in result or "Item 2" in result
assert has_ansi or has_item_panel, (
f"Rich list output has no ANSI codes or item panels — got: {result!r}"
)
@then("the format result should contain color styled list output")
def step_format_result_color_list(context: Context) -> None:
"""Assert color format produces styled output for list data (panel per item)."""
result = context.format_result
assert "name" in result, f"Color list output missing 'name': {result!r}"
has_ansi = "\033[" in result or "\x1b[" in result
has_item_panel = "Item 1" in result or "Item 2" in result
assert has_ansi or has_item_panel, (
f"Color list output has no ANSI codes or item panels — got: {result!r}"
)
+51 -24
View File
@@ -441,12 +441,6 @@ def step_call_format_rich(context: Context) -> None:
context.format_result = _capture_format_output(data, "rich")
@when("I call format_output with a dict and format color")
def step_call_format_color(context: Context) -> None:
data = {"name": "test", "count": 42}
context.format_result = _capture_format_output(data, "color")
@when("I call format_output with a list and format plain")
def step_call_format_list_plain(context: Context) -> None:
data = [
@@ -456,24 +450,6 @@ def step_call_format_list_plain(context: Context) -> None:
context.format_result = _capture_format_output(data, "plain")
@when("I call format_output with a list and format rich")
def step_call_format_list_rich(context: Context) -> None:
data = [
{"name": "a", "count": 1},
{"name": "b", "count": 2},
]
context.format_result = _capture_format_output(data, "rich")
@when("I call format_output with a list and format color")
def step_call_format_list_color(context: Context) -> None:
data = [
{"name": "a", "count": 1},
{"name": "b", "count": 2},
]
context.format_result = _capture_format_output(data, "color")
@when("I call serialize_value with enum and nested data")
def step_call_serialize_with_enum(context: Context) -> None:
context.serialized = _serialize_value(
@@ -484,3 +460,54 @@ def step_call_serialize_with_enum(context: Context) -> None:
"ts": datetime(2025, 6, 1, 12, 0),
}
)
# ------- Additional Then steps -------
@then("the format result should be valid JSON dict")
def step_format_result_json_dict(context: Context) -> None:
parsed = json.loads(context.format_result)
assert isinstance(parsed, dict)
# Verify spec-required envelope fields are present
for field in ("command", "status", "exit_code", "data", "timing", "messages"):
assert field in parsed, f"Envelope field '{field}' missing from JSON result"
# Verify data field contains the original dict
assert isinstance(parsed["data"], dict)
@then("the format result should be valid YAML dict")
def step_format_result_yaml_dict(context: Context) -> None:
parsed = yaml.safe_load(context.format_result)
assert isinstance(parsed, dict)
# Verify spec-required envelope fields are present
for field in ("command", "status", "exit_code", "data", "timing", "messages"):
assert field in parsed, f"Envelope field '{field}' missing from YAML result"
# Verify data field contains the original dict
assert isinstance(parsed["data"], dict)
@then("the format result should contain plain key-value pairs")
def step_format_result_plain(context: Context) -> None:
assert "name: test" in context.format_result
assert "nested:" in context.format_result
assert "items:" in context.format_result
@then("the format result should contain table output")
def step_format_result_table(context: Context) -> None:
assert "name" in context.format_result
assert "count" in context.format_result
@then("the format result should contain separator lines")
def step_format_result_separator(context: Context) -> None:
assert "---" in context.format_result
@then("the serialized result should have string enum values")
def step_serialized_enum_values(context: Context) -> None:
assert context.serialized["state"] == "available"
assert isinstance(context.serialized["nested"], dict)
assert context.serialized["items"][0] == "archived"
assert isinstance(context.serialized["ts"], str)
-389
View File
@@ -1,389 +0,0 @@
"""Step definitions for container_handler.feature.
Tests for the container infrastructure resource handler module
(cleveragents.resource.handlers.container).
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
from behave import given, then, when # type: ignore[attr-defined]
from cleveragents.domain.models.core.resource import (
PhysVirt,
Resource,
ResourceCapabilities,
)
from cleveragents.resource.handlers._base import EMPTY_CONTENT_HASH
from cleveragents.resource.handlers.protocol import ResourceHandler
# ── Handler class registry ───────────────────────────────────
def _get_handler_class(class_name: str) -> type:
"""Import and return a handler class from the container module."""
from cleveragents.resource.handlers import container as _container_mod
cls = getattr(_container_mod, class_name, None)
if cls is None:
raise AssertionError(
f"Class '{class_name}' not found in "
"cleveragents.resource.handlers.container"
)
return cls
def _make_container_resource(
resource_type: str,
location: str | None = None,
) -> Resource:
"""Create a minimal container resource for testing."""
return Resource(
resource_id="01KJ5C5TPMP8GGX3QC83E2MAQS",
resource_type_name=resource_type,
classification=PhysVirt.PHYSICAL,
location=location,
properties={},
capabilities=ResourceCapabilities(
readable=True,
writable=False,
sandboxable=False,
checkpointable=False,
),
)
# ── When steps ───────────────────────────────────────────────
@when("I import the container handler module")
def step_import_container_module(context: Any) -> None:
"""Attempt to import the container handler module."""
try:
import cleveragents.resource.handlers.container as _mod
context.import_error = None # type: ignore[attr-defined]
context.container_module = _mod # type: ignore[attr-defined]
except Exception as exc:
context.import_error = exc # type: ignore[attr-defined]
@when('I import "{class_name}" from the container handler module')
def step_import_class_from_container_module(context: Any, class_name: str) -> None:
"""Attempt to import a specific class from the container handler module."""
try:
cls = _get_handler_class(class_name)
context.imported_class = cls # type: ignore[attr-defined]
context.import_error = None # type: ignore[attr-defined]
except Exception as exc:
context.import_error = exc # type: ignore[attr-defined]
context.imported_class = None # type: ignore[attr-defined]
@when('I resolve the handler "<handler_ref>"')
def step_resolve_handler_ref_template(context: Any, handler_ref: str) -> None:
"""Resolve a handler by reference string (template step)."""
step_resolve_handler_ref(context, handler_ref)
@when('I resolve the handler "{handler_ref}"')
def step_resolve_handler_ref(context: Any, handler_ref: str) -> None:
"""Resolve a handler by reference string."""
from cleveragents.resource.handlers.resolver import (
HandlerResolutionError,
clear_handler_cache,
resolve_handler,
)
clear_handler_cache()
try:
context.resolved_handler = resolve_handler(handler_ref) # type: ignore[attr-defined]
context.resolution_error = None # type: ignore[attr-defined]
except HandlerResolutionError as exc:
context.resolution_error = exc # type: ignore[attr-defined]
context.resolved_handler = None # type: ignore[attr-defined]
@when('I resolve the handler for resource type "{resource_type}"')
def step_resolve_handler_for_type(context: Any, resource_type: str) -> None:
"""Resolve a handler for a resource type via the registry."""
from cleveragents.application.services._resource_registry_data import BUILTIN_TYPES
from cleveragents.resource.handlers.resolver import (
HandlerResolutionError,
clear_handler_cache,
resolve_handler,
)
clear_handler_cache()
handler_ref = None
for type_def in BUILTIN_TYPES:
if type_def["name"] == resource_type:
handler_ref = type_def.get("handler")
break
if handler_ref is None:
context.resolution_error = AssertionError( # type: ignore[attr-defined]
f"Resource type '{resource_type}' not found in BUILTIN_TYPES"
)
context.resolved_handler = None # type: ignore[attr-defined]
return
try:
context.resolved_handler = resolve_handler(handler_ref) # type: ignore[attr-defined]
context.resolution_error = None # type: ignore[attr-defined]
except HandlerResolutionError as exc:
context.resolution_error = exc # type: ignore[attr-defined]
context.resolved_handler = None # type: ignore[attr-defined]
@when("I call resolve on the container handler")
def step_call_resolve(context: Any) -> None:
"""Call resolve() on the handler and capture any exception."""
mock_sandbox = MagicMock()
try:
context.handler.resolve( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
plan_id="test-plan-01",
slot_name="test-slot",
sandbox_manager=mock_sandbox,
access="read_only",
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call read on the container handler")
def step_call_read(context: Any) -> None:
"""Call read() on the handler and capture any exception."""
try:
context.handler.read( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when('I call write on the container handler with data b"test"')
def step_call_write(context: Any) -> None:
"""Call write() on the handler and capture any exception."""
try:
context.handler.write( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
path="",
data=b"test",
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call delete on the container handler")
def step_call_delete(context: Any) -> None:
"""Call delete() on the handler and capture any exception."""
try:
context.handler.delete( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call list_children on the container handler")
def step_call_list_children(context: Any) -> None:
"""Call list_children() on the handler and capture any exception."""
try:
context.handler.list_children( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when('I call diff on the container handler with other_location "/other"')
def step_call_diff(context: Any) -> None:
"""Call diff() on the handler and capture any exception."""
try:
context.handler.diff( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
other_location="/other",
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call discover_children on the container handler")
def step_call_discover_children(context: Any) -> None:
"""Call discover_children() on the handler and capture any exception."""
try:
context.handler.discover_children( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call create_checkpoint on the container handler")
def step_call_create_checkpoint(context: Any) -> None:
"""Call create_checkpoint() on the handler and capture any exception."""
mock_sandbox = MagicMock()
try:
context.handler.create_checkpoint( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
plan_id="test-plan-01",
sandbox_manager=mock_sandbox,
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call rollback_to on the container handler")
def step_call_rollback_to(context: Any) -> None:
"""Call rollback_to() on the handler and capture any exception."""
mock_sandbox = MagicMock()
try:
context.handler.rollback_to( # type: ignore[attr-defined]
resource=context.container_resource, # type: ignore[attr-defined]
plan_id="test-plan-01",
checkpoint_id="ckpt-001",
sandbox_manager=mock_sandbox,
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
@when("I call content_hash on the container handler")
def step_call_content_hash(context: Any) -> None:
"""Call content_hash() on the handler and capture the result."""
try:
context.content_hash_result = context.handler.content_hash( # type: ignore[attr-defined]
context.container_resource # type: ignore[attr-defined]
)
context.raised_exception = None # type: ignore[attr-defined]
except Exception as exc:
context.raised_exception = exc # type: ignore[attr-defined]
context.content_hash_result = None # type: ignore[attr-defined]
# ── Given steps ──────────────────────────────────────────────
@given('a "{class_name}" instance')
def step_handler_instance(context: Any, class_name: str) -> None:
"""Create a handler instance by class name."""
cls = _get_handler_class(class_name)
context.handler = cls() # type: ignore[attr-defined]
context.handler_class_name = class_name # type: ignore[attr-defined]
@given('a container resource of type "{resource_type}" with location "{location}"')
def step_container_resource_with_location(
context: Any, resource_type: str, location: str
) -> None:
"""Create a container resource with a specific location."""
context.container_resource = _make_container_resource( # type: ignore[attr-defined]
resource_type=resource_type,
location=location,
)
@given('a container resource of type "{resource_type}" with no location')
def step_container_resource_no_location(context: Any, resource_type: str) -> None:
"""Create a container resource with no location."""
context.container_resource = _make_container_resource( # type: ignore[attr-defined]
resource_type=resource_type,
location=None,
)
# ── Then steps ───────────────────────────────────────────────
@then("the container handler module import should succeed without errors")
def step_import_succeeded(context: Any) -> None:
"""Assert that the import succeeded."""
assert context.import_error is None, ( # type: ignore[attr-defined]
f"Import failed with: {context.import_error}" # type: ignore[attr-defined]
)
@then("the class should be importable")
def step_class_importable(context: Any) -> None:
"""Assert that the class was imported successfully."""
assert context.import_error is None, ( # type: ignore[attr-defined]
f"Class import failed with: {context.import_error}" # type: ignore[attr-defined]
)
assert context.imported_class is not None # type: ignore[attr-defined]
@then("the instance should satisfy the ResourceHandler protocol")
def step_satisfies_protocol(context: Any) -> None:
"""Assert that the handler instance satisfies ResourceHandler protocol."""
assert isinstance(context.handler, ResourceHandler), ( # type: ignore[attr-defined]
f"{context.handler_class_name} does not satisfy ResourceHandler protocol" # type: ignore[attr-defined]
)
@then("the handler should resolve without HandlerResolutionError")
def step_handler_resolved(context: Any) -> None:
"""Assert that the handler resolved without error."""
assert context.resolution_error is None, ( # type: ignore[attr-defined]
f"Handler resolution failed: {context.resolution_error}" # type: ignore[attr-defined]
)
assert context.resolved_handler is not None # type: ignore[attr-defined]
@then('the handler type_label should be "{expected_label}"')
def step_handler_type_label(context: Any, expected_label: str) -> None:
"""Assert the handler's _type_label class variable."""
actual = getattr(context.handler, "_type_label", None) # type: ignore[attr-defined]
assert actual == expected_label, (
f"Expected _type_label='{expected_label}', got '{actual}'"
)
@then("a container handler NotImplementedError should be raised")
def step_not_implemented_raised(context: Any) -> None:
"""Assert that a NotImplementedError was raised."""
exc = context.raised_exception # type: ignore[attr-defined]
assert exc is not None, "Expected NotImplementedError but no exception was raised"
assert isinstance(exc, NotImplementedError), (
f"Expected NotImplementedError but got {type(exc).__name__}: {exc}"
)
@then("the content_hash result should be the EMPTY_CONTENT_HASH sentinel")
def step_content_hash_is_empty(context: Any) -> None:
"""Assert that content_hash returned the EMPTY_CONTENT_HASH sentinel."""
result = context.content_hash_result # type: ignore[attr-defined]
assert result == EMPTY_CONTENT_HASH, (
f"Expected EMPTY_CONTENT_HASH but got: {result!r}"
)
@then("the content_hash result should be a non-empty hex string")
def step_content_hash_is_hex(context: Any) -> None:
"""Assert that content_hash returned a non-empty hex string."""
result = context.content_hash_result # type: ignore[attr-defined]
assert result is not None, "content_hash returned None"
assert isinstance(result, str), f"Expected str, got {type(result)}"
assert len(result) > 0, "content_hash returned empty string"
assert result != EMPTY_CONTENT_HASH, (
"content_hash returned EMPTY_CONTENT_HASH but expected identity hash"
)
# Verify it's a valid hex string
try:
int(result, 16)
except ValueError as exc:
raise AssertionError(
f"content_hash result is not a valid hex string: {result!r}"
) from exc
+1 -1
View File
@@ -460,7 +460,7 @@ def step_then_tier_svc_singleton(context: Any) -> None:
def step_then_settings_hot(context: Any) -> None:
s = Settings()
assert hasattr(s, "context_max_tokens_hot")
assert s.context_max_tokens_hot == 32000 # Doubled from 16000 to 32000 (M5 fix)
assert s.context_max_tokens_hot == 16000
@then("settings should have context_max_decisions_warm")
@@ -1,85 +0,0 @@
"""Step definitions for db_url_sanitisation.feature.
Tests the ``_sanitise_db_url`` helper and its use in ``build_info_data``.
"""
from __future__ import annotations
from pathlib import Path
from tempfile import mkdtemp
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
# ---------------------------------------------------------------------------
# Given
# ---------------------------------------------------------------------------
@given('the database url is "{url}"')
def step_db_url_given(context: Context, url: str) -> None:
"""Store the raw database URL for processing."""
context.raw_db_url = url
@given('a mock settings object with database url "{db_url}"')
def step_mock_settings_with_db_url(context: Context, db_url: str) -> None:
"""Build a full mock Settings that ``build_info_data`` expects."""
tmpdir = mkdtemp()
mock_settings = MagicMock()
mock_settings.database_url = db_url
mock_settings.data_dir = Path(tmpdir)
mock_settings.storage_path = Path(tmpdir)
mock_settings.config_path = Path(tmpdir) / "config.toml"
mock_settings.log_dir = Path(tmpdir) / "logs"
mock_settings.default_automation_profile = "auto"
mock_settings.has_provider_configured = MagicMock(return_value=False)
mock_settings.configured_provider_names = MagicMock(return_value=[])
mock_settings.debug_enabled = False
context.mock_settings = mock_settings
# ---------------------------------------------------------------------------
# When
# ---------------------------------------------------------------------------
@when("I run sanitise_db_url")
def step_run_sanitise(context: Context) -> None:
"""Call _sanitise_db_url with the stored raw URL."""
from cleveragents.cli.commands.system import _sanitise_db_url
context.sanitised_url = _sanitise_db_url(context.raw_db_url)
@when("I call build_info_data")
def step_call_build_info(context: Context) -> None:
"""Call ``build_info_data`` with the mock settings."""
from cleveragents.cli.commands.system import build_info_data
with patch(
"cleveragents.config.settings.get_settings",
return_value=context.mock_settings,
):
context.info_data = build_info_data()
# ---------------------------------------------------------------------------
# Then
# ---------------------------------------------------------------------------
@then('the sanitised database url should be "{expected}"')
def step_assert_sanitised_url(context: Context, expected: str) -> None:
"""Verify the sanitised URL matches expectation."""
actual = context.sanitised_url
assert actual == expected, f"Expected '{expected}', got '{actual}'"
@then('the database field in info data should be "{expected}"')
def step_assert_info_db_field(context: Context, expected: str) -> None:
"""Verify the sanitised URL appears correctly in build_info_data output."""
actual = context.info_data["database"]
assert actual == expected, f"Expected database field '{expected}', got '{actual}'"
@@ -1,81 +0,0 @@
"""Step implementations for decomposition_max_child_depth.feature.
Tests the enforcement of max_child_depth in DecompositionService._build_hierarchy.
Reuses existing steps from large_project_decomposition_steps.py where applicable.
Only new, unique step patterns are defined here.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import patch
from behave import then, when # type: ignore[import-untyped]
from cleveragents.application.services.decomposition_models import (
DecompositionConfig,
)
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
_DECOMP_SVC_LOGGER = "cleveragents.application.services.decomposition_service.logger"
# ---------------------------------------------------------------------------
# Whens
# ---------------------------------------------------------------------------
@when("I decompose with max_child_depth {mcd:d} and max_depth {md:d}")
def step_when_decompose_max_child_depth_and_max_depth(
context: Any,
mcd: int,
md: int,
) -> None:
cfg = DecompositionConfig(
max_child_depth=mcd,
max_depth=md,
max_files_per_subplan=30,
)
with patch(_DECOMP_SVC_LOGGER) as mock_logger:
context.result = context.svc.decompose(context.files, cfg)
context.mock_logger = mock_logger
@when("I decompose with default max_child_depth")
def step_when_decompose_default_mcd(context: Any) -> None:
cfg = DecompositionConfig(
max_child_depth=5,
max_files_per_subplan=30,
)
with patch(_DECOMP_SVC_LOGGER) as mock_logger:
context.result = context.svc.decompose(context.files, cfg)
context.mock_logger = mock_logger
@when("I create a config with max_child_depth {n:d}")
def step_when_create_bad_max_child_depth(context: Any, n: int) -> None:
try:
DecompositionConfig(max_child_depth=n)
except ValueError as exc:
context.error = exc
# ---------------------------------------------------------------------------
# Thens
# ---------------------------------------------------------------------------
@then('the decomposition warning log should contain "{text}"')
def step_then_warning_log_contains(context: Any, text: str) -> None:
assert context.mock_logger is not None, "Mock logger was not set"
assert context.mock_logger.warning.called, "Expected logger.warning to be called"
args_str = str(context.mock_logger.warning.call_args_list)
assert text in args_str, f"Expected '{text}' in warning calls, got: {args_str}"
@then("max_child_depth should be {n:d}")
def step_then_cfg_max_child_depth(context: Any, n: int) -> None:
assert context.decomp_config.max_child_depth == n
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -63,7 +63,7 @@ def _make_service(
"""Create an ErrorRecoveryService with mocked lifecycle."""
lifecycle = MagicMock()
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
return ErrorRecoveryService(
lifecycle_service=lifecycle,
auto_retry_threshold=auto_retry_threshold,
@@ -574,7 +574,7 @@ def step_given_executor_with_recovery(context: Context) -> None:
plan.definition_of_done = "- Step one\n- Step two"
lifecycle.get_plan.return_value = plan
lifecycle.update_error_details = MagicMock()
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
# Build error recovery service wrapping the same lifecycle
er_service = ErrorRecoveryService(
@@ -646,7 +646,7 @@ def step_given_executor_max_retries(context: Context, n: str) -> None:
plan.definition_of_done = "- Step one"
lifecycle.get_plan.return_value = plan
lifecycle.update_error_details = MagicMock()
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
er_service = ErrorRecoveryService(
lifecycle_service=lifecycle,
+2 -1
View File
@@ -365,7 +365,8 @@ def step_use_estimation_action_and_complete_strategize(context: Context) -> None
# Manually set processing state to PROCESSING then COMPLETE
plan = context.est_lifecycle.get_plan(plan_id)
plan.processing_state = ProcessingState.PROCESSING
context.est_lifecycle.commit_plan(plan)
# TODO: Use public save_plan() once test helpers are refactored
context.est_lifecycle._commit_plan(plan)
context.est_lifecycle.complete_strategize(plan_id)
@@ -85,7 +85,7 @@ def _drive_to_execute(svc: PlanLifecycleService, plan_id: str) -> None:
svc.start_strategize(plan_id)
plan = svc.get_plan(plan_id)
plan.processing_state = ProcessingState.PROCESSING
svc.commit_plan(plan)
svc._commit_plan(plan)
svc.complete_strategize(plan_id)
# complete_strategize calls auto_progress; if still in STRATEGIZE/COMPLETE
# we need to call execute_plan explicitly.
@@ -300,7 +300,7 @@ def step_use_and_complete_strategize_lifecycle(context: Context) -> None:
context.lifecycle_svc.start_strategize(plan_id)
p = context.lifecycle_svc.get_plan(plan_id)
p.processing_state = ProcessingState.PROCESSING
context.lifecycle_svc.commit_plan(p)
context.lifecycle_svc._commit_plan(p)
context.lifecycle_svc.complete_strategize(plan_id)
context.lifecycle_plan = context.lifecycle_svc.get_plan(plan_id)
@@ -426,7 +426,7 @@ def step_use_and_complete_strategize_skip(context: Context) -> None:
context.skip_svc.start_strategize(plan_id)
p = context.skip_svc.get_plan(plan_id)
p.processing_state = ProcessingState.PROCESSING
context.skip_svc.commit_plan(p)
context.skip_svc._commit_plan(p)
context.skip_svc.complete_strategize(plan_id)
context.skip_plan = context.skip_svc.get_plan(plan_id)
@@ -86,7 +86,7 @@ def _create_plan_in_phase(
plan = svc.use_action(action_name)
plan.processing_state = ProcessingState.PROCESSING
plan.phase = phase
svc.commit_plan(plan)
svc._commit_plan(plan)
return plan.identity.plan_id
@@ -61,7 +61,7 @@ def step_mock_errored_plan(context: object, exc_type: str) -> None:
}
)
mock_service.commit_plan.side_effect = fake_commit
mock_service._commit_plan.side_effect = fake_commit
mock_executor = MagicMock()
@@ -203,7 +203,7 @@ def step_invoke_execute(context: object) -> None:
@then("the plan should have been reset to execute/queued for eer")
def step_check_reset(context: object) -> None:
committed = context.eer_committed_plans
assert len(committed) >= 1, "Expected at least one commit_plan call"
assert len(committed) >= 1, "Expected at least one _commit_plan call"
first = committed[0]
assert first["phase"] == PlanPhase.EXECUTE
assert first["processing_state"] == ProcessingState.QUEUED
@@ -12,7 +12,6 @@ Exercises all uncovered lines in
from __future__ import annotations
import json
from typing import Any
from unittest.mock import MagicMock, patch
@@ -938,112 +937,3 @@ def step_epcov_assembled_has_data(context: Context) -> None:
assert result.budget_used >= 0.0, "Expected non-negative budget_used"
assert result.context_hash, "Expected non-empty context_hash"
assert result.strategies_used, "Expected non-empty strategies_used"
# ---------------------------------------------------------------------------
# _resolve_hot_max_tokens: reads hot_max_tokens from context_policy_json
# ---------------------------------------------------------------------------
def _make_assembler_with_policy_json(
policy_json: str | None,
) -> ACMSExecutePhaseContextAssembler:
"""Build an assembler whose DB session returns a row with *policy_json*."""
pr = _make_pipeline_result()
pr.fragments = ()
pr.total_tokens = 5
pr.budget_used = 0.1
pr.strategies_used = ("relevance",)
pr.context_hash = "hash-policy"
pr.preamble = None
pr.provenance_map = {}
mock_pipeline = MagicMock()
mock_pipeline.assemble.return_value = pr
tier_service = MagicMock()
# Return a minimal passing fragment so assemble() does not short-circuit
frag = _make_tiered_fragment(
fragment_id="frag-policy",
content="content",
token_count=5,
metadata={"path": "src/good.py"},
)
tier_service.get_scoped_view.return_value = [frag]
# Build a mock row with context_policy_json
mock_row = MagicMock()
mock_row.context_policy_json = policy_json
# Mock the session so _resolve_hot_max_tokens can query it
mock_session = MagicMock()
mock_session.query.return_value.filter_by.return_value.first.return_value = mock_row
repo = MagicMock()
repo._session.return_value = mock_session
repo.get_context_policy.return_value = ProjectContextPolicy()
return ACMSExecutePhaseContextAssembler(
context_tier_service=tier_service,
project_repository=repo,
acms_pipeline=mock_pipeline,
hot_max_tokens=4096,
)
@given("epcov an assembler with context_policy_json hot_max_tokens 32000")
def step_epcov_assembler_policy_json_32k(context: Context) -> None:
"""Assembler backed by a DB row with hot_max_tokens=32000 in policy JSON.
Mirrors the real storage format written by
``agents project context set --hot-max-tokens 32000``:
``{"acms_config": {"hot_max_tokens": 32000, ...}, ...}``.
"""
policy_json = json.dumps({"acms_config": {"hot_max_tokens": 32000}})
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
@given("epcov an assembler with no hot_max_tokens in context_policy_json")
def step_epcov_assembler_policy_json_no_override(context: Context) -> None:
"""Assembler backed by a DB row with no hot_max_tokens — global fallback."""
policy_json = json.dumps({"acms_config": {"other_setting": "value"}})
context.epcov_assembler = _make_assembler_with_policy_json(policy_json)
@then("epcov the pipeline received budget max_tokens of 32000")
def step_epcov_pipeline_budget_32k(context: Context) -> None:
"""Verify the pipeline was called with max_tokens=32000 in the budget."""
if context.epcov_error is not None:
raise AssertionError(f"Unexpected error: {context.epcov_error}")
pipeline = context.epcov_assembler._pipeline
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
call_args = pipeline.assemble.call_args
budget = call_args.kwargs.get("budget") or (
call_args[1].get("budget") if call_args[1] else None
)
if budget is None and call_args[0]:
budget = call_args[0][2] if len(call_args[0]) > 2 else None
assert budget is not None, "Could not find budget in pipeline.assemble() call"
assert budget.max_tokens == 32000, (
f"Expected budget.max_tokens=32000 (from context_policy_json), "
f"got {budget.max_tokens}"
)
@then("epcov the pipeline received budget max_tokens of 4096")
def step_epcov_pipeline_budget_global(context: Context) -> None:
"""Verify the pipeline fell back to the global hot_max_tokens=4096."""
if context.epcov_error is not None:
raise AssertionError(f"Unexpected error: {context.epcov_error}")
pipeline = context.epcov_assembler._pipeline
assert pipeline.assemble.called, "Pipeline.assemble() was not called"
call_args = pipeline.assemble.call_args
budget = call_args.kwargs.get("budget") or (
call_args[1].get("budget") if call_args[1] else None
)
if budget is None and call_args[0]:
budget = call_args[0][2] if len(call_args[0]) > 2 else None
assert budget is not None, "Could not find budget in pipeline.assemble() call"
assert budget.max_tokens == 4096, (
f"Expected budget.max_tokens=4096 (global fallback), got {budget.max_tokens}"
)
@@ -83,7 +83,7 @@ def _eed_make_lifecycle(plan: MagicMock) -> MagicMock:
lcs.start_execute = MagicMock()
lcs.complete_execute = MagicMock()
lcs.fail_execute = MagicMock()
lcs.commit_plan = MagicMock()
lcs._commit_plan = MagicMock()
return lcs
@@ -152,7 +152,7 @@ def step_eed_run_execute(context: Context) -> None:
"""Run execute and capture the result."""
context.eed_result = context.eed_executor.run_execute(context.eed_plan_id)
# Capture the error_details that were committed
commit_calls = context.eed_lifecycle.commit_plan.call_args_list
commit_calls = context.eed_lifecycle._commit_plan.call_args_list
if commit_calls:
last_plan = commit_calls[-1][0][0]
context.eed_committed_error_details = last_plan.error_details
@@ -169,7 +169,7 @@ def step_eed_run_execute_failure(context: Context) -> None:
except Exception:
context.eed_raised = True
# Capture the error_details that were committed
commit_calls = context.eed_lifecycle.commit_plan.call_args_list
commit_calls = context.eed_lifecycle._commit_plan.call_args_list
if commit_calls:
last_plan = commit_calls[-1][0][0]
context.eed_committed_error_details = last_plan.error_details
@@ -112,21 +112,21 @@ def _create_plan_at_phase(
elif phase == PlanPhase.STRATEGIZE and state == ProcessingState.COMPLETE:
plan.processing_state = ProcessingState.PROCESSING
plan.processing_state = ProcessingState.COMPLETE
svc.commit_plan(plan)
svc._commit_plan(plan)
elif phase == PlanPhase.STRATEGIZE and state == ProcessingState.PROCESSING:
plan.processing_state = ProcessingState.PROCESSING
svc.commit_plan(plan)
svc._commit_plan(plan)
elif phase == PlanPhase.EXECUTE and state == ProcessingState.COMPLETE:
plan.processing_state = ProcessingState.COMPLETE
svc.commit_plan(plan)
svc._commit_plan(plan)
# Transition to Execute
plan.processing_state = ProcessingState.QUEUED
plan.phase = PlanPhase.EXECUTE
svc.commit_plan(plan)
svc._commit_plan(plan)
# Advance to complete
plan.processing_state = ProcessingState.PROCESSING
plan.processing_state = ProcessingState.COMPLETE
svc.commit_plan(plan)
svc._commit_plan(plan)
context.plan = svc.get_plan(plan_id)
context.plan_id = plan_id
-14
View File
@@ -1085,17 +1085,3 @@ def step_check_all_plans_migrated(context: Context) -> None:
assert plan_names == expected_names, (
f"Expected plans {expected_names}, got {plan_names}"
)
@then("the existing plan id should be mapped without type suppression")
def step_check_existing_plan_id_mapped(context: Context) -> None:
"""Check that existing plan id was mapped correctly via assert type narrowing."""
with context.unit_of_work.transaction() as ctx:
project = ctx.projects.get_by_name("existing_plans")
assert project is not None, "Project 'existing_plans' should exist in database"
plans = ctx.plans.get_all_for_project(project.id)
main_plans = [p for p in plans if p.name == "main"]
assert len(main_plans) == 1, "Should have exactly one 'main' plan"
assert main_plans[0].id is not None, (
"Existing plan id must be non-None (assert narrowing validates this)"
)
+12 -52
View File
@@ -92,8 +92,6 @@ def _make_mock_lifecycle(strategy_actor=None, execution_actor=None):
lifecycle = SimpleNamespace(
get_plan=MagicMock(return_value=plan),
get_action=MagicMock(return_value=action),
resolve_actor_provider_model=MagicMock(return_value=None),
resolve_actor_options=MagicMock(return_value=None),
)
return lifecycle
@@ -132,8 +130,8 @@ class _StubContextAssembler:
LLM_NUMBERED_RESPONSE = "1. Create the module\n2. Write unit tests\n3. Update docs"
LLM_FILE_BLOCKS_RESPONSE = (
"FILE: src/main.py\n<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\nprint('hello')\n<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
"FILE: tests/test_main.py\n<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\ndef test_main(): pass\n<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n"
"FILE: src/main.py\n```python\nprint('hello')\n```\n\n"
"FILE: tests/test_main.py\n```python\ndef test_main(): pass\n```\n"
)
@@ -683,41 +681,18 @@ def step_execute_prompt_not_contains(context, needle):
@when("I parse file blocks from LLM output with two files")
def step_parse_file_blocks_two_files(context):
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
context.parsed_entries = LLMExecuteActor._parse_file_blocks(
LLM_FILE_BLOCKS_RESPONSE, "PLAN_TEST"
)
@when("I parse file blocks from empty LLM output")
def step_parse_file_blocks_empty(context):
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
context.parsed_entries = LLMExecuteActor._parse_file_blocks(
"No files here.", "PLAN_EMPTY"
)
@when("I parse file blocks using the CAFS short delimiter format")
def step_parse_file_blocks_cafs(context):
"""M7: exercise the <CAFS>/<CAFE> short-form delimiter pattern."""
llm_output = "FILE: src/cafs_example.py\n<CAFS>\nprint('cafs format')\n</CAFE>\n"
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
llm_output, "PLAN_CAFS"
)
@when("I parse file blocks using the new arrow delimiter format")
def step_parse_file_blocks_arrow(context):
"""M7: exercise the >>>>>>>> CLEVERAGENTS_FILE_START >>>>>>>> format."""
llm_output = (
"FILE: src/arrow_example.py\n"
">>>>>>>> CLEVERAGENTS_FILE_START >>>>>>>>\n"
"print('arrow format')\n"
">>>>>>>> CLEVERAGENTS_FILE_END >>>>>>>>\n"
)
context.parsed_entries, _ = LLMExecuteActor._parse_file_blocks(
llm_output, "PLAN_ARROW"
)
@then("I should get {count:d} changeset entries")
def step_verify_changeset_entry_count(context, count):
assert len(context.parsed_entries) == count, (
@@ -732,16 +707,6 @@ def step_verify_changeset_entry_path(context, idx, expected):
)
@then('I should get 1 changeset entry with path "{expected}"')
def step_verify_single_changeset_entry_path(context, expected):
assert len(context.parsed_entries) == 1, (
f"Expected exactly 1 entry, got {len(context.parsed_entries)}"
)
assert context.parsed_entries[0].path == expected, (
f"Expected path '{expected}', got '{context.parsed_entries[0].path}'"
)
# ---------------------------------------------------------------------------
# LLMExecuteActor._write_to_sandbox
# ---------------------------------------------------------------------------
@@ -760,8 +725,10 @@ def step_create_temp_sandbox(context):
@when("I write generated files to the sandbox")
def step_write_files_to_sandbox(context):
_, blocks = LLMExecuteActor._parse_file_blocks(LLM_FILE_BLOCKS_RESPONSE, "PLAN_WR")
LLMExecuteActor._write_to_sandbox(blocks, context.sandbox_dir)
entries = LLMExecuteActor._parse_file_blocks(LLM_FILE_BLOCKS_RESPONSE, "PLAN_WR")
LLMExecuteActor._write_to_sandbox(
entries, context.sandbox_dir, LLM_FILE_BLOCKS_RESPONSE
)
@then("the sandbox should contain the generated files")
@@ -795,20 +762,13 @@ def step_create_readonly_sandbox(context):
@when("I write generated files to the read-only sandbox")
def step_write_to_readonly_sandbox(context):
# FILE path puts the file inside "src/" which is read-only.
# Use the legacy CLEVERAGENTS sentinel delimiter so _parse_file_blocks
# actually produces file blocks (backtick format was removed in #10878 fix).
llm_output = (
"FILE: src/fail.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"print('fail')\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n"
)
_, blocks = LLMExecuteActor._parse_file_blocks(llm_output, "PLAN_RO")
# FILE path puts the file inside "src/" which is read-only
llm_output = "FILE: src/fail.py\n```python\nprint('fail')\n```\n"
entries = LLMExecuteActor._parse_file_blocks(llm_output, "PLAN_RO")
# Should not raise - OSError is caught and logged inside _write_to_sandbox
context.write_exception = None
try:
LLMExecuteActor._write_to_sandbox(blocks, context.sandbox_dir)
LLMExecuteActor._write_to_sandbox(entries, context.sandbox_dir, llm_output)
except Exception as exc:
context.write_exception = exc
@@ -1,371 +0,0 @@
"""Step definitions for llm_delimiter_regression.feature (#10878).
Re-proves that the CLEVERAGENTS_FILE_START/END sentinel scheme fixes
the triple-backtick delimiter collision bug from #10878.
Two in-line "old" parsers are provided to demonstrate the original bug:
* _old_backtick_nongreedy -- non-greedy ```.``` pattern; truncates content
at the first triple-backtick *inside* a file body but still finds all
FILE blocks (each with truncated content).
* _old_backtick_greedy -- greedy ```.``` pattern; swallows subsequent
FILE blocks inside the first match's "content", returning far fewer
entries than expected.
The current ``LLMExecuteActor._parse_file_blocks`` uses
``<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>`` / ``<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>``
(and the newer ``<CAFS>``/``</CAFE>``) markers, which never collide with
triple-backtick Markdown fences.
Forgejo: #10878
"""
from __future__ import annotations
import re
import subprocess
from behave import given, step, then, when # type: ignore[import-untyped]
from cleveragents.application.services.llm_actors import LLMExecuteActor
# ---------------------------------------------------------------------------
# Old (buggy) backtick-based parsers used to reproduce the pre-fix behaviour
# ---------------------------------------------------------------------------
_LEGACY_START = "<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>"
_LEGACY_END = "<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>"
def _old_backtick_nongreedy(llm_output: str) -> list[str]:
"""Non-greedy old parser.
Finds the correct number of FILE blocks but truncates their content at
the first triple-backtick that appears inside a file body.
"""
pattern = re.compile(
r"FILE:\s*(.+?)\s*\n```\n(.*?)\n```",
re.DOTALL,
)
return [m.group(1).strip() for m in pattern.finditer(llm_output)]
def _old_backtick_greedy(llm_output: str) -> list[str]:
"""Greedy old parser.
The greedy ``(.*)`` causes the *first* match to consume all remaining
content up to the very last triple-backtick line in the document,
swallowing every subsequent FILE block inside the captured "content".
Only one (or very few) entries are returned for an input that contains
many FILE blocks.
"""
pattern = re.compile(
r"FILE:\s*(.+?)\s*\n```\n(.*)\n```",
re.DOTALL,
)
return [m.group(1).strip() for m in pattern.finditer(llm_output)]
# ---------------------------------------------------------------------------
# Helper builders
# ---------------------------------------------------------------------------
def _make_legacy_block(path: str, content: str) -> str:
"""FILE block using the CLEVERAGENTS_FILE_START/END markers."""
return f"FILE: {path}\n{_LEGACY_START}\n{content}\n{_LEGACY_END}\n\n"
def _make_old_backtick_block(path: str, content: str) -> str:
"""FILE block using the old triple-backtick delimiters."""
return f"FILE: {path}\n```\n{content}\n```\n\n"
def _fence_content(label: str) -> str:
"""File body that embeds a triple-backtick Python code fence."""
return (
f"# {label}\n"
"```python\n"
f"def {label.replace(' ', '_')}(): pass\n"
"```\n"
f"# end of {label}\n"
)
# ---------------------------------------------------------------------------
# Given — context setup
# ---------------------------------------------------------------------------
@given("a string containing ````")
def step_context_with_backtick_string(context): # type: ignore[no-untyped-def]
context.backtick_context = "````"
@given("a string ````")
def step_short_backtick_ctx(context): # type: ignore[no-untyped-def]
context.backtick_context = "````"
@given('a mock plan_id "01HQXXXXX"')
def step_plan_id_xxxxx(context): # type: ignore[no-untyped-def]
context.delimiter_plan_id = "01HQXXXXX"
@given('a mock plan_id "01HYYYYYY"')
def step_plan_id_yyyyyy(context): # type: ignore[no-untyped-def]
context.delimiter_plan_id = "01HYYYYYY"
# --- Input-creation steps ---
@step("I create file with LLM output containing code fences in body")
def step_create_old_2files_with_fences(context): # type: ignore[no-untyped-def]
"""Two FILE blocks in old backtick format, each with an embedded code fence.
The non-greedy old parser will find both files but truncate their content
at the first triple-backtick in each body (the opening fence).
"""
context.delim_llm_output = _make_old_backtick_block(
"src/main.py", _fence_content("main")
) + _make_old_backtick_block("src/util.py", _fence_content("util"))
@step(
"I create file with LLM output using unique delimiters and code fences embedded in body"
)
def step_create_new_6files_with_fences(context): # type: ignore[no-untyped-def]
"""Six FILE blocks using CLEVERAGENTS markers with embedded code fences."""
context.delim_llm_output = "".join(
_make_legacy_block(f"src/mod_{i}.py", _fence_content(f"mod_{i}"))
for i in range(1, 7)
)
@step("I create LLM output where file content contains embedded markdown code blocks")
def step_create_old_4files_greedy(context): # type: ignore[no-untyped-def]
"""Four FILE blocks in old backtick format, each with embedded code fences.
The *greedy* old parser will match only the first FILE block and
consume the remaining three as part of the first block's "content".
"""
context.delim_llm_output = (
_make_old_backtick_block("src/a.py", _fence_content("a"))
+ _make_old_backtick_block("src/b.py", _fence_content("b"))
+ _make_old_backtick_block("src/c.py", _fence_content("c"))
+ _make_old_backtick_block("src/d.py", _fence_content("d"))
)
@step(
"I create LLM output using unique sentinels with embedded ```python code blocks in body"
)
def step_create_new_4files_with_fences(context): # type: ignore[no-untyped-def]
"""Four FILE blocks using CLEVERAGENTS markers with embedded code fences."""
context.delim_llm_output = (
_make_legacy_block("src/a.py", _fence_content("a"))
+ _make_legacy_block("src/b.py", _fence_content("b"))
+ _make_legacy_block("src/c.py", _fence_content("c"))
+ _make_legacy_block("src/d.py", _fence_content("d"))
)
@step(
"I create full regression file block output that includes report section plus"
" multiple files each with inline ```python examples"
)
def step_create_full_regression_output(context): # type: ignore[no-untyped-def]
"""Architecture-report header followed by five FILE blocks with CLEVERAGENTS markers."""
prose = "## Architecture Report\n\nThis section describes key modules.\n\n"
files = [
_make_legacy_block(f"src/mod_{i}.py", _fence_content(f"mod_{i}"))
for i in range(1, 6)
]
context.delim_llm_output = prose + "".join(files)
context.delim_expected_count = 5
@step(
"LLM output containing two FILE blocks each with embedded"
" ```python sections in the content"
)
def step_create_new_2files_with_python_fences(context): # type: ignore[no-untyped-def]
"""Two FILE blocks with CLEVERAGENTS markers and embedded Python code fences."""
context.delim_llm_output = _make_legacy_block(
"mod_a.py", _fence_content("module_a")
) + _make_legacy_block("src/mod_b.py", _fence_content("module_b"))
@step(
"an llm_output where the file body ends on a line that is exactly ``` after content"
)
def step_create_trailing_backtick_body(context): # type: ignore[no-untyped-def]
"""Single FILE block whose last body line is exactly three backticks."""
context.delim_llm_output = (
f"FILE: module.py\n{_LEGACY_START}\nline one\nline two\n```\n{_LEGACY_END}\n\n"
)
@step(
"LLM input with backtick-delimited FILE blocks containing code without"
" triple-backticks inside body"
)
def step_create_single_simple_block(context): # type: ignore[no-untyped-def]
"""Single FILE block with CLEVERAGENTS markers and plain content (no code fences).
This tests backward-compatible parsing of simple single-file blocks.
"""
context.delim_llm_output = _make_legacy_block(
"src/simple.py",
"# Simple module\ndef run(): return 0\n",
)
# ---------------------------------------------------------------------------
# When — command-like no-ops (side-effect-free shell invocations)
# ---------------------------------------------------------------------------
@when("I run the command \"echo 'backtick-test'\"")
def step_run_echo_backtick_test(context): # type: ignore[no-untyped-def]
subprocess.run(["echo", "backtick-test"], capture_output=True, check=False)
@when("I run the command \"echo 'old-parser-test'\"")
def step_run_echo_old_parser_test(context): # type: ignore[no-untyped-def]
subprocess.run(["echo", "old-parser-test"], capture_output=True, check=False)
@when("I run the command \"echo 'new-parser-test'\"")
def step_run_echo_new_parser_test(context): # type: ignore[no-untyped-def]
subprocess.run(["echo", "new-parser-test"], capture_output=True, check=False)
@when("I run the command \"echo 'full-regression-test'\"")
def step_run_echo_full_regression(context): # type: ignore[no-untyped-def]
subprocess.run(["echo", "full-regression-test"], capture_output=True, check=False)
@when("I run the command \"echo 'backtick-sanitise'\"")
def step_run_echo_backtick_sanitise(context): # type: ignore[no-untyped-def]
subprocess.run(["echo", "backtick-sanitise"], capture_output=True, check=False)
# --- Parsing steps ---
@when("I parse the LLM output as if it were old backtick-delimited file blocks")
def step_parse_old_nongreedy(context): # type: ignore[no-untyped-def]
context.delim_entries = _old_backtick_nongreedy(context.delim_llm_output)
@when(
"I parse the LLM output with new"
" CLEVERAGENTS_FILE_START/CLEVERAGENTS_FILE_END delimiters"
)
def step_parse_new_cleveragents(context): # type: ignore[no-untyped-def]
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
context.delim_llm_output, plan_id
)
@when("I parse it with the old backtick-style delimiter pattern")
def step_parse_old_greedy(context): # type: ignore[no-untyped-def]
context.delim_entries = _old_backtick_greedy(context.delim_llm_output)
@when("I parse it with CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters")
def step_parse_new_4files(context): # type: ignore[no-untyped-def]
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
context.delim_llm_output, plan_id
)
@when("I parse it with new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiters")
def step_parse_new_full_regression(context): # type: ignore[no-untyped-def]
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
context.delim_llm_output, plan_id
)
@when("I call _parse_file_blocks on that LLM output")
def step_call_parse_file_blocks_llm(context): # type: ignore[no-untyped-def]
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
context.delim_llm_output, plan_id
)
@when("I call _parse_file_blocks on that output")
def step_call_parse_file_blocks(context): # type: ignore[no-untyped-def]
plan_id = getattr(context, "delimiter_plan_id", "DELIM_PLAN")
context.delim_entries, _ = LLMExecuteActor._parse_file_blocks(
context.delim_llm_output, plan_id
)
# ---------------------------------------------------------------------------
# Then — assertions
# ---------------------------------------------------------------------------
@then("the parser should return only 2 entries truncated at first triple-backtick")
def step_assert_2_truncated_entries(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 2, (
f"Expected 2 entries (truncated) from old backtick parser, got {len(entries)}: "
f"{entries}"
)
@then("the parser should return 6 entries not truncated at any triple-backtick")
def step_assert_6_full_entries(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 6, (
f"Expected 6 complete entries from new CLEVERAGENTS parser, got {len(entries)}"
)
@then("the parser should return only 1 entry instead of the expected 4")
def step_assert_1_instead_of_4(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 1, (
f"Expected greedy old parser to return 1 (swallowing other files), "
f"got {len(entries)}: {entries}"
)
@then("the parser should return a result count of 4 entries")
def step_assert_4_entries(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 4, (
f"Expected 4 entries from new CLEVERAGENTS parser, got {len(entries)}"
)
@then("the parser should return the full entry count of all embedded blocks")
def step_assert_full_entry_count(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
expected = getattr(context, "delim_expected_count", 5)
assert len(entries) == expected, (
f"Expected {expected} entries (full regression), got {len(entries)}"
)
@then("it should return 2 changeset entries not 1")
def step_assert_2_not_1(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 2, (
f"Expected 2 entries (new parser handles embedded fences, not 1): "
f"got {len(entries)}"
)
@then("it should return 1 entry")
def step_assert_1_entry(context): # type: ignore[no-untyped-def]
entries = context.delim_entries
assert len(entries) == 1, f"Expected exactly 1 entry, got {len(entries)}"
@@ -1,236 +0,0 @@
"""Step definitions for llm_file_parsing_regression.feature (#10878 TDD regression).
Demonstrates that LLMExecuteActor._parse_file_blocks correctly extracts all FILE
blocks when their body contains embedded triple-backtick markdown code fences,
proving the new delimiter scheme is not vulnerable to the collision that
caused truncated architecture reviews.
Forgejo: #10878
Also exercises related edge cases: trailing backtick line, sentinel text in
body prose without full markers, and git merge-conflict style markers mixed
with sentinel delimiters.
"""
from __future__ import annotations
from behave import given, then, when # type: ignore[import-untyped]
from cleveragents.application.services.llm_actors import LLMExecuteActor
# ---------------------------------------------------------------------------
# Helpers - build LLM output strings matching each scenario's description.
# ---------------------------------------------------------------------------
def _llm_with_fences(n_files, fence_in_each=True, extra_prose="") -> str:
"""Return an ``FILE: ... / delimiter`` block string.
When *fence_in_each* is True the body of every file includes a
`````python ```` `code fence` - exactly the pattern that caused the original bug.
"""
output = extra_prose
for i in range(1, n_files + 1):
path = f"mod_{i}.py" if i == 1 else f"src/mod_{i}.py"
fence_block = "\n```python\nprint('hello')\n```\n" if fence_in_each else ""
output += (
f"FILE: {path}\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
f"# File {i}\n{fence_block}"
"actual code here\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
)
return output
# ---------------------------------------------------------------------------
# Given steps
# ---------------------------------------------------------------------------
@given(
"an LLM response with two FILE blocks, each having ```python sections in the body"
)
def step_two_embedded_fences(context): # type: ignore[no-untyped-def]
context.input_lfm = _llm_with_fences(2, fence_in_each=True)
@given(
"an LLM response mimicking an architecture review with report prose and three FILE blocks each containing markdown code example triple-backticks in their body",
)
def step_arch_review_full(context): # type: ignore[no-untyped-def]
context.input_lfm = _llm_with_fences(
3, fence_in_each=True, extra_prose="## Module Graph\nSome report prose above.\n"
)
@given(
"an LLM response where a single FILE block body ends on a line that is exactly ``` after code content"
)
def step_trailing_backtick(context): # type: ignore[no-untyped-def]
path = "module.py"
context.input_lfm = (
f"FILE: {path}\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"line one\nline two\n```\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
)
@given(
"an LLM response where a FILE block body contains the word 'CLEVERAGENTS_FILE_END' referenced in prose within content and another file follows",
)
def step_body_mentions_sentinel_text(context): # type: ignore[no-untyped-def]
context.input_lfm = (
"FILE: mod_a.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"some text with ```python blocks\n"
"# The body mentions CLEVERAGENTS_FILE_END as prose (no markers)\n"
"rest of content here\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
"FILE: mod_b.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"# second file block\n"
"continues after sentinel-text mention\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
)
@given(
"an LLM response where FILE block bodies include text like '>>>>>>> some_branch' and '<<<<<<< base' but the actual delimiters are CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END markers",
)
def step_mixed_git_markers(context): # type: ignore[no-untyped-def]
context.input_lfm = (
"Prose with <<<<<< some_branch mentioned here\n"
"FILE: mod_a.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"```python\ndef f1(): pass # <<< and >>> in content are fine\n```\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
"FILE: mod_b.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
">>>>>>> base branch also mentioned here\n"
"more code below\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
)
@given('a mock plan_id "01HQREGTEST"')
def step_mock_plan_id_regtest(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQREGTEST"
@given('a mock plan_id "01HQARCHTEST"')
def step_mock_plan_id_archtest(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQARCHTEST"
@given('a mock plan_id "01HQTRAILT"')
def step_mock_plan_id_trail(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQTRAILT"
@given('a mock plan_id "01HQSENTINEL"')
def step_mock_plan_id_sentinel(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQSENTINEL"
@given('a mock plan_id "01HQGITTEST"')
def step_mock_plan_id_gittest(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQGITTEST"
@given('a mock plan_id "01HQESCTEST"')
def step_mock_plan_id_esctest(context): # type: ignore[no-untyped-def]
context.lfm_plan_id = "01HQESCTEST"
@given(
"an LLM response with a single FILE block using legacy markers where the body"
" contains a backslash-escaped <<<<<<< CLEVERAGENTS_FILE_START >>>>>>>"
" sequence that should be preserved as literal text"
)
def step_single_block_with_escaped_marker(context): # type: ignore[no-untyped-def]
"""FILE block whose body mentions the START marker prefixed with a backslash.
The backslash-escaped occurrence must NOT be treated as a block boundary
by the negative-lookbehind regex in _parse_file_blocks.
"""
context.input_lfm = (
"FILE: example.py\n"
"<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"# This file discusses the delimiter format:\n"
"# \\<<<<<<< CLEVERAGENTS_FILE_START >>>>>>>\n"
"# The above line is escaped and must not break parsing.\n"
"<<<<<<< CLEVERAGENTS_FILE_END >>>>>>>\n\n"
)
# ---------------------------------------------------------------------------
# When steps
# ---------------------------------------------------------------------------
@when(
"I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern"
)
def step_parse_with_new_delimiters(context): # type: ignore[no-untyped-def]
"""Exercise LLMExecuteActor._parse_file_blocks with real input."""
plan_id = getattr(context, "lfm_plan_id", "TEST_PLAN") # type: ignore[attr-defined]
input_data = context.input_lfm # type: ignore[attr-defined]
context.lfm_entries, _ = LLMExecuteActor._parse_file_blocks(input_data, plan_id)
@when("I parse file blocks using the new delimiter pattern")
def step_parse_with_new_delimiters_short(context): # type: ignore[no-untyped-def]
"""Alias of the CLEVERAGENTS step for shorter scenario descriptions."""
plan_id = getattr(context, "lfm_plan_id", "TEST_PLAN") # type: ignore[attr-defined]
input_data = context.input_lfm # type: ignore[attr-defined]
context.lfm_entries, _ = LLMExecuteActor._parse_file_blocks(input_data, plan_id)
# ---------------------------------------------------------------------------
# Then steps
# ---------------------------------------------------------------------------
@then("it should return {count:d} changeset entries")
def step_validate_entry_count(context, count): # type: ignore[no-untyped-def]
assert len(context.lfm_entries) == count, ( # type: ignore[attr-defined]
f"Expected {count} entry/entries, got {len(context.lfm_entries)}: "
f"{[e.path for e in context.lfm_entries]}"
)
@then("it should return the full entry count of all embedded blocks")
def step_validate_full_count_in_body(context): # type: ignore[no-untyped-def]
entries = context.lfm_entries # type: ignore[attr-defined]
assert len(entries) == 3, (
f"Expected 3 entries for full arch-review output, got {len(entries)}: "
f"{[e.path for e in entries]}"
)
@then("it should return {count:d} changeset entries (second entry has content in body)")
def step_validate_entry_count_with_body_note(context, count): # type: ignore[no-untyped-def]
assert len(context.lfm_entries) == count, ( # type: ignore[attr-defined]
f"Expected {count} entries, got {len(context.lfm_entries)}: "
f"{[e.path for e in context.lfm_entries]}"
)
@then("changeset entry {idx:d} path should be a file with ```python block inside")
def step_validate_path_has_python_fences(index, context): # type: ignore[no-untyped-def]
entries = context.lfm_entries # type: ignore[attr-defined]
assert len(entries) > index, (
f"Expected at least {index + 1} entries, got {len(entries)}"
)
@then("it should NOT trip on the closing triple backtick")
def step_validate_no_triple_backtick_stop(context): # type: ignore[no-untyped-def]
entries = context.lfm_entries # type: ignore[attr-defined]
assert len(entries) >= 1, (
f"Expected at least one entry when body contains ```; got {len(entries)}"
)
+41 -3
View File
@@ -4,8 +4,7 @@ Exercises every uncovered branch in ``cleveragents.lsp.transport``:
start() error paths, stop() variations, send_message(), read_message()
header/body parsing, and all error handling paths.
Uses the ``ltcov`` prefix on all steps to avoid AmbiguousStep errors. Shared
helper utilities are defined in :mod:`_ltcov_helpers`.
Uses the ``ltcov`` prefix on all steps to avoid AmbiguousStep errors.
"""
from __future__ import annotations
@@ -20,7 +19,46 @@ from behave.runner import Context
from cleveragents.lsp.errors import LspError
from cleveragents.lsp.transport import _MAX_CONTENT_LENGTH, StdioTransport
from _ltcov_helpers import make_mock_process as _make_mock_process
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_mock_process(
*,
poll_return: int | None = None,
returncode: int = 0,
stdin: object | None = "auto",
stdout: object | None = "auto",
stderr: object | None = "auto",
pid: int = 12345,
) -> MagicMock:
"""Create a ``MagicMock`` resembling ``subprocess.Popen``."""
proc = MagicMock(spec=subprocess.Popen)
proc.pid = pid
proc.poll.return_value = poll_return
proc.returncode = returncode
if stdin == "auto":
proc.stdin = MagicMock()
else:
proc.stdin = stdin
if stdout == "auto":
proc.stdout = MagicMock()
else:
proc.stdout = stdout
if stderr == "auto":
proc.stderr = MagicMock()
else:
proc.stderr = stderr
return proc
def _build_lsp_frame(body_dict: dict) -> bytes:
"""Encode a dict as a Content-Length framed LSP message."""
payload = json.dumps(body_dict, separators=(",", ":")).encode("utf-8")
header = f"Content-Length: {len(payload)}\r\n\r\n".encode("ascii")
return header + payload
# ---------------------------------------------------------------------------
# Given steps
@@ -1,113 +0,0 @@
"""Step definitions for LSP StdioTransport post-spawn cleanup tests.
Covers issue #7044: subprocess cleanup when post-Popen initialization fails,
plus explicit is_alive() test coverage. Uses the ``ltcov`` prefix on all steps.
Shared helper utilities are imported from :mod:`_ltcov_helpers`.
"""
from __future__ import annotations
from unittest.mock import patch
from behave import given, then, when
from behave.runner import Context
from _ltcov_helpers import make_mock_process as _make_mock_process
# ---------------------------------------------------------------------------
# Given steps
# ---------------------------------------------------------------------------
@given("ltcov Popen is mocked to succeed with a running process")
def step_ltcov_popen_succeed(context: Context) -> None:
"""Mock subprocess.Popen to return a MagicMock (no exception), simulating
a successful spawn. The mock is stored on context for assertion.
"""
proc = _make_mock_process(poll_return=None, returncode=None)
patcher = patch(
"cleveragents.lsp.transport.subprocess.Popen",
return_value=proc,
)
patcher.start()
context.add_cleanup(patcher.stop)
context._ltcov_mock_popen_return_value = proc
@given("ltcov logger.info after spawn raises RuntimeError")
def step_ltcov_logger_info_raises(context: Context) -> None:
"""Mock ``logger.info`` to raise a RuntimeError, simulating that post-Popen
initialization (e.g. logging the started message) fails.
"""
def _info_raises(*args, **kwargs):
if args and args[0] == "lsp.transport.started":
raise RuntimeError("simulated post-Popen initialization failure")
patcher = patch(
"cleveragents.lsp.transport.logger.info",
side_effect=_info_raises,
)
patcher.start()
context.add_cleanup(patcher.stop)
# ---------------------------------------------------------------------------
# When steps
# ---------------------------------------------------------------------------
@when("ltcov I check if the transport is alive")
def step_ltcov_check_alive(context: Context) -> None:
"""Call is_alive() and store the boolean result."""
context.ltcov_is_alive_result = context.ltcov_transport.is_alive
# ---------------------------------------------------------------------------
# Then steps
# ---------------------------------------------------------------------------
@then("ltcov an error occurred during start()")
def step_ltcov_error_occurred(context: Context) -> None:
"""Assert that start() raised some exception (wrapped or unwrapped)."""
assert context.ltcov_error is not None, "Expected an error but none was raised"
@then("ltcov the mock process should have been terminated")
def step_ltcov_process_terminated(context: Context) -> None:
"""Assert that terminate() was called on the mock subprocess."""
proc = context._ltcov_mock_popen_return_value
proc.terminate.assert_called_once(), "Expected the mock process to be terminated"
@then("ltcov the mock process should have been waited on")
def step_ltcov_process_waited(context: Context) -> None:
"""Assert that wait() was called on the mock subprocess."""
proc = context._ltcov_mock_popen_return_value
proc.wait.assert_called_once(), "Expected the mock process to be waited on"
@then("ltcov the transport should be alive")
def step_ltcov_transport_alive(context: Context) -> None:
"""Assert that is_alive() returns True."""
assert context.ltcov_transport.is_alive, (
"Expected transport to be alive but it reported not alive"
)
@then("ltcov the internal process should not be None")
def step_ltcov_process_not_none(context: Context) -> None:
"""Assert that _process is set to a non-None value."""
assert context.ltcov_transport._process is not None, (
"Expected _process to be set but it is None"
)
@then("ltcov the is_alive result should be false")
def step_ltcov_alive_result_false(context: Context) -> None:
"""Assert that is_alive() returns False."""
assert context.ltcov_is_alive_result is False, (
f"Expected False, got {context.ltcov_is_alive_result}"
)
-117
View File
@@ -1,117 +0,0 @@
"""Step definitions for main() error path coverage tests."""
from __future__ import annotations
import ast
import subprocess
import sys
from io import StringIO
from unittest.mock import patch
from behave import then, when
from behave.runner import Context
PYTHON = sys.executable
MAIN_MODULE = "cleveragents.cli.main"
@when("I run CLI with arguments {args}")
def step_run_cli(context: Context, args: str) -> None:
"""Run ``python -m cleveragents.cli.main <args>`` via subprocess.
The *args* argument is a Python-like list literal from the Gherkin
feature line, e.g. ``["--config-path", "/tmp"]`` or ``["bad_cmd"]``.
We evaluate it with ``ast.literal_eval`` to get a real ``list[str]``,
then invoke the module via subprocess.run().
Pattern follows cli_coverage_steps.py:subprocess.run(shlex.split(cmd))
"""
cmd_args = ast.literal_eval(args) if isinstance(args, str) else []
try:
result = subprocess.run(
[PYTHON, "-m", MAIN_MODULE, *cmd_args],
capture_output=True,
text=True,
timeout=30,
)
context.exit_code = result.returncode
context.output = result.stdout + result.stderr
except subprocess.TimeoutExpired:
context.exit_code = -1
context.output = "Command timed out"
except Exception as exc:
context.exit_code = -2
context.output = f"Subprocess error: {exc}"
# ---------------------------------------------------------------------------
# convert_exit_code helpers — unique step text to avoid collisions
# ---------------------------------------------------------------------------
@when('the main exit-code converter is called with the value "{value}"')
def step_convert_exit_value(context: Context, value: str) -> None:
"""Call convert_exit_code(value — int, string, or special token).
Special tokens recognised in-process:
* "None" Python None
* "abc" literal string 'abc' (triggers TypeError path)
Everything else is cast to int first (to cover the int/str paths).
"""
from cleveragents.cli.main import convert_exit_code
if value == "None":
parsed = None
elif value in ("abc", "def"):
parsed = value # raw string → triggers TypeError path inside int()
else:
try:
parsed = int(value)
except ValueError:
parsed = value # pass through (int(calling_code) will catch)
context.exit_code = convert_exit_code(parsed)
@when("I call the main _print_basic_help")
def step_print_basic_help(context: Context) -> None:
"""Capture output from calling _print_basic_help()."""
from cleveragents.cli.main import _print_basic_help
buf = StringIO()
with patch("typer.echo", side_effect=buf.write):
_print_basic_help()
context.help_output = buf.getvalue()
# ---------------------------------------------------------------------------
# Then steps — unique step texts to avoid collisions with other suites
# ---------------------------------------------------------------------------
@then("the main cli exit code should be {code:d}")
def step_main_exit_code(context: Context, code: int) -> None:
"""Verify the CLI subprocess exit code."""
assert context.exit_code == code, (
f"Expected exit code {code}, got {context.exit_code}"
)
@then('the main cli output contains "{text}"')
def step_main_output_contains(context: Context, text: str) -> None:
"""Verify the CLI subprocess output."""
assert text in context.output, f"Expected '{text}' in output:\n{context.output}"
@then('the main convert_exit_code result is "{expect}"')
def step_main_convert_result(context: Context, expect: str) -> None:
"""Verify the convert_exit_code return value matches a string."""
actual = str(context.exit_code)
assert actual == expect, f"Expected '{expect}', got '{actual}'"
@then("the main _print_basic_help completes ok")
def step_main_print_ok(context: Context) -> None:
"""No-op success — the When-step finished without raising."""
pass
@@ -342,10 +342,7 @@ def step_validate_sql_constructor(context: Any) -> None:
assert context.sql_history_calls
_, kwargs = context.sql_history_calls[0]
assert kwargs["session_id"] == "sql-session"
actual_connection = kwargs.get("connection") or kwargs.get("connection_string")
assert actual_connection == context.connection_string, (
f"Expected {context.connection_string}, got {actual_connection}"
)
assert kwargs["connection_string"] == context.connection_string
assert kwargs.get("table_name") == "message_history"
+2 -3
View File
@@ -354,8 +354,7 @@ def step_call_apply_abort_timeout(context: object) -> None:
def step_create_failing_sandbox(context: object) -> None:
d = tempfile.mkdtemp(prefix="mca-flat-")
context.add_cleanup(shutil.rmtree, d, True)
plan_id = "01TESTFLATFAIL00000000000000"
sandbox = os.path.join(d, "plan-output", plan_id)
sandbox = os.path.join(d, ".cleveragents", "sandbox")
os.makedirs(sandbox)
Path(sandbox, "output.py").write_text("# generated\n")
# Create a read-only destination directory to cause copy failure
@@ -365,7 +364,7 @@ def step_create_failing_sandbox(context: object) -> None:
os.chmod(dst_dir, 0o444)
context.add_cleanup(os.chmod, dst_dir, 0o755)
context.mca_flat_project = d
context.mca_plan_id = plan_id
context.mca_plan_id = "01TESTFLATFAIL00000000000000"
@when("I call _apply_sandbox_changes with the failing flat copy for mca")
@@ -100,7 +100,7 @@ def step_plan_strategize_empty_definition(context: Context) -> None:
plan_id = _create_plan_in_strategize(context, "placeholder")
plan = context.lifecycle_service.get_plan(plan_id)
plan.definition_of_done = None
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
@given("I have a plan with invariants in strategize queued state")
@@ -159,7 +159,7 @@ def step_plan_execute_no_decisions(context: Context) -> None:
context.plan = context.lifecycle_service.get_plan(context.plan_id)
# Ensure no decision_root_id
context.plan.decision_root_id = None
context.lifecycle_service.commit_plan(context.plan)
context.lifecycle_service._commit_plan(context.plan)
@given("the strategize actor is configured to fail")
@@ -159,7 +159,7 @@ def _build_lifecycle_with_capturing_bus(
lifecycle = MagicMock(spec=PlanLifecycleService)
lifecycle.event_bus = event_bus
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
lifecycle._cleanup_devcontainers = MagicMock()
lifecycle._logger = structlog.get_logger("test.lifecycle")
@@ -337,7 +337,7 @@ def step_apply_with_validation_gate(context: Context) -> None:
lifecycle = MagicMock()
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
captured_kwargs: dict[str, Any] = {}
-495
View File
@@ -1,495 +0,0 @@
"""Step definitions for PlanApplyService rendering coverage tests."""
from __future__ import annotations
import datetime
from typing import TYPE_CHECKING, Any
from behave import given, then, when
from behave.runner import Context
if TYPE_CHECKING:
pass
# ---------------------------------------------------------------------------
# Helpers — build plain data dicts for the render functions
# ---------------------------------------------------------------------------
def _make_diff_data(
mode: str = "revert",
original_decision_id: str = "dec-0",
new_decision_id: str | None = "dec-1",
guidance: str = "",
completed_at: bool = False,
patched_hint: str | None = None,
artifacts_path: str | None = None,
) -> dict[str, Any]:
"""Build a minimal correction-diff data dict for render function tests."""
now_iso = datetime.datetime.now(datetime.UTC).isoformat()
summary: dict[str, Any] = {
"correction": "corr-test-001",
"original_decision": original_decision_id,
"mode": mode,
"state": "complete" if completed_at else "executing",
"guidance": guidance or "",
"created_at": now_iso,
"completed_at": now_iso if completed_at else None,
}
reverted_decisions: list[dict[str, Any]] = []
added_decisions: list[dict[str, Any]] = []
if original_decision_id:
reverted_decisions.append(
{"decision_id": original_decision_id, "status": "reverted"}
)
if new_decision_id:
added_decisions.append({"decision_id": new_decision_id, "status": "added"})
comparison = {
"reverted_decisions": reverted_decisions,
"added_decisions": added_decisions,
"decisions_changed": len(reverted_decisions) + len(added_decisions),
"decisions_added": len(added_decisions),
"decisions_reverted": len(reverted_decisions),
}
if patched_hint is None and artifacts_path is None:
patch_preview = [
{
"note": (
"Patch preview is available after correction execution completes. "
"Current state: executing"
),
}
]
elif artifacts_path:
patch_preview = [
{
"note": "Corrected execution artifacts available",
"artifacts_path": artifacts_path,
}
]
else:
patch_preview = [{"note": patched_hint}]
return {
"correction_diff": summary,
"comparison": comparison,
"patch_preview": patch_preview,
}
def _import_render_functions() -> tuple:
"""Lazy-import the render functions."""
from cleveragents.application.services import plan_apply_service as mod
return (
mod._render_correction_diff_plain,
mod._render_correction_diff_rich,
mod._build_correction_diff_dict,
)
# ---------------------------------------------------------------------------
# Given steps
# ---------------------------------------------------------------------------
@given(
'I have a CorrectionAttemptRecord with mode="{mode}", '
'original_decision_id="{orig}", new_decision_id="{new}", '
'guidance="{guidance}", and archived_artifacts_path="{artifacts}"'
)
def step_corr_record_with_artifact(
context: Context,
mode: str,
orig: str,
new: str,
guidance: str,
artifacts: str,
) -> None:
"""Construct a minimal CorrectionAttemptRecord for _build_correction_diff_dict."""
from cleveragents.domain.models.core.correction import (
CorrectionAttemptRecord,
CorrectionAttemptState,
CorrectionMode,
)
record = CorrectionAttemptRecord(
correction_attempt_id="test-corr-001",
plan_id="plan-001",
original_decision_id=orig,
new_decision_id=new if new != "none" else None,
mode=CorrectionMode.REVERT if mode == "revert" else CorrectionMode.APPEND,
guidance=guidance or "",
archived_artifacts_path=artifacts if artifacts not in ("", "none") else None,
state=CorrectionAttemptState.COMPLETE,
)
context.corr_record: CORRECTIONATTEMPTRECORD = record # noqa: F821
@given(
'I have a CorrectionAttemptRecord with mode="{mode}", '
'original_decision_id="{orig}", no new_decision, '
'guidance="{guidance}", and no archived artifacts'
)
def step_corr_record_no_artifact(
context: Context, mode: str, orig: str, guidance: str
) -> None:
"""Construct a CorrectionAttemptRecord without new_decision or archived artifacts."""
from cleveragents.domain.models.core.correction import (
CorrectionAttemptRecord,
CorrectionAttemptState,
CorrectionMode,
)
record = CorrectionAttemptRecord(
correction_attempt_id="test-corr-002",
plan_id="plan-002",
original_decision_id=orig,
new_decision_id=None,
mode=CorrectionMode.REVERT if mode == "revert" else CorrectionMode.APPEND,
guidance=guidance or "",
archived_artifacts_path=None,
state=CorrectionAttemptState.PENDING,
)
context.corr_record: CORRECTIONATTEMPTRECORD = record # noqa: F821
@given(
'I have a correction diff dict with mode="{mode}", '
'original_decision_id="{orig}", guidance="{guidance}", '
'completed_at present, and one added decision "{added}"'
)
def step_diff_dict_with_artifacts(
context: Context, mode: str, orig: str, guidance: str, added: str
) -> None:
"""Build a plain data dict for render function tests."""
# Use the added value as new_decision_id if provided
nid = added if added != "none" and added else None
data = _make_diff_data(
mode=mode,
original_decision_id=orig,
new_decision_id=nid,
guidance=guidance,
completed_at=True,
)
context.diff_data = data
@given(
'I have a correction diff dict with mode="{mode}", '
'original_decision_id="{orig}", no guidance text, '
'and patch preview note about "{hint}"'
)
def step_diff_dict_no_guidance(
context: Context, mode: str, orig: str, hint: str
) -> None:
"""Build a plain data dict with missing fields."""
data = _make_diff_data(
mode=mode,
original_decision_id=orig,
new_decision_id=None,
guidance="",
completed_at=False,
patched_hint=f"Patch preview is available after correction execution completes. "
f"Current state: {hint}",
)
context.diff_data = data
# ---------------------------------------------------------------------------
# When steps
# ---------------------------------------------------------------------------
@when("I call _build_correction_diff_dict on the record")
def step_build_corr_dict(context: Context) -> None:
"""Call the production _build_correction_diff_dict function & save dict output."""
_, _, build_func = _import_render_functions()
record = context.corr_record # type: ignore[attr-defined]
data = build_func(record)
context.diff_data = data
@when("I call _render_correction_diff_plain on the dict")
def step_render_plain(context: Context) -> None:
"""Call plain-text renderer and save output."""
render_plain, _, _ = _import_render_functions()
context.render_output = render_plain(context.diff_data) # type: ignore[attr-defined]
@when("I call _render_correction_diff_rich on the dict")
def step_render_rich(context: Context) -> None:
"""Call rich-text renderer and save output."""
_, render_rich, _ = _import_render_functions()
context.render_output = render_rich(context.diff_data) # type: ignore[attr-defined]
# ---------------------------------------------------------------------------
# Then steps — verify structure & content
# ---------------------------------------------------------------------------
@then('the result should contain "{key1}", "{key2}", and "{key3}" keys')
def step_result_contains_keys(
context: Context, key1: str, key2: str, key3: str
) -> None:
"""Verify the data dict has the expected top-level keys."""
for k in (key1, key2, key3):
assert k in context.diff_data, f"Missing key '{k}' in result"
@then(
"the comparison should have {n_reverted:d} reverted decision and "
"{n_added:d} added decisions"
)
def step_comparison_count(context: Context, n_reverted: int, n_added: int) -> None:
"""Verify the number of reverted/added decisions in comparison."""
cmp = context.diff_data["comparison"]
actual_reverted = len(cmp["reverted_decisions"])
actual_added = len(cmp["added_decisions"])
assert actual_reverted == n_reverted, (
f"Expected {n_reverted} reverted, got {actual_reverted}"
)
assert actual_added == n_added, f"Expected {n_added} added, got {actual_added}"
@then('the patch preview should include the artifacts path "{path}"')
def step_patch_has_artifacts(context: Context, path: str) -> None:
"""Verify patch-preview contains the specified artifacts path."""
patch = context.diff_data["patch_preview"][0]
assert patch.get("artifacts_path") == path, (
f"Expected artifacts_path='{path}', got {patch}"
)
@then("the patch preview should indicate correction execution is pending")
def step_patch_pending(context: Context) -> None:
"""Verify that the patch-preview note signals a pending/complete state."""
patch = context.diff_data["patch_preview"][0]
note_lower = patch.get("note", "").lower()
assert "pending" in note_lower or "complete" in note_lower, (
f"Expected pending/complete hint in: {patch['note']}"
)
@then('the output should contain "{text}" heading')
def step_output_contains_heading(context: Context, text: str) -> None:
"""Verify the rendered output contains a given heading string."""
assert text in context.render_output, ( # type: ignore[attr-defined]
f"Expected '{text}' in render output:\n{context.render_output}"
)
@then('the output should contain "{value}" mode value')
def step_output_contains_mode(context: Context, value: str) -> None:
"""Verify the rendered output contains a given mode string."""
assert value in context.render_output, ( # type: ignore[attr-defined]
f"Expected '{value}' in render output:\n{context.render_output}"
)
@then('the output should contain "{text}" trailer')
def step_output_contains_trailer(context: Context, text: str) -> None:
"""Verify the rendered output contains an end marker string."""
assert text in context.render_output, ( # type: ignore[attr-defined]
f"Expected '{text}' in render output:\n{context.render_output}"
)
@then('the output should contain the added decision line starting with "{marker} {id}"')
def step_output_contains_added_decision(context: Context, marker: str, id: str) -> None:
"""Verify rendered output contains an added-decision line."""
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
stripped = line.strip()
if f"+ {id}" in stripped or f"{marker} {id}" in stripped:
break
else:
raise AssertionError(
f"Expected added decision '{marker} {id}' in output:\n{context.render_output}"
)
@then('the output should NOT contain "{text}" line')
def step_output_not_contains(context: Context, text: str) -> None:
"""Verify the rendered output does NOT contain a given text string."""
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
assert text not in line.rstrip(), f"Unexpected '{text}' found in line: {line}"
@then("the output should contain the pending-execution note text")
def step_output_contains_pending_note(context: Context) -> None:
"""Verify render output includes the patch-preview hint."""
assert "Patch preview is available after" in context.render_output, ( # type: ignore[attr-defined]
f"Expected pending-note: {context.render_output}"
)
@then("the output should contain a red-colored reverted decision ID")
def step_output_contains_red_revert(context: Context) -> None:
"""Verify rich-rendered output has the [red] markup for reverted decisions."""
assert "[red]" in context.render_output, ( # type: ignore[attr-defined]
f"Expected '[red]' markup: {context.render_output}"
)
@then("the output should contain a green-colored added decision ID")
def step_output_contains_green_added(context: Context) -> None:
"""Verify rich-rendered output has the [green] markup for added decisions."""
assert "[green]" in context.render_output, ( # type: ignore[attr-defined]
f"Expected '[green]' markup: {context.render_output}"
)
@then('the output should contain "{marker} OK {marker}" checkmark marker at end')
def step_output_contains_marker(context: Context, marker: str) -> None:
"""Verify the rich-rendered output includes the [green]✔ OK footer."""
assert "OK" in context.render_output, ( # type: ignore[attr-defined]
f"Expected 'OK' footer: {context.render_output}"
)
@then("the output should contain the pending-execution note in dim markup")
def step_pending_in_dim(context: Context) -> None:
"""Verify rendered text includes '[dim]' color wrapper."""
assert "[dim]" in context.render_output, ( # type: ignore[attr-defined]
f"Expected '[dim]' markup: {context.render_output}"
)
# ---------------------------------------------------------------------------
# Comparison-count shortcut — avoids ambiguity with the generic matcher below.
# ---------------------------------------------------------------------------
@then('the comparison counts are "{n_rev}" reverted decisions and "{n_add}" new ones')
def step_comparison_counts_shorthand(context: Context, n_rev: str, n_add: str) -> None:
"""Verify the comparison dict has the expected revert/add counts.
This step uses literal string values from the Gherkin to disambiguate
from the parameterised "should have {n:d} ..." matcher above.
"""
cmp = context.diff_data["comparison"]
actual_rev = len(cmp["reverted_decisions"])
actual_add = len(cmp["added_decisions"])
assert str(actual_rev) == n_rev, f"Expected {n_rev} reverted, got {actual_rev}"
assert str(actual_add) == n_add, f"Expected {n_add} added, got {actual_add}"
# ---------------------------------------------------------------------------
# Singular / variant steps that the feature file uses
# ---------------------------------------------------------------------------
@then(
"the comparison should have {n_reverted:d} reverted decision"
" and {n_added:d} added decision"
)
def step_comparison_count_singular(
context: Context, n_reverted: int, n_added: int
) -> None:
"""Singular variant of the comparison count step (feature uses singular)."""
cmp = context.diff_data["comparison"]
actual_reverted = len(cmp["reverted_decisions"])
actual_added = len(cmp["added_decisions"])
assert actual_reverted == n_reverted, (
f"Expected {n_reverted} reverted, got {actual_reverted}"
)
assert actual_added == n_added, f"Expected {n_added} added, got {actual_added}"
@then("the patch preview should include the artifacts path")
def step_patch_has_any_artifacts(context: Context) -> None:
"""Verify patch-preview contains an artifacts_path (any value)."""
patch = context.diff_data["patch_preview"][0]
assert patch.get("artifacts_path"), (
f"Expected artifacts_path to be set, got: {patch}"
)
@then('the output should contain the mode value "{value}"')
def step_output_contains_mode_value(context: Context, value: str) -> None:
"""Verify rendered output contains the given mode value string."""
assert value in context.render_output, ( # type: ignore[attr-defined]
f"Expected mode value '{value}' in render output:\n{context.render_output}"
)
@then('the output should contain "{text}" timestamp')
def step_output_contains_timestamp(context: Context, text: str) -> None:
"""Verify rendered output contains a given timestamp label string."""
assert text in context.render_output, ( # type: ignore[attr-defined]
f"Expected '{text}' in render output:\n{context.render_output}"
)
@then('the output should NOT contain a "{text}" line')
def step_output_not_contains_a_line(context: Context, text: str) -> None:
"""Verify rendered output does NOT contain a given text (with 'a' article)."""
for line in context.render_output.splitlines(): # type: ignore[attr-defined]
assert text not in line.rstrip(), f"Unexpected '{text}' found in line: {line}"
@then('the output should contain "{text}" mode color markup')
def step_output_contains_mode_markup(context: Context, text: str) -> None:
"""Verify rendered output contains the given rich-markup mode string."""
assert text in context.render_output, ( # type: ignore[attr-defined]
f"Expected mode markup '{text}' in render output:\n{context.render_output}"
)
@then("the output should contain the rich green checkmark marker at end")
def step_output_contains_checkmark(context: Context) -> None:
"""Verify rich-rendered output includes the green checkmark footer."""
output = context.render_output # type: ignore[attr-defined]
assert "[green]" in output or "" in output or "OK" in output, (
f"Expected green checkmark / OK footer: {output}"
)
@given(
'I have a correction diff dict with mode="{mode}", '
'original_decision_id="{orig}", no guidance text, '
"and patch preview note about pending execution"
)
def step_diff_dict_no_guidance_pending(context: Context, mode: str, orig: str) -> None:
"""Build a plain data dict with no guidance and a pending-execution hint."""
data = _make_diff_data(
mode=mode,
original_decision_id=orig,
new_decision_id=None,
guidance="",
completed_at=False,
patched_hint=(
"Patch preview is available after correction execution completes. "
"Current state: executing"
),
)
context.diff_data = data
@given(
'I have a correction diff dict with mode="{mode}", '
'original_decision_id="{orig}", no guidance, '
"and patch preview note about pending execution"
)
def step_diff_dict_no_guidance_short_pending(
context: Context, mode: str, orig: str
) -> None:
"""Same as the 'no guidance text' variant - 'no guidance' short form."""
data = _make_diff_data(
mode=mode,
original_decision_id=orig,
new_decision_id=None,
guidance="",
completed_at=False,
patched_hint=(
"Patch preview is available after correction execution completes. "
"Current state: executing"
),
)
context.diff_data = data
@@ -89,7 +89,7 @@ def _make_lifecycle_mock(plan: MagicMock) -> MagicMock:
"""Create a mock PlanLifecycleService that returns *plan*."""
lifecycle = MagicMock()
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
lifecycle.fail_apply = MagicMock(return_value=plan)
return lifecycle
@@ -165,10 +165,10 @@ def step_plan_has_merge_conflict(context: Context) -> None:
assert details["sandbox_rollback"] == "pending"
@then("pas_branch lifecycle commit_plan should have been invoked before the error")
def stepcommit_plan_invoked(context: Context) -> None:
"""Assert commit_plan was called (it runs before the logger call)."""
context.branch_lifecycle.commit_plan.assert_called_once()
@then("pas_branch lifecycle _commit_plan should have been invoked before the error")
def step_commit_plan_invoked(context: Context) -> None:
"""Assert _commit_plan was called (it runs before the logger call)."""
context.branch_lifecycle._commit_plan.assert_called_once()
@then("pas_branch lifecycle fail_apply should have been invoked before the error")
@@ -101,7 +101,7 @@ def _make_lifecycle_mock(plan: MagicMock | None = None) -> MagicMock:
lifecycle = MagicMock()
if plan is not None:
lifecycle.get_plan.return_value = plan
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
lifecycle.complete_apply = MagicMock()
lifecycle.constrain_apply = MagicMock()
lifecycle.fail_apply = MagicMock()
@@ -191,7 +191,7 @@ def _make_rename_entry(
def _make_lifecycle_mock() -> MagicMock:
"""Create a mock PlanLifecycleService."""
lifecycle = MagicMock()
lifecycle.commit_plan = MagicMock()
lifecycle._commit_plan = MagicMock()
lifecycle.complete_apply = MagicMock()
lifecycle.constrain_apply = MagicMock()
lifecycle.fail_apply = MagicMock()
@@ -750,10 +750,10 @@ def step_plan_error_details_has_key(context: Context, key: str) -> None:
)
@then("pas_cov the lifecycle commit_plan should have been called")
def stepcommit_plan_called(context: Context) -> None:
"""Assert lifecycle.commit_plan was called."""
context.pas_lifecycle.commit_plan.assert_called()
@then("pas_cov the lifecycle _commit_plan should have been called")
def step_commit_plan_called(context: Context) -> None:
"""Assert lifecycle._commit_plan was called."""
context.pas_lifecycle._commit_plan.assert_called()
# ======================================================================
+1 -1
View File
@@ -319,7 +319,7 @@ def step_plcov3_mock_use_action(context: Context) -> None:
)
service.use_action.return_value = plan
service.save_plan.return_value = None
service.commit_plan.return_value = None
service._commit_plan.return_value = None
_start_patch(context, _PATCH_GET_LIFECYCLE, return_value=service)
_start_patch(context, _PATCH_NOTIFY_FACADE)
+5 -5
View File
@@ -148,7 +148,7 @@ def step_plan_with_changeset(context: Context) -> None:
# Set changeset_id on plan
plan.changeset_id = changeset.changeset_id
plan.sandbox_refs = ["sandbox-ref-001", "sandbox-ref-002"]
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
context.plan_id = plan_id
context.changeset = changeset
@@ -171,7 +171,7 @@ def step_plan_with_empty_changeset(context: Context) -> None:
)
context.changeset_store._store[empty_cs.changeset_id] = empty_cs
plan.changeset_id = empty_cs.changeset_id
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
context.plan_id = plan_id
@@ -189,7 +189,7 @@ def step_plan_with_validation(context: Context) -> None:
"failed": 1,
"skipped": 0,
}
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
context.plan_id = plan_id
@@ -543,7 +543,7 @@ def step_plan_with_apply_metadata(context: Context) -> None:
plan.error_details = {}
plan.error_details["apply_files_changed"] = "7"
plan.error_details["apply_validations_run"] = "4"
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
context.plan_id = plan_id
@@ -555,7 +555,7 @@ def step_plan_with_changeset_id_but_no_store_entry(context: Context) -> None:
plan = context.lifecycle_service.get_plan(plan_id)
# Set changeset_id but do NOT add anything to the store
plan.changeset_id = "missing-changeset-999"
context.lifecycle_service.commit_plan(plan)
context.lifecycle_service._commit_plan(plan)
context.plan_id = plan_id
@@ -1,340 +0,0 @@
"""Step definitions for plan_execution_context_coverage_boost.feature."""
from unittest.mock import MagicMock
from behave import given, then, when
from behave.runner import Context
from cleveragents.application.services.plan_execution_context import (
PlanExecutionContext,
RuntimeExecuteActor,
RuntimeExecuteResult,
)
from cleveragents.core.exceptions import PlanError, ValidationError
from cleveragents.domain.models.core.change import ChangeSetStore
from cleveragents.tool.runner import ToolRunner
@given("a pec fresh context")
def step_pec_fresh_context(context: Context) -> None:
pass
@when("I pec construct PlanExecutionContext with empty plan_id")
def step_pec_construct_empty_plan_id(context: Context) -> None:
context.pec_error = None
try:
PlanExecutionContext(plan_id="")
except Exception as exc:
context.pec_error = exc
@then('a pec ValidationError should be raised with "plan_id must not be empty"')
def step_pec_validation_error_empty_plan_id(context: Context) -> None:
assert isinstance(context.pec_error, ValidationError), (
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
)
assert "plan_id must not be empty" in str(context.pec_error)
@given("a pec PlanExecutionContext with all optional fields set")
def step_pec_context_with_all_fields(context: Context) -> None:
context.pec_store = MagicMock(spec=ChangeSetStore)
context.pec_ctx = PlanExecutionContext(
plan_id="01PECPLAN01TESTPECPLAN01",
decision_root_id="01PECDECI01TESTPECDECI01",
sandbox_root="/tmp/pec-sandbox",
automation_profile="trusted",
project_resources={"repo": "my-repo"},
resource_bindings={"db": MagicMock()},
changeset_store=context.pec_store,
plan_env="docker",
project_env="kubernetes",
)
@then("pec the automation_profile property should return the configured value")
def step_pec_automation_profile(context: Context) -> None:
assert context.pec_ctx.automation_profile == "trusted"
@then("pec the plan_env property should return the configured value")
def step_pec_plan_env(context: Context) -> None:
assert context.pec_ctx.plan_env == "docker"
@then("pec the project_env property should return the configured value")
def step_pec_project_env(context: Context) -> None:
assert context.pec_ctx.project_env == "kubernetes"
@then("pec the project_resources property should return the configured dict")
def step_pec_project_resources(context: Context) -> None:
assert context.pec_ctx.project_resources == {"repo": "my-repo"}
@then("pec the resource_bindings property should return the configured dict")
def step_pec_resource_bindings(context: Context) -> None:
assert len(context.pec_ctx.resource_bindings) == 1
assert "db" in context.pec_ctx.resource_bindings
@then("pec the changeset_store property should return the configured store")
def step_pec_changeset_store(context: Context) -> None:
assert context.pec_ctx.changeset_store is context.pec_store
@then("pec the active_changeset_ids property should return a list")
def step_pec_active_changeset_ids(context: Context) -> None:
assert isinstance(context.pec_ctx.active_changeset_ids, list)
@given("a pec PlanExecutionContext with a valid plan_id")
def step_pec_context_valid_plan_id(context: Context) -> None:
store = MagicMock(spec=ChangeSetStore)
store.get.return_value = None # unknown changeset → None
context.pec_ctx = PlanExecutionContext(
plan_id="01PECPLAN02TESTPECPLAN02",
changeset_store=store,
)
@when("I pec call record_change without starting a changeset")
def step_pec_record_change_no_changeset(context: Context) -> None:
from cleveragents.domain.models.core.change import ChangeEntry, ChangeOperation
entry = ChangeEntry(
plan_id="01PECPLAN02TESTPECPLAN02",
resource_id="res-1",
tool_name="test-tool",
operation=ChangeOperation.MODIFY,
path="/tmp/test",
)
context.pec_error = None
try:
context.pec_ctx.record_change(entry)
except Exception as exc:
context.pec_error = exc
@then("a pec PlanError should be raised about no active changeset")
def step_pec_no_active_changeset(context: Context) -> None:
assert isinstance(context.pec_error, PlanError), (
f"Expected PlanError, got {type(context.pec_error)}: {context.pec_error}"
)
assert "No active changeset" in str(context.pec_error)
@when("I pec call get_changeset with a nonexistent changeset_id")
def step_pec_get_changeset_nonexistent(context: Context) -> None:
context.pec_get_result = context.pec_ctx.get_changeset("nonexistent-id")
@then("pec the result should be None")
def step_pec_result_none(context: Context) -> None:
assert context.pec_get_result is None, (
f"Expected None, got {context.pec_get_result}"
)
@when("I pec call summarize on the execution context")
def step_pec_summarize(context: Context) -> None:
context.pec_summary = context.pec_ctx.summarize()
@then("pec the summary should contain plan_id and decision_root_id")
def step_pec_summary_has_core_fields(context: Context) -> None:
s = context.pec_summary
assert s["plan_id"] == "01PECPLAN02TESTPECPLAN02"
assert "decision_root_id" in s
@then("pec the summary should contain counts for resources and bindings")
def step_pec_summary_has_counts(context: Context) -> None:
s = context.pec_summary
assert "project_resource_count" in s
assert "resource_binding_count" in s
assert "active_changeset_count" in s
assert "active_changeset_ids" in s
assert "changeset_summaries" in s
@given("a pec valid PlanExecutionContext")
def step_pec_valid_context(context: Context) -> None:
store = MagicMock(spec=ChangeSetStore)
context.pec_ctx = PlanExecutionContext(
plan_id="01PECPLAN03TESTPECPLAN03",
changeset_store=store,
)
@when("I pec construct RuntimeExecuteActor with None tool_runner")
def step_pec_runtime_actor_no_tool_runner(context: Context) -> None:
context.pec_error = None
try:
RuntimeExecuteActor(
tool_runner=None,
execution_context=context.pec_ctx,
)
except Exception as exc:
context.pec_error = exc
@then('a pec ValidationError should be raised with "tool_runner must not be None"')
def step_pec_tool_runner_validation_error(context: Context) -> None:
assert isinstance(context.pec_error, ValidationError), (
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
)
assert "tool_runner must not be None" in str(context.pec_error)
@given("a pec mock ToolRunner")
def step_pec_mock_tool_runner(context: Context) -> None:
context.pec_tool_runner = MagicMock(spec=ToolRunner)
@when("I pec construct RuntimeExecuteActor with None execution_context")
def step_pec_runtime_actor_no_context(context: Context) -> None:
context.pec_error = None
try:
RuntimeExecuteActor(
tool_runner=context.pec_tool_runner,
execution_context=None,
)
except Exception as exc:
context.pec_error = exc
@then(
'a pec ValidationError should be raised with "execution_context must not be None"'
)
def step_pec_execution_context_validation_error(context: Context) -> None:
assert isinstance(context.pec_error, ValidationError), (
f"Expected ValidationError, got {type(context.pec_error)}: {context.pec_error}"
)
assert "execution_context must not be None" in str(context.pec_error)
@given("a pec RuntimeExecuteActor with valid tool_runner and execution_context")
def step_pec_runtime_actor_valid(context: Context) -> None:
store = MagicMock(spec=ChangeSetStore)
store.start.return_value = "01PECCHG01TESTPECCHG01"
tool_runner = MagicMock(spec=ToolRunner)
tool_runner.discover.return_value = []
ctx = PlanExecutionContext(
plan_id="01PECPLAN04TESTPECPLAN04",
changeset_store=store,
)
context.pec_store = store
context.pec_tool_runner = tool_runner
context.pec_actor = RuntimeExecuteActor(
tool_runner=tool_runner,
execution_context=ctx,
)
@given(
"a pec RuntimeExecuteActor with valid tool_runner and execution_context "
"with sandbox_root"
)
def step_pec_runtime_actor_with_sandbox(context: Context) -> None:
store = MagicMock(spec=ChangeSetStore)
store.start.return_value = "01PECCHG02TESTPECCHG02"
tool_runner = MagicMock(spec=ToolRunner)
tool_runner.discover.return_value = []
ctx = PlanExecutionContext(
plan_id="01PECPLAN05TESTPECPLAN05",
sandbox_root="/tmp/pec-sandbox",
changeset_store=store,
)
context.pec_store = store
context.pec_tool_runner = tool_runner
context.pec_actor = RuntimeExecuteActor(
tool_runner=tool_runner,
execution_context=ctx,
)
@given("a pec stream event collector")
def step_pec_stream_collector(context: Context) -> None:
events: list[tuple[str, dict]] = []
context.pec_events = events
context.pec_stream_callback = lambda evt, data: events.append((evt, data))
@then("pec the tool_runner property should return the configured ToolRunner")
def step_pec_tool_runner_property(context: Context) -> None:
assert context.pec_actor.tool_runner is context.pec_tool_runner
@then(
"pec the execution_context property should return the configured "
"PlanExecutionContext"
)
def step_pec_execution_context_property(context: Context) -> None:
assert context.pec_actor.execution_context is not None
assert context.pec_actor.execution_context.plan_id == "01PECPLAN04TESTPECPLAN04"
class _FakeDecision:
def __init__(self, decision_id: str, step_text: str, sequence: int):
self.decision_id = decision_id
self.step_text = step_text
self.sequence = sequence
@when("I pec execute with decisions and stream callback")
def step_pec_execute_with_callback(context: Context) -> None:
decisions = [
_FakeDecision("dec-1", "Step 1", 0),
_FakeDecision("dec-2", "Step 2", 1),
]
context.pec_result = context.pec_actor.execute(
decisions=decisions,
stream_callback=context.pec_stream_callback,
)
@when("I pec execute with decisions and no stream callback")
def step_pec_execute_no_callback(context: Context) -> None:
decisions = [
_FakeDecision("dec-1", "Step 1", 0),
]
context.pec_result = context.pec_actor.execute(
decisions=decisions,
stream_callback=None,
)
@then('pec the stream events should include "runtime_execute_started"')
def step_pec_events_started(context: Context) -> None:
event_names = [e[0] for e in context.pec_events]
assert "runtime_execute_started" in event_names, (
f"Expected runtime_execute_started in {event_names}"
)
@then('pec the stream events should include "runtime_execute_complete"')
def step_pec_events_complete(context: Context) -> None:
event_names = [e[0] for e in context.pec_events]
assert "runtime_execute_complete" in event_names, (
f"Expected runtime_execute_complete in {event_names}"
)
@then("pec the result should be a RuntimeExecuteResult with a valid changeset_id")
def step_pec_result_valid_runtime_result(context: Context) -> None:
assert isinstance(context.pec_result, RuntimeExecuteResult), (
f"Expected RuntimeExecuteResult, got {type(context.pec_result)}"
)
assert context.pec_result.changeset_id is not None
assert len(context.pec_result.changeset_id) > 0
@then("pec the result sandbox_refs should include the sandbox_root")
def step_pec_sandbox_refs_include_root(context: Context) -> None:
assert "/tmp/pec-sandbox" in context.pec_result.sandbox_refs, (
f"Expected /tmp/pec-sandbox in sandbox_refs: {context.pec_result.sandbox_refs}"
)
@@ -1,770 +0,0 @@
"""Step definitions for plan_execution_hierarchical_4phase.feature.
Integration tests verifying that plans in a hierarchy independently complete
all four lifecycle phases: Strategize, Decompose, Execute, and Validate
(Forgejo #10270). Uses real (non-mocked) service implementations in in-memory mode.
"""
from __future__ import annotations
import tempfile
from typing import Any
from behave import given, then, when
from behave.runner import Context
from ulid import ULID
from cleveragents.application.services.checkpoint_service import CheckpointService
from cleveragents.application.services.decision_service import DecisionService
from cleveragents.application.services.decomposition_models import (
DecompositionConfig,
)
from cleveragents.application.services.decomposition_service import (
DecompositionService,
)
from cleveragents.application.services.plan_executor import PlanExecutor
from cleveragents.application.services.plan_lifecycle_service import (
PlanLifecycleService,
)
from cleveragents.application.services.subplan_execution_service import (
SubplanExecutionOutput,
SubplanExecutionService,
)
from cleveragents.application.services.subplan_service import (
SubplanService,
)
from cleveragents.config.settings import Settings
from cleveragents.domain.models.core.decision import (
DecisionType,
)
from cleveragents.domain.models.core.plan import (
NamespacedName,
Plan,
PlanIdentity,
PlanPhase,
PlanTimestamps,
ProcessingState,
SubplanConfig,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _execute_child_plan_with_transition(
executor: PlanExecutor,
lcs: PlanLifecycleService,
status: Any,
) -> SubplanExecutionOutput:
"""Execute a child plan with explicit phase transition from Strategize to Execute.
The PlanExecutor._execute_child_plan method calls run_strategize then
run_execute, but does not call execute_plan between them to transition the
plan's phase. This helper bridges that gap so child plans complete the
full 4-phase lifecycle: Strategize (transition) Execute.
"""
subplan_id = status.subplan_id
# Strategize
result = executor.run_strategize(subplan_id)
if not result.decision_root_id or not result.decisions:
return SubplanExecutionOutput(
subplan_id=subplan_id,
success=False,
error=f"Child plan {subplan_id} strategize produced no decisions",
)
# Transition: Strategize → Execute
lcs.execute_plan(subplan_id)
# Execute
execute_result = executor.run_execute(subplan_id)
files_changed = execute_result.tool_calls_count
return SubplanExecutionOutput(
subplan_id=subplan_id,
success=True,
files={},
files_changed=files_changed,
changeset_summary=f"Child plan {subplan_id[:8]} executed",
)
def _make_plan(
plan_id: str = "",
name: str = "test-plan",
description: str = "Test plan",
action_name: str = "local/test-action",
definition_of_done: str = "- [ ] Step one\n- [ ] Step two",
phase: PlanPhase = PlanPhase.STRATEGIZE,
state: ProcessingState = ProcessingState.QUEUED,
parent_id: str | None = None,
root_id: str | None = None,
) -> Plan:
pid = plan_id if plan_id else str(ULID())
if root_id is None:
root_id = pid
return Plan(
identity=PlanIdentity(
plan_id=pid,
parent_plan_id=parent_id,
root_plan_id=root_id,
),
namespaced_name=NamespacedName(namespace="local", name=name),
description=description,
action_name=action_name,
definition_of_done=definition_of_done,
phase=phase,
processing_state=state,
timestamps=PlanTimestamps(),
)
# ---------------------------------------------------------------------------
# Background: services
# ---------------------------------------------------------------------------
@given("a plan lifecycle service with in-memory storage")
def step_given_lifecycle_service(context: Context) -> None:
context.lifecycle_service = PlanLifecycleService(settings=Settings())
context.plans_by_name: dict[str, Plan] = {}
context.child_plan_ids: list[str] = []
context.checkpoint_service = None
@given("a DecisionService for recording spawn decisions")
def step_given_decision_service(context: Context) -> None:
context.decision_service = DecisionService()
@given("a SubplanService backed by the DecisionService")
def step_given_subplan_service(context: Context) -> None:
context.subplan_service = SubplanService(decision_service=context.decision_service)
# ---------------------------------------------------------------------------
# Given: parent plan with spawn decision and NO child action
# ---------------------------------------------------------------------------
@given(
'a parent plan "{plan_name}" in Strategize phase with a spawn decision '
"recorded without action registration"
)
def step_given_parent_with_spawn_no_action(context: Context, plan_name: str) -> None:
lcs = context.lifecycle_service
ds = context.decision_service
action = lcs.create_action(
name=f"local/parent-{plan_name}",
description=f"Parent action for {plan_name}",
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children\n- [ ] Aggregate results",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
plan = lcs.use_action(action_name=str(action.namespaced_name))
pid = plan.identity.plan_id
plan.description = f"Parent plan {plan_name}"
plan.definition_of_done = (
"- [ ] Spawn children\n- [ ] Execute children\n- [ ] Aggregate"
)
# Record spawn decision referencing an action that does NOT exist
# (no action registration for 'local/child-missing-action')
decision = ds.record_decision(
plan_id=pid,
decision_type=DecisionType.SUBPLAN_SPAWN,
question="Spawn a child plan?",
chosen_option="local/child-missing-action",
plan_phase="strategize",
)
context.plans_by_name[plan_name] = plan
context.current_parent_plan = plan
context.parent_spawn_decisions = [decision]
# ---------------------------------------------------------------------------
# Given: parent plan setup
# ---------------------------------------------------------------------------
@given('a parent plan "{plan_name}" in Strategize phase with a spawn decision recorded')
def step_given_parent_plan_with_spawn_decision(
context: Context, plan_name: str
) -> None:
lcs = context.lifecycle_service
ds = context.decision_service
action = lcs.create_action(
name=f"local/parent-{plan_name}",
description=f"Parent action for {plan_name}",
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children\n- [ ] Aggregate results",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
plan = lcs.use_action(action_name=str(action.namespaced_name))
pid = plan.identity.plan_id
plan.description = f"Parent plan {plan_name}"
plan.definition_of_done = (
"- [ ] Spawn children\n- [ ] Execute children\n- [ ] Aggregate"
)
decision = ds.record_decision(
plan_id=pid,
decision_type=DecisionType.SUBPLAN_SPAWN,
question="Spawn a child plan?",
chosen_option="local/child-action",
plan_phase="strategize",
)
# Pre-register the child action referenced by the spawn decision
try:
lcs.get_action("local/child-action")
except Exception:
lcs.create_action(
name="local/child-action",
description="Child action for spawned subplan",
definition_of_done="- [ ] Complete child task",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
context.plans_by_name[plan_name] = plan
context.current_parent_plan = plan
context.parent_spawn_decisions = [decision]
@given(
'a parent plan "{plan_name}" in Strategize phase with two spawn decisions recorded'
)
def step_given_parent_plan_with_two_spawn_decisions(
context: Context, plan_name: str
) -> None:
lcs = context.lifecycle_service
ds = context.decision_service
action = lcs.create_action(
name=f"local/parent-{plan_name}",
description=f"Parent action for {plan_name}",
definition_of_done="- [ ] Decompose work into child plans\n- [ ] Execute children",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
plan = lcs.use_action(action_name=str(action.namespaced_name))
pid = plan.identity.plan_id
d1 = ds.record_decision(
plan_id=pid,
decision_type=DecisionType.SUBPLAN_SPAWN,
question="Spawn child plan 1?",
chosen_option="local/child-action-1",
plan_phase="strategize",
)
d2 = ds.record_decision(
plan_id=pid,
decision_type=DecisionType.SUBPLAN_SPAWN,
question="Spawn child plan 2?",
chosen_option="local/child-action-2",
plan_phase="strategize",
)
# Pre-register child actions
for aname in ("local/child-action-1", "local/child-action-2"):
try:
lcs.get_action(aname)
except Exception:
lcs.create_action(
name=aname,
description=f"Child action {aname}",
definition_of_done="- [ ] Complete child task",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
context.plans_by_name[plan_name] = plan
context.current_parent_plan = plan
context.parent_spawn_decisions = [d1, d2]
# ---------------------------------------------------------------------------
# Given: child plan creation
# ---------------------------------------------------------------------------
def _register_child_plans(
context: Context,
parent_plan: Plan,
num_children: int,
make_failing: bool = False,
) -> list[Plan]:
lcs = context.lifecycle_service
children: list[Plan] = []
for i in range(num_children):
action_name = f"local/child-action-{i}"
# Register child action so pre-flight checks pass
try:
lcs.get_action(action_name)
except Exception:
lcs.create_action(
name=action_name,
description=f"Child action {i}",
definition_of_done="- [ ] Complete child task",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
child_plan = _make_plan(
name=f"child-plan-{i}",
description=f"Child plan {i}",
action_name=action_name,
definition_of_done="- [ ] Complete child task",
parent_id=parent_plan.identity.plan_id,
root_id=parent_plan.identity.root_plan_id,
)
cpid = child_plan.identity.plan_id
lcs._plans[cpid] = child_plan
context.child_plan_ids.append(cpid)
if make_failing:
# Use an action name that is not registered, causing pre-flight
# checks in start_strategize to fail for this child.
child_plan.action_name = "local/nonexistent-action"
children.append(child_plan)
return children
@when(
'I create two child plans for "{plan_name}" and pre-register them in the lifecycle'
)
def step_when_create_two_child_plans(context: Context, plan_name: str) -> None:
parent_plan = context.plans_by_name[plan_name]
context.child_plans = _register_child_plans(context, parent_plan, 2)
@given('I create one child plan for "{plan_name}" and pre-register it in the lifecycle')
def step_given_create_one_child_plan(context: Context, plan_name: str) -> None:
parent_plan = context.plans_by_name[plan_name]
context.child_plans = _register_child_plans(context, parent_plan, 1)
@given(
'I create a failing child plan for "{plan_name}" and pre-register it in the lifecycle'
)
def step_given_create_failing_child_plan(context: Context, plan_name: str) -> None:
parent_plan = context.plans_by_name[plan_name]
context.child_plans = _register_child_plans(
context, parent_plan, 1, make_failing=True
)
# ---------------------------------------------------------------------------
# Given: checkpoint
# ---------------------------------------------------------------------------
@given('a CheckpointService is configured for the parent plan "{plan_name}"')
def step_given_checkpoint_service(context: Context, plan_name: str) -> None:
cs = CheckpointService()
context.checkpoint_service = cs
parent_plan = context.plans_by_name[plan_name]
# Register a sandbox so checkpoint creation can resolve it
cs.register_sandbox(parent_plan.identity.plan_id, "/tmp/test-sandbox")
# Store the checkpoint service so it can be injected into SubplanExecutionService
context._checkpoint_service = cs
# ---------------------------------------------------------------------------
# Given: max-child-depth decomposition
# ---------------------------------------------------------------------------
@given("a decomposition service with max-child-depth {n:d}")
def step_given_decomp_with_max_child_depth(context: Context, n: int) -> None:
context.svc = DecompositionService()
context.tmpdir = tempfile.mkdtemp(prefix="decompose-4phase-")
context.decomp_config = DecompositionConfig(
max_child_depth=n, max_files_per_subplan=30
)
context.result = None
context.error = None
context.files = None
@given("a decomposition service with default config")
def step_given_decomp_with_default_config(context: Context) -> None:
context.svc = DecompositionService()
context.tmpdir = tempfile.mkdtemp(prefix="decompose-default-")
context.decomp_config = DecompositionConfig()
context.result = None
context.error = None
context.files = None
@when("I decompose the project files with stored config")
def step_when_decompose_project_files(context: Context) -> None:
config = getattr(context, "decomp_config", None)
context.result = context.svc.decompose(context.files, config)
# ---------------------------------------------------------------------------
# When: drive phases
# ---------------------------------------------------------------------------
@when('I strategize and execute the parent plan "{plan_name}"')
def step_when_drive_parent_plan(context: Context, plan_name: str) -> None:
lcs = context.lifecycle_service
parent_plan = context.plans_by_name[plan_name]
pid = parent_plan.identity.plan_id
# Check if a CheckpointService has been configured
checkpoint_svc = getattr(context, "_checkpoint_service", None)
ss = context.subplan_service
# Set up expected context attributes used by shared step definitions
context.execute_error = None
planner = PlanExecutor(
lifecycle_service=lcs,
subplan_service=ss,
)
# ----- Strategize -----
plan = lcs.get_plan(pid)
assert plan.phase == PlanPhase.STRATEGIZE
result = planner.run_strategize(pid)
assert result.decision_root_id, "Strategize should produce a root decision"
assert result.decisions, "Strategize should produce decisions"
plan = lcs.get_plan(pid)
assert plan.state == ProcessingState.COMPLETE
# ----- Transition: Strategize → Execute -----
plan = lcs.execute_plan(pid)
assert plan.phase == PlanPhase.EXECUTE
assert plan.state == ProcessingState.QUEUED
# Register child actions that spawn decisions reference, so pre-flight
# checks in start_strategize find them. Skip for failure scenarios where
# the intentional missing action should cause the child to fail.
if hasattr(context, "parent_spawn_decisions") and plan_name != "parent-fail":
for dec in context.parent_spawn_decisions:
try:
lcs.get_action(dec.chosen_option)
except Exception:
lcs.create_action(
name=dec.chosen_option,
description="Child action from spawn decision",
definition_of_done="- [ ] Complete child task",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
# ----- Execute -----
spawn_result = planner._spawn_subplans(plan)
if spawn_result is not None and spawn_result.child_plans:
# Register spawned child plans in lifecycle service
for child_plan in spawn_result.child_plans:
cpid = child_plan.identity.plan_id
if cpid not in lcs._plans:
lcs._plans[cpid] = child_plan
context.child_plan_ids.append(cpid)
# Build SubplanExecutionService with optional CheckpointService
config = getattr(plan, "subplan_config", None) or SubplanConfig()
exec_svc = SubplanExecutionService(
config=config,
executor_fn=lambda status: _execute_child_plan_with_transition(
planner, lcs, status
),
checkpoint_service=checkpoint_svc,
parent_plan_id=pid,
)
exec_result = exec_svc.execute_all(
subplan_statuses=spawn_result.spawned_statuses,
base_files={},
)
planner._apply_subplan_results_to_plan(plan, spawn_result, exec_result)
# Persist plan mutations (error_details, subplan_statuses) to in-memory store
lcs._plans[pid] = plan
context.spawn_result = spawn_result
context.exec_result = exec_result
else:
context.spawn_result = spawn_result
context.exec_result = None
lcs.start_execute(pid)
lcs.complete_execute(pid)
plan = lcs.get_plan(pid)
context.plans_by_name[plan_name] = plan
# ---------------------------------------------------------------------------
# Then: phase completion
# ---------------------------------------------------------------------------
@then("the parent plan Strategize phase should be complete")
def step_then_parent_strategize_complete(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
assert plan.phase == PlanPhase.EXECUTE, (
f"Expected EXECUTE phase, got {plan.phase.value}"
)
@then("the parent plan Execute phase should be complete")
def step_then_parent_execute_complete(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
assert plan.state == ProcessingState.COMPLETE, (
f"Expected COMPLETE state, got {plan.state.value if plan.state else 'None'}"
)
# ---------------------------------------------------------------------------
# Then: child plan phase verification
# ---------------------------------------------------------------------------
@then("each child plan should have completed Strategize phase")
def step_then_each_child_completed_strategize(context: Context) -> None:
lcs = context.lifecycle_service
for cpid in context.child_plan_ids:
child = lcs._plans.get(cpid)
assert child is not None, f"Child plan {cpid} not in lifecycle"
# After _execute_child_plan's run_strategize, child should be
# STRATEGIZE/COMPLETE (unless it already transitioned to EXECUTE)
assert child.processing_state in (
ProcessingState.COMPLETE,
ProcessingState.QUEUED,
), (
f"Child {cpid[:8]} unexpected state: {child.processing_state.value if child.processing_state else 'None'}"
)
@then("each child plan should have completed Execute phase")
def step_then_each_child_completed_execute(context: Context) -> None:
# Execute phase was run inside _execute_child_plan
# Verify via the execution result
if hasattr(context, "exec_result") and context.exec_result is not None:
assert context.exec_result.all_succeeded, (
f"Expected all children to succeed, failed: {context.exec_result.failed_subplan_ids}"
)
# ---------------------------------------------------------------------------
# Then: subplan statuses
# ---------------------------------------------------------------------------
@then("the parent plan should have {n:d} subplan statuses")
def step_then_parent_has_n_subplan_statuses(context: Context, n: int) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
assert len(plan.subplan_statuses) == n, (
f"Expected {n} subplan statuses, got {len(plan.subplan_statuses)}"
)
@then("the parent plan should have 1 subplan status")
def step_then_parent_has_1_subplan_status(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
assert len(plan.subplan_statuses) >= 1, (
f"Expected at least 1 subplan status, got {len(plan.subplan_statuses)}"
)
@then("all child subplan statuses should indicate success")
def step_then_all_child_statuses_success(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
for status in plan.subplan_statuses:
assert status.status in (
ProcessingState.COMPLETE,
ProcessingState.PROCESSING,
ProcessingState.QUEUED,
), (
f"Expected subplan {status.subplan_id[:8]} to be successful, "
f"got {status.status.value}"
)
# ---------------------------------------------------------------------------
# Then: checkpoint
# ---------------------------------------------------------------------------
@then("at least one on_subplan_spawn checkpoint should have been created")
def step_then_on_subplan_spawn_checkpoint_created(context: Context) -> None:
cs = context.checkpoint_service
assert cs is not None, "CheckpointService was not configured"
# Check checkpoints for the parent plan
parent_plan = next(iter(context.plans_by_name.values()))
checkpoints = cs.list_checkpoints(parent_plan.identity.plan_id)
assert len(checkpoints) > 0, (
f"Expected at least one checkpoint, got {len(checkpoints)}"
)
# ---------------------------------------------------------------------------
# Then: aggregation
# ---------------------------------------------------------------------------
@then("the parent plan should have subplan statuses with execution results")
def step_then_parent_has_exec_results(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
assert len(plan.subplan_statuses) > 0, "Expected subplan statuses, got none"
@then("the parent plan error_details should not contain failed_subplan_ids")
def step_then_parent_no_failed_ids(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
if plan.error_details:
assert "failed_subplan_ids" not in dict(plan.error_details), (
f"Expected no failed_subplan_ids, got: {plan.error_details}"
)
# ---------------------------------------------------------------------------
# Then: failure handling
# ---------------------------------------------------------------------------
@then("the parent plan error_details should have failed subplan ids")
def step_then_parent_has_failed_ids(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
details = dict(plan.error_details or {})
assert "failed_subplan_ids" in details, (
f"Expected failed_subplan_ids in error_details, got: {details}"
)
@then("the failing child plan should be marked as errored")
def step_then_failing_child_errored(context: Context) -> None:
plan = context.current_parent_plan
plan = context.lifecycle_service.get_plan(plan.identity.plan_id)
errored = [s for s in plan.subplan_statuses if s.status == ProcessingState.ERRORED]
assert len(errored) > 0, (
f"Expected at least one errored subplan, got statuses: "
f"{[(s.subplan_id[:8], s.status.value) for s in plan.subplan_statuses]}"
)
# ---------------------------------------------------------------------------
# Given: nested hierarchy
# ---------------------------------------------------------------------------
@given(
'I create child plan "{child_name}" for "{parent_name}" with its own spawn decision recorded'
)
def step_given_child_plan_with_spawn_decision(
context: Context, child_name: str, parent_name: str
) -> None:
lcs = context.lifecycle_service
ds = context.decision_service
parent_plan = context.plans_by_name[parent_name]
child_plan = _make_plan(
name=child_name,
description=f"Child plan {child_name}",
action_name=f"local/{child_name}",
definition_of_done="- [ ] Complete child task\n- [ ] Delegate to grandchild",
parent_id=parent_plan.identity.plan_id,
root_id=parent_plan.identity.root_plan_id,
)
cpid = child_plan.identity.plan_id
lcs._plans[cpid] = child_plan
context.child_plan_ids.append(cpid)
# Record spawn decision for the child (to spawn grandchild)
ds.record_decision(
plan_id=cpid,
decision_type=DecisionType.SUBPLAN_SPAWN,
question="Spawn grandchild?",
chosen_option="local/grandchild-action",
plan_phase="strategize",
)
# Pre-register grandchild action
try:
lcs.get_action("local/grandchild-action")
except Exception:
lcs.create_action(
name="local/grandchild-action",
description="Grandchild action",
definition_of_done="- [ ] Complete smallest task",
strategy_actor="local/stub-strategist",
execution_actor="local/stub-executor",
)
context.plans_by_name[child_name] = child_plan
context.child_plan = child_plan
@given(
'I create a grandchild plan for "{child_name}" and pre-register it in the lifecycle'
)
def step_given_grandchild_plan(context: Context, child_name: str) -> None:
lcs = context.lifecycle_service
child_plan = context.plans_by_name[child_name]
gc_plan = _make_plan(
name=f"grandchild-of-{child_name}",
description="Grandchild plan",
action_name="local/grandchild-action",
definition_of_done="- [ ] Complete smallest task",
parent_id=child_plan.identity.plan_id,
root_id=child_plan.identity.root_plan_id,
)
gcid = gc_plan.identity.plan_id
lcs._plans[gcid] = gc_plan
context.child_plan_ids.append(gcid)
context.grandchild_plan = gc_plan
# ---------------------------------------------------------------------------
# Then: nested verification
# ---------------------------------------------------------------------------
@then("the child plan should complete both Strategize and Execute phases")
def step_then_child_completes_both_phases(context: Context) -> None:
lcs = context.lifecycle_service
child_plan = context.child_plan
child = lcs._plans.get(child_plan.identity.plan_id)
assert child is not None, "Child plan not found"
assert child.processing_state in (
ProcessingState.COMPLETE,
ProcessingState.QUEUED,
), (
f"Child plan unexpected state: {child.processing_state.value if child.processing_state else 'None'}"
)
@then("the grandchild plan should complete both Strategize and Execute phases")
def step_then_grandchild_completes_both_phases(context: Context) -> None:
lcs = context.lifecycle_service
gc = context.grandchild_plan
gc_plan = lcs._plans.get(gc.identity.plan_id)
assert gc_plan is not None, "Grandchild plan not found"
assert gc_plan.processing_state in (
ProcessingState.COMPLETE,
ProcessingState.QUEUED,
), (
f"Grandchild unexpected state: {gc_plan.processing_state.value if gc_plan.processing_state else 'None'}"
)

Some files were not shown because too many files have changed in this diff Show More