Renamed all 11 task-type confidence threshold fields in AutomationProfile from phase-transition semantics to spec-defined task-type semantics. Updated all 8 built-in profiles, CLI formatting, YAML schema, services, and all Behave/Robot tests referencing the old field names. Post-review fixes: - Fixed 24 stale old field names in M6 fixture files (automation_profiles.json, autonomy_guardrails.json) - Added model_validator(mode='before') to detect legacy field names and raise actionable ValueError with rename mapping - Added semantic bridge comments in PlanLifecycleService mapping task-type thresholds to phase-transition gates - Added threshold_field to structured log messages for observability - Restored categorised CLI automation-profile show output to match spec (Phase Transitions / Decision Automation / Self-Repair / Execution Controls) instead of flat list - Added missing access_network field to spec show output examples (Rich, Plain, JSON, YAML variants) - Aligned ADR-017 profile fields table to all 11 fields with descriptions matching spec Automatable Tasks table - Aligned automation_profiles.md threshold descriptions with spec - Added spec section references in phase_reversion.md, error_recovery.md, and plan_execute.md for field naming context - Extended repository roundtrip test to assert all 11 threshold fields - Fixed benchmark _make_profile() passing safety fields as top-level kwargs instead of via SafetyProfile sub-model (incompatible with extra="forbid") - Aligned CLI JSON/YAML output structure for automation-profile show with the specification grouped format (phase_transitions, decision_automation, self_repair, execution_controls) - Moved safety boolean fields into the Execution Controls section of Rich output per spec examples - Reverted auto profile description to "Fully automatic except apply" per specification (line 16703, line 28406) - Improved bridge comments in test steps with semantic context for threshold-to-gate mappings ISSUES CLOSED: #902
13 KiB
Plan Execute: Strategize & Execute Integration
Overview
The plan executor connects the PlanLifecycleService to stub actors that
drive plans through the Strategize and Execute phases. In M1, these
actors are local-only stubs (no LLM calls); future milestones will integrate
real AI providers.
When a PlanExecutionContext is provided, the execute phase delegates to
RuntimeExecuteActor for full tool-calling runtime integration with
changeset capture through ChangeSetStore.
Architecture
PlanExecutor
├── StrategizeStubActor (read-only, produces decision tree)
├── ExecuteStubActor (legacy stub: sandbox + ChangeSetCapture)
├── RuntimeExecuteActor (runtime: ToolRunner + ChangeSetStore)
├── PlanExecutionContext (plan metadata + resource bindings)
├── PlanLifecycleService (phase transitions, persistence)
└── ErrorRecoveryService (optional — error recording + retry logic)
Execution Modes
| Mode | Actor | Trigger | Output |
|---|---|---|---|
| Stub | ExecuteStubActor | No execution_context |
ExecuteResult |
| Runtime | RuntimeExecuteActor | execution_context is provided |
RuntimeExecuteResult |
PlanExecutionContext
The PlanExecutionContext bridges plan metadata into the tool runtime:
from cleveragents.application.services.plan_execution_context import (
PlanExecutionContext,
)
from cleveragents.domain.models.core.change import InMemoryChangeSetStore
ctx = PlanExecutionContext(
plan_id="01HGZ...",
decision_root_id="01HGZ...",
sandbox_root="/tmp/sandbox",
automation_profile="trusted",
project_resources={"repo": {"path": "/code"}},
changeset_store=InMemoryChangeSetStore(),
)
# Start a changeset for tracking mutations
changeset_id = ctx.start_changeset()
# Record changes during execution
ctx.record_change(entry)
# Retrieve changeset
cs = ctx.get_changeset(changeset_id)
# Summarize context state
summary = ctx.summarize()
Fields
| Field | Type | Required | Description |
|---|---|---|---|
plan_id |
str |
Yes | ULID of the plan |
decision_root_id |
str | None |
No | Root decision from strategize |
sandbox_root |
str | None |
No | Sandbox filesystem path |
automation_profile |
str | None |
No | Automation profile name |
project_resources |
dict[str, Any] |
No | Project resource metadata |
resource_bindings |
dict[str, BoundResource] |
No | Resolved resource bindings |
changeset_store |
ChangeSetStore |
No | Defaults to InMemoryChangeSetStore |
RuntimeExecuteActor
Wraps ToolRunner to execute strategy decisions with changeset capture:
from cleveragents.application.services.plan_execution_context import (
RuntimeExecuteActor,
RuntimeExecuteResult,
)
actor = RuntimeExecuteActor(
tool_runner=runner,
execution_context=ctx,
)
result: RuntimeExecuteResult = actor.execute(decisions)
RuntimeExecuteResult Fields
| Field | Type | Description |
|---|---|---|
changeset_id |
str |
ULID of the produced changeset |
tool_call_count |
int |
Number of tool calls made |
sandbox_refs |
list[str] |
Sandbox reference paths |
decision_ids_processed |
list[str] |
Processed decision node IDs |
execution_duration_ms |
float |
Wall-clock execution time (ms) |
PlanExecutor Runtime Mode
The PlanExecutor auto-selects runtime vs stub mode:
from cleveragents.application.services.plan_executor import PlanExecutor
# Stub mode (no execution_context)
executor = PlanExecutor(lifecycle_service=lifecycle, tool_runner=runner)
assert not executor.has_runtime
# Runtime mode (with execution_context)
executor = PlanExecutor(
lifecycle_service=lifecycle,
tool_runner=runner,
execution_context=ctx,
)
assert executor.has_runtime
assert executor.changeset_store is not None
# Execute auto-dispatches to RuntimeExecuteActor
result = executor.run_execute(plan_id)
CLI Executor Wiring (_get_plan_executor)
The plan execute CLI command constructs a PlanExecutor via the
internal _get_plan_executor() helper in
cleveragents.cli.commands.plan. This helper resolves the
ProviderRegistry from the DI container and builds
LLMStrategizeActor / LLMExecuteActor for real LLM calls.
def _get_plan_executor(lifecycle_service: PlanLifecycleService | None = None):
"""Build a PlanExecutor wired with real LLM actors.
Args:
lifecycle_service: Optional pre-existing PlanLifecycleService.
When provided the executor shares the same service (and its
in-memory plan cache) as the caller.
"""
Shared Lifecycle Service Instance
Because PlanLifecycleService is registered as a Factory provider
in the DI container, each container.plan_lifecycle_service() call
returns a new instance with its own in-memory _plans cache. The
plan execute handler creates one service for CLI-level state reads
and passes it to _get_plan_executor() so the executor shares the
same instance:
# In the plan execute CLI handler:
service = _get_lifecycle_service()
executor = _get_plan_executor(lifecycle_service=service) # shared instance
This is critical because executor.run_strategize() mutates plan
state through the lifecycle service (e.g. advancing from
strategize/complete to execute/queued via auto_progress). If
the executor used a separate service instance, the CLI handler's
subsequent service.get_plan() call would return stale cached state,
causing spurious "not in an executable state" errors.
Note: When
lifecycle_serviceis omitted (e.g. from test code or standalone scripts),_get_plan_executorfalls back to creating a fresh instance from the container.
Properties
| Property | Type | Description |
|---|---|---|
has_runtime |
bool |
True if execution_context is set |
changeset_store |
ChangeSetStore | None |
Store from execution context |
execution_context |
PlanExecutionContext | None |
The execution context |
ChangeSetStore Wiring
The ChangeSetStore protocol defines the interface for changeset persistence:
class ChangeSetStore(Protocol):
def start(self, plan_id: str) -> str: ...
def record(self, changeset_id: str, entry: ChangeEntry) -> None: ...
def get(self, changeset_id: str) -> SpecChangeSet | None: ...
def get_for_plan(self, plan_id: str) -> list[SpecChangeSet]: ...
def summarize(self, changeset_id: str) -> dict[str, Any]: ...
InMemoryChangeSetStore is the default for M1. Database-backed
implementations will be added in D1 milestone.
Phase Lifecycle
| Phase | Actor | Mode | Output |
|---|---|---|---|
| Strategize | StrategizeStubActor | Read-only | Decision tree, invariant records |
| Execute | RuntimeExecuteActor | Runtime | ChangeSet via ChangeSetStore |
| Execute | ExecuteStubActor | Stub | ChangeSet via ChangeSetCapture |
Strategize Phase
The strategize phase is read-only: it produces a decision tree from the
action's definition_of_done without modifying any resources.
Decision Tree
The stub actor parses definition_of_done into discrete steps, each
represented as a StrategyDecision node with a ULID identifier:
from cleveragents.application.services.plan_executor import (
PlanExecutor,
StrategizeResult,
)
executor = PlanExecutor(
lifecycle_service=lifecycle,
tool_runner=runner,
execution_context=ctx, # optional — enables runtime mode
error_recovery_service=er_service, # optional — enables retry logic
)
result: StrategizeResult = executor.run_strategize(plan_id)
# result.decision_root_id -> ULID of root node
# result.decisions -> list[StrategyDecision]
# result.invariant_records -> list[dict] (stub enforcement records)
Invariant Propagation
Project and action invariants are propagated into the strategize context. In M1, enforcement is stubbed (all invariants are accepted). Full reconciliation via the Invariant Reconciliation Actor lands in D2.
Execute Phase
The execute phase uses sandbox resources with tool calls routed through
ToolRunner and captured by ChangeSetCapture.
ChangeSet Capture
All tool mutations during execute are recorded in a ChangeSet:
result: ExecuteResult = executor.run_execute(plan_id)
# result.changeset_id -> ULID of the changeset
# result.changeset -> ChangeSet with entries
# result.tool_calls_count -> int
# result.sandbox_refs -> list[str]
Metadata Persistence
After execute completes, the following metadata is persisted on the Plan:
changeset_id: The ChangeSet identifiersandbox_refs: List of sandbox reference pathserror_details: Tool call count and sandbox ref count
Phase Guards
- Execute requires Strategize COMPLETE: The executor validates that the
plan has completed strategize (has a
decision_root_id) before allowing execute to proceed. - Phase validation: Both
run_strategize()andrun_execute()verify the plan is in the correct phase before proceeding.
Error Handling
Failures in either phase are captured with full error context:
error_message: The exception message stringerror_details: Dict withexception_type,traceback,mode, and structured error-recovery metadata (category, retry count, recovery hints) when anErrorRecoveryServiceis attached- The plan transitions to
ERROREDprocessing state
Automatic Retry (D1b)
When an ErrorRecoveryService is provided to the executor, the
Execute phase (stub mode) includes an automatic retry loop:
- On failure the error is classified (transient, validation, etc.)
and recorded via
ErrorRecoveryService.record_error(). - If the error is retriable and retry attempts remain, the execute actor is re-invoked automatically.
- If retries are exhausted or the error is non-retriable, the plan
transitions to
ERROREDwith full recovery hints stored inerror_details.
Retry limits default to 3 and are controlled by the automation
profile's modify_config threshold (named per spec § Automatable
Tasks; controls transient-failure retry gating).
Use agents plan errors <plan_id> to inspect error decisions,
retry history, and recovery suggestions for a plan.
See Error Recovery Reference for full details.
Manual Recovery
Plans in ERRORED state without automatic retry can be recovered by:
- Reviewing errors via
agents plan errors <plan_id> - Resetting the plan's processing state back to
QUEUED - Re-running the failed phase via the executor
Streaming Hooks
Both phases accept an optional stream_callback:
Event Types
| Event | Phase | Actor | Description |
|---|---|---|---|
strategize_started |
Strat. | Stub | Phase processing began |
strategize_decisions |
Strat. | Stub | Decisions produced |
strategize_complete |
Strat. | Stub | Phase completed |
execute_started |
Execute | Stub | Stub execute began |
execute_step |
Execute | Stub | Stub decision step |
execute_complete |
Execute | Stub | Stub execute completed |
runtime_execute_started |
Execute | Runtime | Runtime execute began |
runtime_execute_step |
Execute | Runtime | Runtime decision step |
runtime_execute_complete |
Execute | Runtime | Runtime execute completed |
Module Reference
cleveragents.application.services.plan_execution_contextPlanExecutionContext: Execution context bridging plan to runtimeRuntimeExecuteActor: Tool-calling execute actorRuntimeExecuteResult: Runtime execution output model
cleveragents.application.services.plan_executorPlanExecutor: Orchestrator with runtime/stub mode selectionStrategizeStubActor: Local-only strategize actorExecuteStubActor: Local-only execute actorStrategyDecision,StrategizeResult,ExecuteResult: Data modelsStreamCallback: Type alias for streaming callbacks
cleveragents.application.services.error_recovery_service: Error recovery integration (optional)