freemo
a2da043cbd
CI / lint (pull_request) Successful in 25s
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 21s
CI / typecheck (pull_request) Successful in 39s
CI / security (pull_request) Successful in 37s
CI / build (pull_request) Successful in 26s
CI / integration_tests (pull_request) Successful in 4m43s
CI / unit_tests (pull_request) Successful in 11m2s
CI / docker (pull_request) Successful in 58s
CI / benchmark-regression (pull_request) Successful in 18m58s
CI / coverage (pull_request) Successful in 24m24s
Adds PlanExecutionContext carrying plan metadata and delegating changeset ops to ChangeSetStore. RuntimeExecuteResult captures execution output (changeset_id, tool_call_count, sandbox_refs, decision_ids_processed, execution_duration_ms). RuntimeExecuteActor dispatches StrategyDecision lists through ToolRunner with full changeset capture and optional streaming callbacks. PlanExecutor gains execution_context param with has_runtime / changeset_store / execution_context properties and _run_execute_with_runtime / _run_execute_with_stub split. 31 Behave scenarios, 5 Robot smoke tests, ASV benchmark suite, and reference documentation. Ref: Day-14 Rebaseline – M1.2 Plan-execute runtime wiring [Jeff]
222 lines
8.6 KiB
Gherkin
222 lines
8.6 KiB
Gherkin
@phase2 @plan @execute @runtime
|
|
Feature: Plan Execute Runtime Integration
|
|
As a system operator
|
|
I want the plan executor to dispatch through the tool-calling actor runtime
|
|
And capture changesets during execution
|
|
So that plan execution is fully traceable and auditable
|
|
|
|
Background:
|
|
Given a fresh in-memory changeset store
|
|
And a valid plan ID
|
|
|
|
# -- PlanExecutionContext creation --
|
|
|
|
@context
|
|
Scenario: Create PlanExecutionContext with required plan_id
|
|
When I create a PlanExecutionContext with a valid plan_id
|
|
Then the context plan_id should match the provided value
|
|
|
|
@context
|
|
Scenario: Create PlanExecutionContext with all optional fields
|
|
When I create a PlanExecutionContext with all optional fields
|
|
Then the context should have the correct decision_root_id
|
|
And the context should have the correct sandbox_root
|
|
And the context should have the correct automation_profile
|
|
|
|
@context @error
|
|
Scenario: PlanExecutionContext rejects empty plan_id
|
|
When I attempt to create a PlanExecutionContext with empty plan_id
|
|
Then the plan execution context should raise a ValidationError containing "plan_id"
|
|
|
|
@context @error
|
|
Scenario: PlanExecutionContext rejects None plan_id
|
|
When I attempt to create a PlanExecutionContext with None plan_id
|
|
Then the plan execution context should raise a ValidationError containing "plan_id"
|
|
|
|
@context
|
|
Scenario: PlanExecutionContext defaults to InMemoryChangeSetStore
|
|
When I create a PlanExecutionContext without specifying a changeset store
|
|
Then the context changeset_store should be an InMemoryChangeSetStore
|
|
|
|
@context
|
|
Scenario: PlanExecutionContext accepts custom changeset store
|
|
When I create a PlanExecutionContext with a custom changeset store
|
|
Then the context changeset_store should be the provided store
|
|
|
|
@context
|
|
Scenario: PlanExecutionContext defaults project_resources to empty dict
|
|
When I create a PlanExecutionContext with a valid plan_id
|
|
Then the context project_resources should be an empty dict
|
|
|
|
@context
|
|
Scenario: PlanExecutionContext defaults resource_bindings to empty dict
|
|
When I create a PlanExecutionContext with a valid plan_id
|
|
Then the context resource_bindings should be an empty dict
|
|
|
|
# -- Changeset operations --
|
|
|
|
@changeset
|
|
Scenario: start_changeset creates a changeset with ULID
|
|
Given a PlanExecutionContext
|
|
When I call start_changeset
|
|
Then the returned changeset_id should be a valid ULID string
|
|
And the active_changeset_ids should contain the new ID
|
|
|
|
@changeset
|
|
Scenario: record_change persists an entry into the active changeset
|
|
Given a PlanExecutionContext with an active changeset
|
|
When I record a ChangeEntry into the context
|
|
Then the changeset should contain the recorded entry
|
|
|
|
@changeset
|
|
Scenario: record_change without active changeset raises PlanError
|
|
Given a PlanExecutionContext without an active changeset
|
|
When I attempt to record a ChangeEntry
|
|
Then a PlanError should be raised about no active changeset
|
|
|
|
@changeset
|
|
Scenario: Multiple changes recorded into same changeset
|
|
Given a PlanExecutionContext with an active changeset
|
|
When I record three ChangeEntry instances
|
|
Then the changeset should contain exactly three entries
|
|
|
|
@changeset
|
|
Scenario: get_changeset retrieves existing changeset
|
|
Given a PlanExecutionContext with an active changeset and one recorded entry
|
|
When I call get_changeset with the active changeset_id
|
|
Then the returned changeset should not be None
|
|
And the changeset plan_id should match
|
|
|
|
@changeset
|
|
Scenario: get_changeset returns None for unknown ID
|
|
Given a PlanExecutionContext
|
|
When I call get_changeset with a nonexistent ID
|
|
Then the returned changeset should be None
|
|
|
|
# -- Summarize --
|
|
|
|
@summarize
|
|
Scenario: summarize includes all expected fields
|
|
Given a PlanExecutionContext with an active changeset and one recorded entry
|
|
When I call summarize on the context
|
|
Then the summary should contain plan_id
|
|
And the summary should contain resource_binding_count of 0
|
|
And the summary should contain decision_root_id
|
|
|
|
@summarize
|
|
Scenario: summarize with no changesets shows zero resource counts
|
|
Given a PlanExecutionContext
|
|
When I call summarize on the context
|
|
Then the summary resource_binding_count should be 0
|
|
|
|
# -- RuntimeExecuteResult model --
|
|
|
|
@model
|
|
Scenario: RuntimeExecuteResult validates with valid data
|
|
When I create a RuntimeExecuteResult with valid fields
|
|
Then the runtime execute result should have a correct changeset_id
|
|
And the runtime execute result tool_call_count should be non-negative
|
|
|
|
@model @error
|
|
Scenario: RuntimeExecuteResult rejects negative tool_call_count
|
|
When I attempt to create a RuntimeExecuteResult with negative tool_call_count
|
|
Then the runtime result should fail validation
|
|
|
|
@model @error
|
|
Scenario: RuntimeExecuteResult rejects empty changeset_id
|
|
When I attempt to create a RuntimeExecuteResult with empty changeset_id
|
|
Then the runtime result should fail validation
|
|
|
|
# -- RuntimeExecuteActor --
|
|
|
|
@actor
|
|
Scenario: RuntimeExecuteActor executes with stub decisions
|
|
Given a PlanExecutionContext
|
|
And a ToolRunner with an empty registry
|
|
And a RuntimeExecuteActor
|
|
When I execute with a list of two stub decisions
|
|
Then the runtime execute result should be a RuntimeExecuteResult
|
|
And the runtime execute result changeset_id should be a valid ULID
|
|
And the runtime execute result decision_ids_processed should have two entries
|
|
|
|
@actor
|
|
Scenario: RuntimeExecuteActor with streaming callback
|
|
Given a PlanExecutionContext
|
|
And a ToolRunner with an empty registry
|
|
And a RuntimeExecuteActor
|
|
And a stream callback collector
|
|
When I execute with one stub decision and the stream callback
|
|
Then the callback should have received runtime_execute_started event
|
|
And the callback should have received runtime_execute_step event
|
|
And the callback should have received runtime_execute_complete event
|
|
|
|
@actor
|
|
Scenario: RuntimeExecuteActor records sandbox_refs when sandbox_root is set
|
|
Given a PlanExecutionContext with sandbox_root "/tmp/sandbox"
|
|
And a ToolRunner with an empty registry
|
|
And a RuntimeExecuteActor
|
|
When I execute with one stub decision
|
|
Then the runtime execute result sandbox_refs should contain "/tmp/sandbox"
|
|
|
|
@actor @error
|
|
Scenario: RuntimeExecuteActor rejects None tool_runner
|
|
Given a PlanExecutionContext
|
|
When I attempt to create a RuntimeExecuteActor with None tool_runner
|
|
Then the plan execution context should raise a ValidationError
|
|
|
|
@actor @error
|
|
Scenario: RuntimeExecuteActor rejects None execution_context
|
|
Given a ToolRunner with an empty registry
|
|
When I attempt to create a RuntimeExecuteActor with None execution_context
|
|
Then the plan execution context should raise a ValidationError
|
|
|
|
# -- PlanExecutor integration --
|
|
|
|
@executor
|
|
Scenario: PlanExecutor has_runtime is True with tool_calling_runtime
|
|
Given a mock lifecycle service
|
|
And a PlanExecutionContext
|
|
And a ToolRunner with an empty registry
|
|
When I create a PlanExecutor with a mock tool_calling_runtime
|
|
Then has_runtime should be True
|
|
|
|
@executor
|
|
Scenario: PlanExecutor has_runtime is False without execution_context
|
|
Given a mock lifecycle service
|
|
When I create a PlanExecutor without execution_context
|
|
Then has_runtime should be False
|
|
|
|
@executor
|
|
Scenario: PlanExecutor changeset_store returns store from context
|
|
Given a mock lifecycle service
|
|
And a PlanExecutionContext with a known changeset store
|
|
When I create a PlanExecutor with the execution_context
|
|
Then changeset_store should be the same object as the context store
|
|
|
|
@executor
|
|
Scenario: PlanExecutor changeset_store returns None without context
|
|
Given a mock lifecycle service
|
|
When I create a PlanExecutor without execution_context
|
|
Then changeset_store should be None
|
|
|
|
@executor
|
|
Scenario: PlanExecutor execution_context property returns context
|
|
Given a mock lifecycle service
|
|
And a PlanExecutionContext
|
|
When I create a PlanExecutor with the execution_context
|
|
Then execution_context property should return the context
|
|
|
|
# -- Error handling --
|
|
|
|
@error @executor
|
|
Scenario: PlanExecutor rejects None lifecycle_service
|
|
When I attempt to create a PlanExecutor with None lifecycle_service
|
|
Then the plan execution context should raise a ValidationError containing "lifecycle_service"
|
|
|
|
@error @executor
|
|
Scenario: PlanExecutor run_execute rejects empty plan_id
|
|
Given a mock lifecycle service
|
|
And a PlanExecutor without execution_context
|
|
When I attempt to run execute with an empty plan_id
|
|
Then the plan execution context should raise a ValidationError containing "plan_id"
|