test(e2e): workflow example 7 — CI/CD integration, automated PR review and fix (ci profile) #804

Merged
CoreRasurae merged 3 commits from test/e2e-wf07-cicd into master 2026-03-26 18:26:12 +00:00
9 changed files with 621 additions and 18 deletions
+69 -1
View File
@@ -262,6 +262,38 @@
and auto-discovery. Updated `git-checkout` child types. Includes YAML
configs, Behave BDD tests, Robot tests, and ASV benchmarks. (#330)
- Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)
- Modified `auto_progress()` to complete the Apply phase immediately after
transitioning from Execute to Apply, since Apply is a metadata transition
with no LLM processing. This ensures `plan execute` drives the plan to
the terminal `applied` state when the automation profile permits (ci,
full-auto profiles with `auto_apply < 1.0`).
Extracted `_complete_apply_if_queued()` helper that consolidates the
Apply-completion pattern (start_apply + complete_apply) into a single
method with error recovery (calls `fail_apply` on failure) and async-job
guard (skips inline completion when async execution is enabled to avoid
orphaning enqueued jobs). Used by `auto_progress()`,
`lifecycle_apply_plan()`, and `try_auto_run()`.
Added `PlanLifecycleService.try_auto_run()` that drives plans through all
lifecycle phases (Strategize → Execute → Apply) when automation-profile
thresholds allow automatic progression; a threshold of 1.0 stops the plan
at that phase boundary for human approval.
Fixed `lifecycle-apply` CLI leaving plans stuck in `apply/queued` without
completing. The command now calls `_complete_apply_if_queued()` when the
plan is in Apply/queued, driving it to the terminal `applied` state.
Fixed stale RICH output in `lifecycle_apply_plan` that printed
"Plan is now in Apply phase (queued)" after the plan had already reached
terminal `applied` state; now branches on `plan.is_terminal`.
Fixed SQLite UNIQUE constraint violation in
`LifecyclePlanRepository.update()`: added `session.flush()` after
`clear()` on child collections (project_links, arguments, invariants)
before re-inserting rows.
Added `state` alias in `_plan_spec_dict()` JSON output for spec §Example 7
`jq` compatibility.
Updated plan execute and lifecycle-apply reference documentation.
(`src/cleveragents/application/services/plan_lifecycle_service.py`,
`src/cleveragents/cli/commands/plan.py`,
`src/cleveragents/infrastructure/database/repositories.py`,
`docs/reference/plan_cli.md`) (#753)
- Fixed `plan execute` CLI failing with "Plan is not in an executable state
(current: strategize/queued)" after strategize completed successfully.
Root cause: `_get_plan_executor()` created a second `PlanLifecycleService`
@@ -418,8 +450,44 @@
under the 500-line limit. `Verify Plan In List` and `Full Flow Apply Step`
keywords use hard assertions instead of WARN fallbacks. Profile Precedence
test documents that action > global precedence requires production wiring
not yet present in `PlanLifecycleService.use_action`.
not yet present in `PlanLifecycleService.use_action`.
(`robot/e2e/m6_acceptance.robot`, `robot/e2e/common_e2e.resource`) (#746)
- Added E2E Robot Framework test for Specification Workflow Example 7: CI/CD
Integration — Automated PR Review and Fix. Exercises the `ci` automation
profile (headless, non-interactive) with JSON output and log-level
configuration, idempotent resource and project registration with `--branch`
and `--description` flags, three-validation registration (source/mode/code)
with project attachment and `project show` verification, action creation
with spec-aligned name (`local/review-pr`), complete `definition_of_done`,
`invariants`, and `arguments`, plan launch with `--arg` flags, explicit
`plan execute` for lifecycle progression, `plan status` terminal-state
assertion, plan diff JSON validation, and JSON output verification.
Resource/project naming follows spec convention (`local/ci-workspace` project,
`local/ci-main` resource). Entity creation commands tolerate "already
exists" for CI re-runnability. `Extract JSON Field` keyword handles CLI
debug log lines preceding JSON via `JSONDecoder.raw_decode(strict=False)`.
Fail-fast `expected_rc` only where the spec mandates error suppression
(`2>/dev/null || true`); first `resource add` and `project create` now
assert `expected_rc=${0}`. Config assertions use stdout-only matching and
exact equality for the `ci` profile value. Project idempotency verified
with occurrence count. Empty plan-diff stdout logged as warning.
Validation naming aligned with spec (`local/ci-lint` per §Example 7).
All `Run Process` calls include `on_timeout=kill` per codebase CI
stability standard. Added `on_timeout=kill` to `Run CleverAgents
Command` and `Create Temp Git Repo` keywords in `common_e2e.resource`
for consistent timeout handling across all E2E suites.
Dynamic actor selection based on available API keys (same pattern as
`m6_acceptance.robot`) avoids runtime failure when only one provider
key is set. `Poll Plan Until Terminal` keyword now integrated into
the CI Plan Launch test case per spec Step 3 polling loop. Replaced
local `Extract JSON Field` with shared `Safe Parse Json Field` from
`common_e2e.resource`. Added `Force Tags E2E` and per-test
`[Teardown]` blocks. Added `WF07 Suite Setup` keyword for database
initialisation.
Added `robot/common_vars.py` module placeholder for shared Robot
Framework variables.
(`robot/e2e/wf07_cicd.robot`, `robot/e2e/common_e2e.resource`,
`robot/common_vars.py`) (#753)
- Fixed `agents session list`, `agents session create`, and other session
subcommands raising `AttributeError: 'DynamicContainer' object has no
attribute 'db'` after `agents init`. Root cause: `_get_session_service()`
+14 -3
View File
@@ -9,8 +9,8 @@ The `agents plan` command group manages plans in the CleverAgents v3 plan lifecy
| `agents plan use` | Create plan from action + project(s) |
| `agents plan lifecycle-list` | List plans with optional filters |
| `agents plan status` | Show plan status / details |
| `agents plan execute` | Transition to Execute phase |
| `agents plan lifecycle-apply` | Transition to Apply phase |
| `agents plan execute` | Run Strategize + Execute (auto-apply if profile permits) |
| `agents plan lifecycle-apply` | Transition to Apply and complete it |
| `agents plan cancel` | Cancel a non-terminal plan |
| `agents plan diff` | Show ChangeSet as unified diff |
| `agents plan artifacts` | Show ChangeSet ID, sandbox refs, summary|
@@ -131,6 +131,12 @@ phase and processes it inline:
- **Strategize/complete** — transitions to Execute and runs it.
- **Execute/queued** — runs the execute phase to completion.
When the automation profile's `auto_apply` threshold is met (< 1.0),
the execute command also drives the plan through the Apply phase to
the terminal `applied` state. This means `plan execute` with a
`ci` or `full-auto` profile completes the full lifecycle in a single
invocation: Strategize → Execute → Apply.
When no plan ID is given, auto-selects the single eligible plan.
```bash
@@ -147,11 +153,16 @@ for details.
## `agents plan lifecycle-apply`
Transition a plan from Execute to Apply phase. Because Apply is a
Transition a plan to Apply phase and complete it. Because Apply is a
destructive operation (it merges sandbox changesets into real project
resources), a confirmation prompt is displayed by default. Pass
`--yes` / `-y` to skip the prompt in scripts or CI pipelines.
When the plan is in Execute/complete, transitions to Apply. When the
plan is in Apply/queued (e.g. auto-progressed by `plan execute`),
completes the apply processing, driving the plan to the terminal
`applied` state. Apply is a metadata transition (no LLM call).
### Synopsis
```bash
+1 -1
View File
@@ -46,7 +46,7 @@ Feature: Automation Levels
Given I have a plan with automation level "full_automation" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "apply"
And the automated plan processing state should be "queued"
And the automated plan processing state should be "applied"
# Plan-Level Override Tests
@@ -293,13 +293,32 @@ def step_service_has_strategize_plan(context) -> None:
@given("the service has a complete execute plan for apply")
def step_service_has_execute_plan(context) -> None:
plan = _make_plan(
pre_plan = _make_plan(
plan_id=_ULIDS[1],
name="local/apply-plan",
phase=PlanPhase.EXECUTE,
processing_state=ProcessingState.COMPLETE,
)
context.mock_lifecycle_service.apply_plan.return_value = plan
apply_queued_plan = _make_plan(
plan_id=_ULIDS[1],
name="local/apply-plan",
phase=PlanPhase.APPLY,
processing_state=ProcessingState.QUEUED,
)
applied_plan = _make_plan(
plan_id=_ULIDS[1],
name="local/apply-plan",
phase=PlanPhase.APPLY,
processing_state=ProcessingState.APPLIED,
)
# lifecycle_apply_plan calls get_plan twice: first to check read_only
# and phase, then again after apply_plan to drive _complete_apply_if_queued.
context.mock_lifecycle_service.get_plan.side_effect = [
pre_plan,
apply_queued_plan,
]
context.mock_lifecycle_service.apply_plan.return_value = apply_queued_plan
context.mock_lifecycle_service._complete_apply_if_queued.return_value = applied_plan
context._apply_plan_id = _ULIDS[1]
+1
View File
@@ -0,0 +1 @@
"""Common Robot Framework variables shared across test suites."""
+344
View File
@@ -0,0 +1,344 @@
*** Settings ***
Documentation E2E test for Workflow Example 7: CI/CD integration,
... automated PR review and fix (ci profile).
...
... Exercises the real CleverAgents CLI with zero mocking,
... covering ci-profile configuration, idempotent resource and
... project registration, validation registration with project
... attachment, CI plan launch with action args and JSON output,
... polling for plan completion, and JSON output verification.
...
... Tests form an implicit sequential pipeline via shared
... database state: each test creates CLI entities consumed
... by later tests. Execution order is file order.
Resource common_e2e.resource
Suite Setup WF07 Suite Setup
Suite Teardown E2E Suite Teardown
Force Tags E2E
*** Variables ***
${CI_PROJECT} local/ci-workspace
${CI_RESOURCE} local/ci-main
${CI_LINT_VAL} local/ci-lint
${CI_TYPECHECK_VAL} local/ci-typecheck
${CI_TESTS_VAL} local/ci-tests
*** Test Cases ***
WF07 E2E CI Profile Configuration
[Documentation] Set and verify the ci automation profile, json format,
... and log level per specification Step 1.
[Tags] E2E
[Teardown] Log CI Profile Configuration teardown complete
# Initialize the CleverAgents data directory and database
# expected_rc=None: init may report the CWD is already initialized; this
# is harmless because CLEVERAGENTS_HOME is set to a fresh temp dir by
# Suite Setup.
Run CleverAgents Command init --yes expected_rc=None
# Automation profile
Run CleverAgents Command config set core.automation-profile ci
${get_profile}= Run CleverAgents Command config get core.automation-profile --format plain
# Match the value line in plain-text output (key/value/source/type block).
# Regex avoids false positives from debug lines containing "ci" substrings.
Should Match Regexp ${get_profile.stdout} (?m)^value:\\s+ci\\s*$
... Expected value line 'value: ci' in config output: ${get_profile.stdout}
# Output format
Run CleverAgents Command config set core.format json
${get_format}= Run CleverAgents Command config get core.format --format plain
Should Match Regexp ${get_format.stdout} (?m)^value:\\s+json\\s*$
... Expected value line 'value: json' in config output: ${get_format.stdout}
# Log level (spec Step 1)
Run CleverAgents Command config set core.log.level WARN
${get_log}= Run CleverAgents Command config get core.log.level --format plain
Should Match Regexp ${get_log.stdout} (?m)^value:\\s+WARN\\s*$
... Expected value line 'value: WARN' in config output: ${get_log.stdout}
WF07 E2E Idempotent Resource Registration
[Documentation] Register a git-checkout resource twice with --path and
... --branch, and verify it appears in the resource list.
[Tags] E2E
[Teardown] Log Idempotent Resource Registration teardown complete
${repo_dir}= Create Temp Git Repo With Issues ci-repo
# First registration (--branch main per spec Step 3).
Run CleverAgents Command resource add git-checkout ${CI_RESOURCE}
... --path ${repo_dir} --branch main expected_rc=${0}
# Second registration (idempotent — may succeed or fail gracefully)
Run CleverAgents Command resource add git-checkout ${CI_RESOURCE}
... --path ${repo_dir} --branch main expected_rc=None
# Verify it appears in the list
${list_out}= Run CleverAgents Command resource list --format plain
Output Should Contain ${list_out} ci-main
# Verify exactly one occurrence (idempotency: two adds must not duplicate)
${count}= Get Count ${list_out.stdout} ci-main
Should Be True ${count} == 1 Resource ci-main should appear exactly once (idempotency check)
WF07 E2E Idempotent Project Registration
[Documentation] Create a project linked to the resource twice and verify
... it appears in the project list (idempotent).
[Tags] E2E
[Teardown] Log Idempotent Project Registration teardown complete
# First creation with resource link and description (per spec Step 3)
Run CleverAgents Command project create ${CI_PROJECT}
... --description CI workspace for automated PR review
... --resource ${CI_RESOURCE} expected_rc=${0}
# Second creation (idempotent — may succeed or fail gracefully)
Run CleverAgents Command project create ${CI_PROJECT}
... --description CI workspace for automated PR review
... --resource ${CI_RESOURCE} expected_rc=None
${list_out}= Run CleverAgents Command project list --format plain
Output Should Contain ${list_out} ci-workspace
# Verify exactly one occurrence (idempotency: two creates must not duplicate).
# Use the full namespaced name because plain-text output includes both
# namespaced_name and name fields per entry — bare "ci-workspace" matches twice.
${count}= Get Count ${list_out.stdout} ${CI_PROJECT}
Should Be True ${count} == 1 Project ${CI_PROJECT} should appear exactly once (idempotency check)
WF07 E2E Validation Registration
[Documentation] Register lint, typecheck, and test validations, attach
... them to the project, and verify they appear in the tool
... list and project show output.
[Tags] E2E
[Teardown] Log Validation Registration teardown complete
# --- Lint validation ---
${lint_yaml}= Catenate SEPARATOR=\n
... name: ${CI_LINT_VAL}
... description: Lint check validation for CI
... source: custom
... mode: required
... code: |
... ${SPACE}${SPACE}def run(inputs):
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}return {"passed": True, "message": "lint check passed"}
${lint_path}= Set Variable ${SUITE_HOME}${/}ci-lint.yaml
Create File ${lint_path} ${lint_yaml}
Run CleverAgents Command validation add --config ${lint_path} expected_rc=None
# --- Typecheck validation ---
${type_yaml}= Catenate SEPARATOR=\n
... name: ${CI_TYPECHECK_VAL}
... description: Type check validation for CI
... source: custom
... mode: required
... code: |
... ${SPACE}${SPACE}def run(inputs):
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}return {"passed": True, "message": "typecheck passed"}
${type_path}= Set Variable ${SUITE_HOME}${/}ci-typecheck.yaml
Create File ${type_path} ${type_yaml}
Run CleverAgents Command validation add --config ${type_path} expected_rc=None
# --- Test validation ---
${test_yaml}= Catenate SEPARATOR=\n
... name: ${CI_TESTS_VAL}
... description: Test runner validation for CI
... source: custom
... mode: required
... code: |
... ${SPACE}${SPACE}def run(inputs):
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}return {"passed": True, "message": "tests passed"}
${test_path}= Set Variable ${SUITE_HOME}${/}ci-tests.yaml
Create File ${test_path} ${test_yaml}
Run CleverAgents Command validation add --config ${test_path} expected_rc=None
# --- Attach validations to project (spec Step 3) ---
Run CleverAgents Command validation attach
... --project ${CI_PROJECT} ${CI_RESOURCE} ${CI_LINT_VAL} expected_rc=None
Run CleverAgents Command validation attach
... --project ${CI_PROJECT} ${CI_RESOURCE} ${CI_TYPECHECK_VAL} expected_rc=None
Run CleverAgents Command validation attach
... --project ${CI_PROJECT} ${CI_RESOURCE} ${CI_TESTS_VAL} expected_rc=None
# Verify validations appear in tool list
${list_out}= Run CleverAgents Command tool list --type validation --format plain
Output Should Contain ${list_out} ci-lint
Output Should Contain ${list_out} ci-typecheck
Output Should Contain ${list_out} ci-tests
# Verify the project itself is accessible (project show does not display
# attached validations — validation registration is verified by tool list above)
${project_out}= Run CleverAgents Command project show ${CI_PROJECT}
... --format plain
Log Project show output: ${project_out.stdout}
Output Should Contain ${project_out} ci-workspace
WF07 E2E CI Plan Launch
[Documentation] Create an action and launch a plan with --automation-profile ci,
... --format json, and --arg flags per specification Steps 2-3.
... Polls for plan completion. Requires LLM API keys.
[Tags] E2E
[Teardown] Log CI Plan Launch teardown complete
Skip If No LLM Keys
# Dynamically select actor based on available API keys.
# Prefer OpenAI first to reduce Anthropic credit-quota flakiness.
${has_openai}= Evaluate bool(__import__('os').environ.get('OPENAI_API_KEY', ''))
${has_anthropic}= Evaluate bool(__import__('os').environ.get('ANTHROPIC_API_KEY', ''))
IF ${has_openai}
${actor}= Set Variable openai/gpt-4o
ELSE IF ${has_anthropic}
${actor}= Set Variable anthropic/claude-sonnet-4-20250514
ELSE
${actor}= Set Variable openai/gpt-4o
END
# Create action with spec-aligned fields and dynamic actor
${action_yaml}= Catenate SEPARATOR=\n
... name: local/review-pr
... description: Automatically review a PR and fix issues
... strategy_actor: ${actor}
... execution_actor: ${actor}
... automation_profile: ci
... reusable: true
... state: available
... definition_of_done: |
... ${SPACE}${SPACE}- All lint issues are resolved
... ${SPACE}${SPACE}- Type checking passes
... ${SPACE}${SPACE}- Test coverage does not decrease
... ${SPACE}${SPACE}- Security scan passes
... ${SPACE}${SPACE}- All fixes are committed to the PR branch
... arguments:
... ${SPACE}${SPACE}- name: pr_branch
... ${SPACE}${SPACE}${SPACE}${SPACE}type: string
... ${SPACE}${SPACE}${SPACE}${SPACE}required: true
... ${SPACE}${SPACE}${SPACE}${SPACE}description: Branch name of the PR
... ${SPACE}${SPACE}- name: base_branch
... ${SPACE}${SPACE}${SPACE}${SPACE}type: string
... ${SPACE}${SPACE}${SPACE}${SPACE}required: false
... ${SPACE}${SPACE}${SPACE}${SPACE}default: main
... ${SPACE}${SPACE}${SPACE}${SPACE}description: Base branch to compare against
... invariants:
... ${SPACE}${SPACE}- "Only modify files that are already changed in the PR"
... ${SPACE}${SPACE}- "Do not change the intent of any code — only fix style, types, and test issues"
... ${SPACE}${SPACE}- "All fixes must include a comment explaining what was changed and why"
${action_path}= Set Variable ${SUITE_HOME}${/}review-pr-action.yaml
Create File ${action_path} ${action_yaml}
# expected_rc=None: action may already exist if Suite Setup cleanup failed
# and a prior E2E run left stale state; CI re-runnability requires tolerance.
Run CleverAgents Command action create --config ${action_path} expected_rc=None
# Launch plan with --arg flags per spec Step 3
${plan_result}= Run CleverAgents Command
... plan use local/review-pr ${CI_PROJECT}
... --automation-profile ci
... --format json
... --arg pr_branch=main
... --arg base_branch=main
... expected_rc=None timeout=180s
# Parse and validate JSON output
${stdout}= Set Variable ${plan_result.stdout}
${plan_id}= Safe Parse Json Field ${stdout} plan_id
IF '${plan_id}' != '${EMPTY}'
Log Plan launched with ID: ${plan_id}
# Drive the plan through its lifecycle. Per the specification
# (§Automation Profiles), ``plan use`` returns the plan in
# strategize/queued phase. ``plan execute`` drives the plan
# through Strategize → Execute, and auto_progress completes
# Apply when the ci profile (auto_apply=0.0) permits it.
${exec_result}= Run CleverAgents Command plan execute ${plan_id}
... --format json expected_rc=None timeout=180s
Should Be Equal As Integers ${exec_result.rc} 0
... plan execute failed (rc=${exec_result.rc}): ${exec_result.stderr}
# Poll plan status until it reaches a terminal state (spec Step 3
# polling loop).
${terminal_state}= Poll Plan Until Terminal ${plan_id}
Log Plan reached state: ${terminal_state}
Should Be Equal As Strings ${terminal_state} applied
... Plan did not reach 'applied' state. Final state: ${terminal_state}
# If applied, verify plan diff is parseable JSON (spec Step 3 final command)
${diff_result}= Run CleverAgents Command plan diff ${plan_id}
... --format json expected_rc=None
Log Plan diff output: ${diff_result.stdout}
# Validate the diff output is parseable JSON. The CLI may emit
# debug/log lines before the JSON payload, so we use raw_decode
# to locate the first valid JSON object.
IF len($diff_result.stdout) > 0
TRY
${parsed_diff}= Evaluate json.loads($diff_result.stdout) json
EXCEPT
TRY
${idx}= Evaluate $diff_result.stdout.index('{')
${parsed_diff}= Evaluate json.JSONDecoder().raw_decode($diff_result.stdout, $idx)[0] json
EXCEPT
Log Plan diff stdout is not valid JSON; may contain only log lines. WARN
${parsed_diff}= Set Variable ${NONE}
END
END
IF $parsed_diff is not None
Should Be True isinstance($parsed_diff, (dict, list))
... Expected JSON dict or list from plan diff
END
ELSE
Log Plan diff returned empty stdout; plan may have applied with no changes. WARN
END
ELSE
# Fail meaningfully when plan_id extraction fails
Fail Plan use command did not return a valid plan_id. rc=${plan_result.rc} stdout: ${plan_result.stdout} stderr: ${plan_result.stderr}
END
WF07 E2E JSON Output Verification
[Documentation] Verify that --format json produces valid, parseable JSON
... output from the version command.
[Tags] E2E
[Teardown] Log JSON Output Verification teardown complete
${result}= Run CleverAgents Command version --format json
Should Not Be Empty ${result.stdout}
# Parse as JSON and verify structure
${stdout}= Set Variable ${result.stdout}
${parsed}= Evaluate json.loads($stdout) json
Should Be True isinstance($parsed, dict)
... Expected JSON dict from version command, got: ${result.stdout}
Output Should Contain ${result} version
*** Keywords ***
WF07 Suite Setup
[Documentation] E2E Suite Setup plus database initialisation.
E2E Suite Setup
# Initialise the database so config/resource/project commands work.
${init}= Run CleverAgents Command init --force --yes
Should Be Equal As Integers ${init.rc} 0
Create Temp Git Repo With Issues
[Documentation] Create a temporary git repo containing Python files with
Outdated
Review

P2 — Duplicate keyword: Extract JSON Field duplicates the functionality of Safe Parse Json Field already defined in common_e2e.resource. Both solve the same problem (extracting a JSON field from CLI output that may have log lines before JSON) with different implementations. Consolidate into one keyword in common_e2e.resource to avoid divergent maintenance.

**P2 — Duplicate keyword**: `Extract JSON Field` duplicates the functionality of `Safe Parse Json Field` already defined in `common_e2e.resource`. Both solve the same problem (extracting a JSON field from CLI output that may have log lines before JSON) with different implementations. Consolidate into one keyword in `common_e2e.resource` to avoid divergent maintenance.
... lint and type issues for the CI review scenario.
... Returns the path to the created repository.
[Arguments] ${name}=test-repo
${repo_dir}= Create Temp Git Repo ${name}
# Ensure the branch is named 'main' for spec consistency
${rename_result}= Run Process git branch -M main cwd=${repo_dir}
... timeout=30s on_timeout=kill
Should Be Equal As Integers ${rename_result.rc} 0
... Failed to rename branch to main: ${rename_result.stderr}
# Add Python file with lint issues (unused imports, missing type hints,
# unused variable) and type errors to simulate a PR needing review.
${py_content}= Catenate SEPARATOR=\n
... import os
Outdated
Review

P2 — Dead code: Poll Plan Until Terminal keyword is defined but never called from any test case. The tests use synchronous plan execute + plan status instead. Remove or use it.

**P2 — Dead code**: `Poll Plan Until Terminal` keyword is defined but never called from any test case. The tests use synchronous `plan execute` + `plan status` instead. Remove or use it.
... import sys
... ${EMPTY}
... ${EMPTY}
... def greet(name: str) -> int:
... ${SPACE}${SPACE}${SPACE}${SPACE}msg = "Hello, " + name
... ${SPACE}${SPACE}${SPACE}${SPACE}return msg
... ${EMPTY}
... ${EMPTY}
... def unused_function():
... ${SPACE}${SPACE}${SPACE}${SPACE}x = 42
... ${SPACE}${SPACE}${SPACE}${SPACE}pass
Create File ${repo_dir}${/}app.py ${py_content}
# Verify git operations succeed
${add_result}= Run Process git add . cwd=${repo_dir}
... timeout=30s on_timeout=kill
Should Be Equal As Integers ${add_result.rc} 0
... git add failed: ${add_result.stderr}
${commit_result}= Run Process git commit -m Add application with lint issues cwd=${repo_dir}
... timeout=30s on_timeout=kill
Should Be Equal As Integers ${commit_result.rc} 0
... git commit failed: ${commit_result.stderr}
RETURN ${repo_dir}
Poll Plan Until Terminal
[Documentation] Poll plan status until it reaches a terminal state.
... Polling exit states (not all are terminal in the domain
... model — ``errored`` is recoverable via ``resume_plan``
... but will not auto-advance, so polling should exit):
... applied, constrained, errored, cancelled.
... Returns the state string, or 'timeout'.
[Arguments] ${plan_id} ${max_polls}=18 ${interval}=10s
FOR ${i} IN RANGE ${max_polls}
${result}= Run CleverAgents Command plan status ${plan_id}
... --format json expected_rc=None
${state}= Safe Parse Json Field ${result.stdout} state
IF '${state}' in ['applied', 'constrained', 'cancelled', 'errored']
RETURN ${state}
END
Sleep ${interval}
END
RETURN timeout
@@ -390,7 +390,8 @@ class PlanLifecycleService:
Args:
plan_id: The plan ULID.
phase: The plan phase (``"execute"`` or ``"apply"``).
phase: The plan phase (``"strategize"``, ``"execute"``,
or ``"apply"``).
Returns:
The created ``AsyncJob`` if enqueued, or ``None`` if async
2
@@ -1528,6 +1529,54 @@ class PlanLifecycleService:
and profile.auto_apply < 1.0
)
def _complete_apply_if_queued(self, plan_id: str) -> Plan:
"""Drive a plan from Apply/QUEUED to terminal APPLIED.
Apply is a metadata-only transition (no LLM call), so it is
safe to complete synchronously. If the plan is not in
Apply/QUEUED, returns it unchanged.
When async execution is enabled (server mode with a job store),
the Apply phase is handled by the async worker; this method
skips inline completion to avoid orphaning the queued async job.
On failure the plan is moved to ``errored`` via
:meth:`fail_apply` so it is never stuck in ``apply/processing``
with no recovery path.
Args:
plan_id: The plan ULID.
Returns:
The (possibly updated) Plan.
"""
plan = self.get_plan(plan_id)
if plan.phase != PlanPhase.APPLY:
return plan
if plan.processing_state != ProcessingState.QUEUED:
return plan
# When async execution is enabled, an async job was already
# enqueued by apply_plan(); let the async worker handle it.
if self.settings.async_enabled and self._job_store is not None:
self._logger.info(
"Skipping inline Apply completion — async job enqueued",
plan_id=plan_id,
)
return plan
try:
self.start_apply(plan_id)
return self.complete_apply(plan_id)
except Exception as exc:
self._logger.error(
"Apply completion failed; recording error state",
plan_id=plan_id,
error=str(exc),
)
return self.fail_apply(plan_id, str(exc))
def auto_progress(self, plan_id: str) -> Plan:
"""If the plan's automation level permits, advance to the next phase.
@@ -1563,7 +1612,92 @@ class PlanLifecycleService:
"Auto-progressing plan from Execute to Apply",
plan_id=plan_id,
)
return self.apply_plan(plan_id)
plan = self.apply_plan(plan_id)
# Apply phase is a metadata transition (no LLM call), so
# complete it immediately when auto-progressing. This
# ensures ``plan execute`` drives the plan to terminal
# ``applied`` state when the automation profile permits
# (spec §Automation Profiles: auto_apply < 1.0).
return self._complete_apply_if_queued(plan_id)
return plan
def try_auto_run(self, plan_id: str) -> Plan:
"""Drive the plan through its lifecycle automatically.
When the plan's automation profile permits, this method advances
the plan synchronously through Strategize Execute Apply:
* ``auto_strategize < 1.0`` start + complete Strategize
* ``auto_execute < 1.0`` start + complete Execute
* ``auto_apply < 1.0`` start + complete Apply
Each completion step calls :meth:`auto_progress` internally,
which transitions the plan to the next phase when appropriate.
If any threshold is ``1.0`` (human approval required), the plan
stops at that phase boundary. The method is idempotent: calling
it on a plan already in a terminal state simply returns the plan.
Args:
plan_id: The plan ULID.
Returns:
The (possibly advanced) Plan.
Raises:
NotFoundError: If plan not found.
"""
plan = self.get_plan(plan_id)
if plan.is_terminal:
return plan
profile = self._resolve_profile_for_plan(plan)
# --- Strategize phase: QUEUED → PROCESSING → COMPLETE -----------
if (
plan.phase == PlanPhase.STRATEGIZE
and plan.state == ProcessingState.QUEUED
and profile.auto_strategize < 1.0
):
self._logger.info(
"Auto-running strategize phase",
plan_id=plan_id,
)
self.start_strategize(plan_id)
# complete_strategize() calls auto_progress() which may
# transition the plan to Execute/QUEUED.
plan = self.complete_strategize(plan_id)
plan = self.get_plan(plan_id)
# --- Execute phase: QUEUED → PROCESSING → COMPLETE --------------
if (
plan.phase == PlanPhase.EXECUTE
and plan.state == ProcessingState.QUEUED
and profile.auto_execute < 1.0
):
self._logger.info(
"Auto-running execute phase",
plan_id=plan_id,
)
self.start_execute(plan_id)
# complete_execute() calls auto_progress() which may
# transition the plan to Apply/QUEUED.
plan = self.complete_execute(plan_id)
plan = self.get_plan(plan_id)
# --- Apply phase: QUEUED → PROCESSING → APPLIED (terminal) ------
if (
plan.phase == PlanPhase.APPLY
and plan.state == ProcessingState.QUEUED
and profile.auto_apply < 1.0
):
self._logger.info(
"Auto-running apply phase",
plan_id=plan_id,
)
plan = self._complete_apply_if_queued(plan_id)
return plan
+35 -8
View File
@@ -151,6 +151,9 @@ def _plan_spec_dict(plan: Any) -> dict[str, object]:
"namespaced_name": str(plan.namespaced_name),
"phase": plan.phase.value,
"processing_state": plan.processing_state.value,
# Spec §Example 7 uses ``jq -r '.state'`` — provide a
# convenience alias so both field names work.
"state": plan.processing_state.value,
"project_links": [
{
"project_name": link.project_name,
1
@@ -1782,10 +1785,16 @@ def use_action(
console.print(format_output(data, fmt))
else:
_print_lifecycle_plan(plan, title="Plan Created")
console.print(
"\n[dim]Plan is now in Strategize phase (queued). "
"Run 'agents plan execute <id>' when ready.[/dim]"
)
if plan.is_terminal:
console.print(
f"\n[dim]Plan completed with state: "
f"{plan.state.value if plan.state else 'unknown'}.[/dim]"
)
else:
console.print(
"\n[dim]Plan is now in Strategize phase (queued). "
"Run 'agents plan execute <id>' when ready.[/dim]"
)
except ActionNotAvailableError as e:
console.print(f"[red]Action not available:[/red] {e}")
@@ -2091,15 +2100,33 @@ def lifecycle_apply_plan(
# Normal path: plan is in Execute/complete → transition
plan = service.apply_plan(plan_id)
# Apply phase is a metadata transition (no LLM call). When the
# plan is in Apply/queued, complete it to the terminal ``applied``
# state. This intentionally bypasses the automation profile's
# ``auto_apply`` threshold: explicitly running ``lifecycle-apply``
# IS the human approval action for the manual profile.
plan = service.get_plan(plan_id)
if (
plan.phase == PlanPhase.APPLY
and plan.processing_state == ProcessingState.QUEUED
):
plan = service._complete_apply_if_queued(plan_id)
if fmt != OutputFormat.RICH.value:
data = _plan_spec_dict(plan)
console.print(format_output(data, fmt))
else:
_print_lifecycle_plan(plan, title="Plan Applying")
console.print(
"\n[dim]Plan is now in Apply phase (queued). "
"Changes will be applied to the project(s).[/dim]"
)
if plan.is_terminal:
console.print(
f"\n[dim]Plan completed in Apply phase with state: "
f"{plan.state.value if plan.state else 'unknown'}.[/dim]"
)
else:
console.print(
"\n[dim]Plan is now in Apply phase (queued). "
"Changes will be applied to the project(s).[/dim]"
)
except InvalidPhaseTransitionError as e:
console.print(f"[red]Invalid transition:[/red] {e}")
@@ -1404,7 +1404,6 @@ class LifecyclePlanRepository:
row.arguments_rel.clear() # type: ignore[union-attr]
row.invariants_rel.clear() # type: ignore[union-attr]
session.flush()
now_iso = plan.timestamps.updated_at.isoformat()
for pl in getattr(plan, "project_links", []) or []:
row.project_links_rel.append( # type: ignore[union-attr]