start_strategize() built its action_registry from the in-memory _actions
dict only, so fresh CLI processes (e.g. `plan execute` after a separate
`plan use`) failed with PreflightRejection: "Action not found in
registry". The PreflightRejection (extending bare Exception, not
CleverAgentsError) escaped the CLI error handler, producing an opaque
"Error [500] INTERNAL: An unexpected error occurred" message.
Additionally, `plan execute` only transitioned the plan to
execute/queued without running the execute phase, leaving the plan stuck
and making `lifecycle-apply` fail with PlanNotReadyError.
Changes:
- start_strategize() loads the plan's action from the persistence layer
(via get_action()) before building the preflight action_registry.
- plan execute CLI catches PreflightRejection for user-friendly errors.
- plan execute CLI runs the execute phase inline via PlanExecutor so
the plan progresses through execute/queued -> execute/complete.
- lifecycle-apply CLI handles plans already auto-progressed to
apply/queued by complete_execute()'s auto_progress() call.
ISSUES CLOSED: #746
Implemented Robot Framework E2E test suite for M6 autonomy hardening
acceptance criteria. Tests exercise the real CleverAgents CLI with zero
mocking, covering session lifecycle, automation profiles, project setup,
plan lifecycle via A2A facade, guard enforcement, and a full autonomy
acceptance flow. LLM-dependent tests use Skip If No LLM Keys for
graceful degradation when API keys are unavailable.
Hardened shared E2E keywords (common_e2e.resource):
- Safe JSON parsing with rfind-bounded extraction and error wrapping (C1, M1)
- Multi-object fallback: last-line reverse scan for multi-JSON output (M5)
- Moved Safe Parse Json Field to common_e2e.resource for reuse (L1)
- Migrated deprecated Run Keyword If to IF/ELSE blocks (M4)
- Added cwd parameter to Run CleverAgents Command (M6)
- API key protection via inline evaluation instead of RF variables (S1)
- Git return-code assertions in Create Temp Git Repo (L1)
- Removed unused Collections library import (L2)
- Warning log on directory removal failure instead of silent ignore (L6)
Hardened m6_acceptance.robot:
- Force Tags E2E instead of per-test [Tags] (L3)
- Per-test [Teardown] for resource cleanup, including Init test (M2, M9, L5)
- Collision-safe uuid4 hex suffix instead of randint (L7)
- Initialized session_id to EMPTY before test body for safe teardown (L2)
- ELSE branches with WARN log on conditional assertions (M3, M4, M6)
- Guard enforcement checks specific automation-profile fields (M5)
- Strengthened ci assertions with case-sensitive matching and JSON
field parsing for config get verification (M9)
- Eliminated Python code injection via string interpolation (H2)
- Added assertions to Full Flow Apply Step keyword (H4)
- Removed redundant config set in Guard test (M10)
- Accurate CHANGELOG entry describing actual test scope (C2)
Post-review fixes applied:
- Verify all 8 built-in profiles (manual, review, supervised, cautious,
trusted, auto, ci, full-auto) instead of 4 (M1)
- Session delete confirms removal via re-list (M2)
- Full Flow Apply Step verifies plan phase transition (M3)
- Plan execute output asserts plan_id presence + phase parsing (M4)
- Safe Parse Json Field gains two-strategy approach: outer-bracket
extraction then last-line reverse scan fallback (M5)
- Removed redundant automation profile reset in Config test (L1)
- JSON-quoted assertions ("ci", "auto") prevent false-positive
substring matches on short profile names (L3)
New E2E tests covering remaining acceptance criteria:
- Guard enforcement with custom profile: registers a profile with
explicit denylist, budget cap, and tool-call limits, then verifies
all guard fields via automation-profile show (AC-4 / H1)
- Profile precedence resolution: sets global profile to "review",
creates plan with --automation-profile trusted, asserts plan output
shows "trusted" not "review" (AC-5 / C2)
- Event queue via plan lifecycle transitions: creates plan, captures
initial state, executes, verifies state transition proving domain
event bus delivered and processed events (AC-3 / C1)
- Hierarchical decomposition via plan tree: creates plan with full-auto
profile, executes, runs plan tree --format json, verifies decision
nodes and children structure (AC-6 / C3)
Robot Framework uses dots as hierarchy separators in suite names
(e.g. "E2E.M6 Acceptance"). The E2E Suite Setup keyword replaced
spaces with underscores but preserved dots, producing directory names
like "E2E.M6_Acceptance". The CLI init command derives the project
name from Path.cwd().name, and Project.validate_name() rejects dots
("Name must be alphanumeric with hyphens, underscores, or spaces").
Added a second Replace String call to convert dots to underscores so
the sanitized suite name passes Project name validation.
Also increased subprocess timeouts in m3_e2e_verification.robot (60s->120s)
and m4_e2e_verification.robot (30s->120s) to prevent flaky CI failures from
Python startup overhead.
The plan lifecycle E2E tests call "plan use local/code-review" but
actions are user-defined entities that must be explicitly registered
via "action create --config <yaml>" before use. Without the action,
plan use failed with "Action 'local/code-review' not found" and all
LLM-dependent tests skipped.
Added action registration to M6 Suite Setup that dynamically selects
the actor matching the available API key (anthropic/claude-sonnet-4
when ANTHROPIC_API_KEY is set, openai/gpt-4o otherwise) and creates
the action YAML inline.
ISSUES CLOSED: #746
The `plan use` CLI set the --automation-profile, actor, and
execution-environment overrides on the in-memory Plan object after
use_action() had already persisted it to the database. Subsequent CLI
invocations (separate processes) loaded the plan from the database
without the overrides, so `plan execute` could never see the automation
profile.
Additionally, `plan execute` required the plan to be in
Strategize/complete state, but nothing ran the strategize phase between
`plan use` and `plan execute`. The auto_strategize field existed on
the AutomationProfile model but was never implemented — plans created
with auto_strategize=0.0 (ci, full-auto, trusted, etc.) stayed in
Strategize/queued indefinitely.
Changes:
- Call service._commit_plan(plan) after applying post-creation overrides
in the `plan use` CLI command so that automation-profile, actor, and
execution-environment changes survive across process boundaries.
- In the `plan execute` CLI command, detect plans still in
Strategize/queued and run the strategize phase inline via
PlanExecutor.run_strategize() before transitioning to Execute.
After inline strategize, if auto_progress already advanced the plan
to Execute (e.g. when auto_execute=0.0), skip the explicit
execute_plan() call and report the current state.
- Widen the auto-select query to include both QUEUED and COMPLETE plans
(QUEUED plans are now eligible because strategize will run inline).
- Update the "no plans ready" test to use an empty plan list (the old
test returned a QUEUED plan which is now eligible).
- Add BDD scenarios covering inline strategize, auto-progress handling,
and automation-profile persistence.
Refs: #746
list_plans() only read from the in-memory self._plans dict, but the
DI container creates PlanLifecycleService via providers.Factory (a new
instance per call), so every CLI invocation started with an empty dict.
Plans created by "plan use" were persisted to the database via
UnitOfWork.transaction() but "plan lifecycle-list" could never find
them because it never queried the database.
Added a database query path to list_plans() that calls
LifecyclePlanRepository.list_all() when persistence is enabled,
mirroring the existing database fallback in get_plan(). Also added
the list_all() method to LifecyclePlanRepository.
Refs: #746
The format_output() function returned a string that callers passed to
Rich console.print(), which wraps long lines at terminal width. This
injected literal newline characters into JSON string values (e.g. in
definition_of_done fields), producing invalid JSON that downstream
parsers could not decode (JSONDecodeError: Invalid control character).
For machine-readable formats (json, yaml, plain), format_output() now
writes the rendered output directly to sys.stdout and returns an empty
string. This preserves the exact serialization from json.dumps/
yaml.dump without Rich text processing artifacts.
Refs: #746
The automation-profile CLI commands used an _InMemoryProfileRepository
(a Python dict) that lost all data between CLI process invocations.
Profiles created with "automation-profile add" were invisible to
subsequent "automation-profile show" or "list" calls because each CLI
command is a separate process with a fresh empty dict.
Changes:
- Replaced _InMemoryProfileRepository with the real
AutomationProfileRepository from the infrastructure layer, wired
via the DI container following the same pattern as tool.py and
session.py.
- Added auto_commit support to AutomationProfileRepository (matching
the existing SessionRepository pattern) so that CLI commands running
outside a UnitOfWork commit each operation automatically.
- Added safety_json and guards_json Text columns to the
automation_profiles table (Alembic migration m6_005) for full-fidelity
round-trip of the AutomationGuard and SafetyProfile sub-models.
Previously, guards and several safety fields (max_cost_per_plan,
max_retries_per_step, etc.) were silently dropped on persistence.
- Updated _from_domain, _to_domain, and _update_row to serialize and
deserialize the full guard and safety sub-models via JSON, with
backward-compatible fallback to legacy scalar columns.
Refs: #746
Luis Mendes
