Robot Framework E2E test suite exercising the complete M6 success
criteria verification sequence:
- Porting action creation from YAML config via CLI
- Plan use + execute via CLI with mocked lifecycle service
- Hierarchical decomposition: 4+ levels (root + L1-L4 = 21+ plans)
- Decision correction recomputes only affected subtree (CorrectionImpact)
- Parallel execution scales to 15 concurrent subplans (10+ required)
- Realistic porting task: full ACTION → STRATEGIZE → EXECUTE → APPLY
lifecycle with 10 subplans completing autonomously
- Plan apply transitions to APPLIED terminal state
- SubplanFailureHandler retry/stop-others logic verification
- SubplanConfig supports all execution modes and merge strategies
- Decision tree structure: PROMPT_DEFINITION root with
SUBPLAN_PARALLEL_SPAWN children and superseded_by flow
Ten subcommands in the Python helper, each printing a sentinel
string on success. All subcommands verified passing locally.
Closes#407
Replace dict[str, Any] with a private _YamlMapping type alias
(dict[str, object]) in AgentSkillSpec Pydantic fields and all
internal helper function signatures. Add isinstance guard for
allowed-tools list iteration to satisfy pyright.
Refs: #160
P0: reject register() after close_all() with RuntimeError.
P1: catch CancelledError in close_all(), use WeakKeyDictionary for
cancellation_reasons to prevent memory leak, guard StateManager
update_state/reset/load_checkpoint/time_travel after close().
P2: contextlib.suppress in __del__ for partial construction, re-cancel
pending tasks in cleanup_tasks_async, handle late tasks added during
await window, guard AcpEventQueue.publish() after close with _is_closed
flag and is_closed property, fix ASV TimeRegisterBatch crash.
Tests: 5 new Behave scenarios (T1-T4 + is_closed), log handler and
event loop cleanup in after_scenario (T5-T6).
Docs: async_safety.md updated for register-after-close and state
mutation guards.
ISSUES CLOSED: #321
Add AsyncResourceTracker (core/async_cleanup.py) providing a central
registry for async resources with timeout-bounded close_all(), async
context manager support, and a __del__ finalizer that logs leaked
resources by name.
Enhance LangGraphBridge with cleanup_tasks_async() that awaits
in-flight tasks with a deadline instead of fire-and-forget cancel().
Add cancellation_reasons dict to trace why tasks were cancelled.
Add StateManager.close() to properly release checkpoint file handles
and complete the RxPY BehaviorSubject. Add AcpEventQueue.close() to
dispose all subscriptions.
Includes 14 Behave scenarios (67 steps), Robot integration tests,
ASV benchmarks, and docs/reference/async_safety.md.
ISSUES CLOSED: #321
After rebasing onto master, m4_001_decision_tables and
d0_002_merge_changeset_and_locks were both leaf heads. Update
down_revision so the decision migration chains after the merge
migration, restoring a single Alembic head.
ISSUES CLOSED: #171
Add decision persistence layer with DecisionRepository, DecisionModel,
and Alembic migration. Includes tree queries (BFS traversal via deque,
path-to-root), superseded lookup, ordered decision path retrieval,
concrete Decision type annotations (via TYPE_CHECKING), and comprehensive
test coverage (Behave BDD, Robot Framework, ASV benchmarks). Updated
database_schema.md, CHANGELOG.md, and CONTRIBUTORS.md.
ISSUES CLOSED: #171
- Patch CorrectionService at its module path, not the local-import site
- Use request_correction/execute_correction/analyze_impact (real API)
- Replace invalid 27-char subplan_id values with valid 26-char ULIDs
- Add required --guidance flag to dry-run CLI invocation
- Fix feature file assertions to match actual CLI output
Resolves CI failures in unit_tests (job 4) and coverage (job 6) for
run 645 on PR #441.
- decision-tree-view, decision-explain, and decision-tree-persistence
now invoke 'plan status --format plain' via mocked lifecycle service
so regressions in CLI rendering/serialization are caught
- plan-generates-decisions now asserts use_action was called by the CLI
and verifies plan status renders the strategize phase after creation
- Updated robot test case documentation to reflect CLI integration
Replace hard-coded '11 decision types' with dynamic len(DecisionType) in
the assertion and remove the literal from scenario names, step patterns,
docstrings, and docs so the tests stay correct when new DecisionType
values are added.
- Replace dead after_scenario() hook with context.add_cleanup() calls
so patchers are properly stopped (Behave only runs hooks from
environment.py)
- Fix invalid ULID constants: pad DECISION/CORRECTION to 26 chars,
replace invalid 'I' with 'J' in INVARIANT_ULID
- Add temp file cleanup for NamedTemporaryFile(delete=False) via
context.add_cleanup() in behave steps and try/finally in robot helper
- Remove decision-persistence doc sections from testing.md that belong
in PR #438, not this PR
The jitter-spread scenario used millisecond-bucketed wall-clock
timestamps to assert that retried operations did not cluster. On busy
CI runners — especially when a preceding scenario's time.sleep patch
leaked through a late cleanup — tenacity's retry waits became no-ops
and all five operations landed in the same millisecond bucket, tripping
the max_in_bucket <= 3 assertion.
Three hardening changes:
1. Jitter test: switch from time.time() to time.monotonic_ns() and
replace the fragile bucket assertion with a unique-timestamp count
(>= 2 distinct readings among 5 sequential operations).
2. Timeout test: eagerly restore time.sleep in a try/finally block so
subsequent scenarios never observe the patched no-op, regardless of
behave's cleanup ordering.
3. Async circuit breaker: lower recovery_timeout from 0.2 s to 0.1 s
(the sleep step already waits 0.2 s) to give a wider safety margin
on slow CI machines.
The CLI plain format output does not include the literal text "Dry Run"
(that string only appears in the Rich panel title). Assert against
"risk_level" which is present in the dry-run impact output instead.
- Patch CorrectionService at its definition site instead of the lazy
import location in plan.py (same fix as M3 helper)
- Replace hardcoded fake subplan/correction IDs with real ULIDs
generated via ulid.ULID() to satisfy Pydantic pattern validation
- Add missing --guidance flag to correction dry-run CLI invocation
- Fix subplan-status-sequential assertion to check for plan_id
(subplan_count is not part of the status JSON output)
The CorrectionService import in plan.py is a lazy import inside the
correct() function body, so it does not exist as a module-level attribute.
Patch the class at its definition site instead:
cleveragents.application.services.correction_service.CorrectionService
Fixes CI integration_tests failure for PR #439.
Robot Framework E2E test suite for M3 milestone verification covering:
- Plan execution generating decisions during Strategize phase
- Decision tree viewing with parent-child relationships and BFS traversal
- Decision explanation with full context snapshot verification
- Invariant add/list via CLI and InvariantService with scope filtering
- Dry-run correction via CorrectionService with impact analysis
- Live revert correction execution with decision re-creation
- Context snapshot round-trip serialisation assertions
- Decision tree persistence via model_dump/model_validate
- Correction revert re-execution from decision point
- Invariant enforcement during strategize with merge precedence
ISSUES CLOSED: #404
The 'Test Concurrent Retries With Jitter' test asserted that 5 concurrent
retry timestamps spread across >10ms. On busy CI runners thread scheduling
can compress wakeups into a narrower window, causing spurious failures.
Lowered the threshold from 10ms to 1ms — still validates that jitter produces
non-identical delays while tolerating CI scheduling variance.
brent.edwards
hamza.khyari
aditya
khyari hamza
freemo