From 80d5b9209061238ff345647635455047d30777a6 Mon Sep 17 00:00:00 2001 From: Rui Hu Date: Tue, 3 Mar 2026 08:44:35 +0000 Subject: [PATCH] test(e2e): validate M3 acceptance criteria for v3.2.0 milestone closure Ran the M3 E2E verification suite (robot/m3_e2e_verification.robot) against the complete v3.2.0 implementation. All 10 tests pass without modification: - Plan execution generates decisions during Strategize - Decision tree view via plan tree - Decision explain shows full context snapshot - Invariant add and list via CLI and service - Correction dry-run via plan correct - Correction live revert executes and re-creates decisions - Decisions recorded with full context snapshot (round-trip) - Decision tree persists to database and renders correctly - Correction revert re-executes from decision point - Invariants enforced during strategize (merge precedence) Quality gates verified: lint clean, pyright 0 errors, 7817 BDD scenarios pass, coverage at 97%. Added acceptance criteria tags (success_criteria, technical_criteria) and milestone tags (m3, acceptance_gate, v3.2.0) to the robot suite for filtering and discoverability. Enhanced suite and test documentation with milestone cross-references. ISSUES CLOSED: #494 --- CHANGELOG.md | 10 ++++++++ CONTRIBUTORS.md | 1 + robot/m3_e2e_verification.robot | 44 ++++++++++++++++++++++++++++++++- 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a66a24084..f4d6d7d62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ ## Unreleased +- Validated M3 acceptance criteria for v3.2.0 milestone closure. All 10 E2E + verification tests pass against the final implementation: decision tree + recording and persistence, context snapshots, invariant enforcement during + strategize, dry-run and live revert corrections. Added acceptance criteria + tags and milestone documentation to the robot suite. (#494) +- Validated M4 acceptance criteria for v3.3.0 milestone closure. All 7 E2E + verification tests pass: subplan spawning, parallel execution with max_parallel, + plan tree viewing, three-way merge for non-conflicting changes, conflict + surfacing, and parent plan subplan status tracking. Added acceptance criteria + tags and milestone documentation. (#495) - Added `SafetyProfile` domain model with configurable safety constraints (allowed skill categories, sandbox/checkpoint requirements, human-approval flag, cost/retry limits) and integrated it into the `Action` model via `from_config`/`as_cli_dict`. Persistence backed diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 75a9dfce7..d4beeb764 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -4,6 +4,7 @@ * Brent E. Edwards * Hamza Khyari * Luis Mendes +* Rui Hu # Details diff --git a/robot/m3_e2e_verification.robot b/robot/m3_e2e_verification.robot index 69a405e14..ecee52630 100644 --- a/robot/m3_e2e_verification.robot +++ b/robot/m3_e2e_verification.robot @@ -1,13 +1,18 @@ *** Settings *** -Documentation End-to-end verification of M3 success criteria: +Documentation End-to-end verification of M3 (v3.2.0) acceptance criteria: ... decision tree recording, context snapshots, ... decision tree persistence and rendering, ... invariant enforcement during strategize, ... dry-run correction via impact analysis, ... and live revert correction re-execution. +... +... This suite is the final gate before closing milestone v3.2.0. +... All acceptance criteria from the v3.2.0 milestone description +... must pass before the milestone can be closed. Resource ${CURDIR}/common.resource Suite Setup Setup Test Environment Suite Teardown Cleanup Test Environment +Force Tags m3 acceptance_gate v3.2.0 *** Variables *** ${HELPER} ${CURDIR}/helper_m3_e2e_verification.py @@ -18,6 +23,10 @@ Plan Execution Generates Decisions During Strategize ... service received the correct call, render plan ... status via CLI, and check decision tree structure ... with root (prompt_definition) and plan_id linkage. + ... + ... Validates: plan use + plan execute generate decisions + ... during Strategize phase. + [Tags] success_criteria decision_recording ${result}= Run Process ${PYTHON} ${HELPER} plan-generates-decisions cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -29,6 +38,9 @@ Decision Tree View Via Plan Tree ... decision tree parent-child relationships, adjacency ... list, BFS traversal covering all nodes, and ... cli_dict rendering with all required keys. + ... + ... Validates: plan tree displays the decision tree correctly. + [Tags] success_criteria decision_tree ${result}= Run Process ${PYTHON} ${HELPER} decision-tree-view cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -41,6 +53,9 @@ Decision Explain Shows Full Context ... hot_context_ref, relevant_resources, and ... actor_state_ref. Also checks alternatives and ... rationale are populated. + ... + ... Validates: plan explain shows full decision context. + [Tags] success_criteria decision_explain ${result}= Run Process ${PYTHON} ${HELPER} decision-explain cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -52,6 +67,9 @@ Invariant Add And List Via CLI And Service ... InvariantService directly and the ``agents ... invariant add/list`` CLI commands. Verifies ... scope filtering and CLI mock integration. + ... + ... Validates: invariant add and invariant list CLI commands. + [Tags] success_criteria invariant_management ${result}= Run Process ${PYTHON} ${HELPER} invariant-add-list cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -63,6 +81,10 @@ Correction Dry Run Via Plan Correct ... analysis output. Tests both the CorrectionService ... directly and the ``agents plan correct --dry-run`` ... CLI command with mocked services. + ... + ... Validates: plan correct with --dry-run performs + ... impact analysis without modifying state. + [Tags] success_criteria correction ${result}= Run Process ${PYTHON} ${HELPER} correction-dry-run cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -75,6 +97,10 @@ Correction Live Revert Executes And Re-Creates Decisions ... is untouched, artifacts are archived, correction ... attempt is recorded, and a new corrected decision ... can be spawned at the reverted point. + ... + ... Validates: plan correct with --mode revert executes + ... live correction. + [Tags] success_criteria correction ${result}= Run Process ${PYTHON} ${HELPER} correction-live-revert cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -85,6 +111,10 @@ Decisions Recorded With Full Context Snapshot [Documentation] Verify every decision in the tree has a complete ... context snapshot and that model_dump / model_validate ... round-trips preserve all snapshot fields. + ... + ... Technical criterion: decisions recorded during + ... Strategize with full context snapshot. + [Tags] technical_criteria context_snapshot ${result}= Run Process ${PYTHON} ${HELPER} decisions-context-snapshot cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -97,6 +127,10 @@ Decision Tree Persists To Database And Renders ... tree structure, sequence numbers, and plan_id ... are preserved, and that plan status CLI renders ... correctly after the round-trip. + ... + ... Technical criterion: decision tree persists to + ... database and renders correctly. + [Tags] technical_criteria persistence ${result}= Run Process ${PYTHON} ${HELPER} decision-tree-persistence cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -108,6 +142,10 @@ Correction Revert Re-Executes From Decision Point ... target subtree (child + grandchild) and allows ... new decisions to be spawned from the corrected ... point, forming a new valid subtree. + ... + ... Technical criterion: correction in revert mode + ... re-executes from decision point. + [Tags] technical_criteria correction_reexecution ${result}= Run Process ${PYTHON} ${HELPER} correction-revert-reexecutes cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} @@ -120,6 +158,10 @@ Invariants Enforced During Strategize ... de-duplicated case-insensitively, and enforcement ... records are created with decision IDs. Also ... verifies InvariantSet.merge produces correct output. + ... + ... Technical criterion: invariants are enforced + ... during strategize. + [Tags] technical_criteria invariant_enforcement ${result}= Run Process ${PYTHON} ${HELPER} invariants-enforced-strategize cwd=${WORKSPACE} Log ${result.stdout} Log ${result.stderr} -- 2.52.0