cb1403c24d
CI / lint (push) Successful in 15s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 24s
CI / security (push) Successful in 34s
CI / typecheck (push) Successful in 54s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 3m14s
CI / unit_tests (push) Successful in 3m34s
CI / docker (push) Successful in 41s
CI / coverage (push) Successful in 4m26s
CI / benchmark-publish (push) Has been cancelled
## Summary
Validates all 10 M3 (v3.2.0) acceptance criteria via the E2E verification suite. This is the final gate before closing milestone v3.2.0.
### Single Commit (Squashed per CONTRIBUTING.md)
Branch history was squashed from 2 commits into 1 (`e1665c6e`) to comply with the one-commit-per-issue rule. The single commit includes:
1. **Original work**: Suite-level Force Tags, per-test acceptance criteria tags, enhanced documentation with milestone cross-references, CHANGELOG and CONTRIBUTORS entries.
2. **Review feedback fixes** (addressing Luis's CRITICAL/HIGH findings): Replaced fixture-self-assertion patterns with real CLI-path validation for all 10 acceptance criteria, added persistence-backed decision tree checks, robust JSON output parsing, explicit Robot process timeouts, and clearer correction tags.
3. **Rebase onto current master** (`fff6bb38`): Clean rebase resolving CHANGELOG conflict, as requested by PM (freemo).
4. **Bug fix — `async_enabled` attribute**: `PlanLifecycleService.execute_plan()` accesses `self.settings.async_enabled` (`plan_lifecycle_service.py:283`). The test helper's `_make_settings()` used `create_autospec(Settings)` but didn't set this attribute, causing `plan execute` CLI tests to fail with `AttributeError`. Fixed by adding `settings.async_enabled = False`.
5. **Timeout fixes for parallel execution**: When `pabot` runs 16 parallel Robot processes, cold-start import times + Alembic migrations cause 30s timeouts to be insufficient. Increased timeouts from 30s to 120s in 3 robot files: `actor_context_management.robot`, `decision_di_wiring_smoke.robot`, `changeset_persistence.robot`.
### Files Changed (7)
| File | Change |
|------|--------|
| `robot/helper_m3_e2e_verification.py` | Rewritten for real CLI-path validation; `_make_settings()` fixed with `async_enabled = False` |
| `robot/m3_e2e_verification.robot` | Force Tags, per-test tags, timeouts, enhanced docs |
| `robot/actor_context_management.robot` | Timeout 30s → 120s (line 92) |
| `robot/decision_di_wiring_smoke.robot` | Timeout 30s → 120s (lines 14, 21) |
| `robot/changeset_persistence.robot` | `${TIMEOUT}` 30s → 120s (line 12) |
| `CHANGELOG.md` | Entry for #494 |
| `CONTRIBUTORS.md` | Added Rui Hu |
### Acceptance Criteria Verified (10/10)
**Success Criteria:**
- `plan use` + `plan execute` generates decisions during Strategize
- `plan tree` displays decision tree correctly (via CLI invocation)
- `plan explain` shows full decision context (via CLI invocation)
- `invariant add --project` / `invariant list --project` work with project scope
- `plan correct --dry-run` performs impact analysis (via CLI invocation)
- `plan correct --mode=revert` executes live correction (via CLI invocation)
**Technical Criteria:**
- Decisions recorded with full context snapshot
- Decision tree persists to database and renders correctly
- Correction in revert mode re-executes from decision point
- Invariants enforced during strategize
### Quality Gates (Full Nox Suite)
| Session | Result | Details |
|---------|--------|---------|
| `lint` | PASS | ruff clean |
| `format` | PASS | ruff format clean |
| `typecheck` | PASS | pyright 0 errors, 0 warnings |
| `security_scan` | PASS | bandit clean |
| `dead_code` | PASS | vulture clean |
| `unit_tests` | PASS | 8500 BDD scenarios, 0 failures |
| `integration_tests` | PASS | 1194/1194 tests, 0 failures |
| `docs` | PASS | mkdocs build clean |
| `build` | PASS | wheel built |
| `benchmark` | PASS | performance benchmarks pass |
| `coverage_report` | PASS | 96.96% → rounds to 97% (meets `--fail-under=97`) |
Closes #494
Reviewed-on: #559
Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
89 lines
3.9 KiB
Plaintext
89 lines
3.9 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for ChangeSet persistence via CLI plan artifacts output.
|
|
Library OperatingSystem
|
|
Library Process
|
|
Library Collections
|
|
|
|
Suite Setup Set Suite Variables
|
|
Force Tags changeset persistence
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
# Normal duration: ~5-10s per test. Timeout raised from 30s to 120s for
|
|
# pabot cold-start (16 parallel processes) + Alembic migration overhead.
|
|
${TIMEOUT} 120s
|
|
|
|
*** Keywords ***
|
|
Set Suite Variables
|
|
Set Suite Variable ${AGENTS} ${PYTHON} -m cleveragents.cli.main
|
|
|
|
Run Agents Command
|
|
[Arguments] @{args}
|
|
${result}= Run Process ${PYTHON} -m cleveragents.cli.main @{args}
|
|
... timeout=${TIMEOUT}
|
|
... env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true
|
|
... env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true
|
|
... env:NO_COLOR=1
|
|
RETURN ${result}
|
|
|
|
*** Test Cases ***
|
|
Plan Artifacts Shows Changeset ID Field
|
|
[Documentation] Verify plan artifacts output contains changeset_id field.
|
|
${result}= Run Agents Command plan artifacts --help
|
|
Should Contain ${result.stdout} plan_id
|
|
... msg=Plan artifacts help should mention plan_id
|
|
|
|
Plan Diff Help Shows Format Options
|
|
[Documentation] Verify plan diff command exposes format options.
|
|
${result}= Run Agents Command plan diff --help
|
|
Should Contain ${result.stdout} format
|
|
... msg=Plan diff help should mention format option
|
|
|
|
Plan Artifacts Help Contains Expected Text
|
|
[Documentation] Verify plan artifacts help text is present.
|
|
${result}= Run Agents Command plan artifacts --help
|
|
Should Contain ${result.stdout} artifacts
|
|
... msg=Help text should mention artifacts
|
|
|
|
Changeset Persistence Module Is Importable
|
|
[Documentation] Verify the changeset_repository module can be imported.
|
|
${result}= Run Process ${PYTHON} -c
|
|
... from cleveragents.infrastructure.database.changeset_repository import SqliteChangeSetStore; print("OK")
|
|
... timeout=${TIMEOUT}
|
|
... env:PYTHONPATH=src
|
|
Should Be Equal As Strings ${result.stdout.strip()} OK
|
|
|
|
SqliteChangeSetStore Round Trip Via CLI Script
|
|
[Documentation] Run a Python script that exercises SqliteChangeSetStore round-trip.
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import sys
|
|
... from sqlalchemy import create_engine
|
|
... from sqlalchemy.orm import sessionmaker
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... from cleveragents.infrastructure.database.changeset_repository import SqliteChangeSetStore
|
|
... from cleveragents.domain.models.core.change import ChangeEntry, ChangeOperation
|
|
... engine = create_engine("sqlite:///:memory:", connect_args={"check_same_thread": False})
|
|
... Base.metadata.create_all(engine)
|
|
... factory = sessionmaker(bind=engine)
|
|
... store = SqliteChangeSetStore(factory)
|
|
... cs_id = store.start("plan-robot-1")
|
|
... entry = ChangeEntry(plan_id="plan-robot-1", resource_id="res-1", tool_name="file-write", operation=ChangeOperation.CREATE, path="src/test.py", after_hash="abc123")
|
|
... store.record(cs_id, entry)
|
|
... factory().commit()
|
|
... cs = store.get(cs_id)
|
|
... assert cs is not None
|
|
... assert len(cs.entries) == 1
|
|
... summary = store.summarize(cs_id)
|
|
... assert summary["total"] == 1
|
|
... assert summary["creates"] == 1
|
|
... store.delete_for_plan("plan-robot-1")
|
|
... factory().commit()
|
|
... cs2 = store.get(cs_id)
|
|
... assert cs2 is None or len(cs2.entries) == 0
|
|
... print("PASS")
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... timeout=${TIMEOUT}
|
|
... env:PYTHONPATH=src
|
|
Should Contain ${result.stdout} PASS
|
|
... msg=SqliteChangeSetStore round-trip failed: ${result.stderr}
|