6 Commits

Author	SHA1	Message	Date
freemo	8ea00f5185	fix: restore CI quality tests to passing state (#4175) ... Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me> Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>	2026-04-08 11:02:14 +00:00
brent.edwards	3eecb79003	test(plan): TDD failing tests for checkpoint real rollback (bug #822) ... TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839	2026-03-16 01:45:50 +00:00
hurui200320	317d015260	test(e2e): validate M4 acceptance criteria for v3.3.0 milestone closure ... Strengthen M4 E2E CLI acceptance tests and address all review findings from the self-review on PR #681 (3 blocking, 7 non-blocking issues). Blocking fixes: - Remove tautological domain assertions from cli_plan_execute() that verified values the test itself constructed via _mock_parent_plan and could never fail. Replace with a comment documenting that subplan info cannot be verified from CLI output alone. - Fix unguarded positional arg access in plan_diff() that would raise IndexError if the production CLI passes plan_id as a keyword argument. Now uses kwargs-with-positional-fallback pattern. - Split the 1074-line helper_m4_e2e_verification.py into four focused modules under the CONTRIBUTING.md 500-line limit: helper_m4_e2e_common.py (237 lines) — constants, helpers, factories helper_m4_e2e_domain.py (495 lines) — domain model tests helper_m4_e2e_cli.py (426 lines) — CLI integration tests helper_m4_e2e_verification.py (81 lines) — thin dispatcher Non-blocking improvements: - Add ProjectLink isinstance type guard before iterating project_links. - Wrap all assert_called_once*() calls in try/except with _fail() pattern to produce standardised FAIL: diagnostics instead of raw AssertionError. - Make project_links extraction symmetric with action_name (kwargs + positional fallback at index 1). - Replace broad "execute" substring match with word-boundary regex r"\bexecute\b" to avoid false positives from incidental substrings. - Extract _make_subplan_status() factory, _assert_exit_code() and _assert_mock_called_once*() wrappers to eliminate DRY violations (SubplanStatus construction repeated 3x, exit-code checks 4x). - Replace non-deterministic datetime.now() calls with frozen constant _FROZEN_NOW = datetime(2026, 3, 1, 12, 0, 0) for deterministic fixtures. - Rename generic _DECISION_ULID_N constants to descriptive names (_DECISION_PROMPT_DEF, _DECISION_STRATEGY, _DECISION_PARALLEL_SPAWN, _DECISION_SPAWN_API, _DECISION_SPAWN_UI). Quality gates: - lint: PASS - format: PASS (1404 files unchanged) - typecheck: PASS (0 errors) - unit_tests: 10,431 scenarios, 0 failures - integration_tests: 1,452 tests, 0 failures - coverage_report: PASS (98%, threshold 97%) - security_scan: PASS - dead_code: PASS - docs: PASS - build: PASS ISSUES CLOSED: #495	2026-03-13 14:03:37 +08:00
Brent E. Edwards	d88dad94f6	fix(test): resolve race condition in M4 validation integration test ... Three-pronged fix for intermittent pabot-parallel race condition in M4 validation integration tests: 1. Composable Setup Database Isolation keyword in common.resource gives each suite a unique CLEVERAGENTS_DATABASE_URL so concurrent pabot workers never contend on the same SQLite file. 2. Per-suite CLEVERAGENTS_HOME directories prevent shared temp directory cleanup from racing between workers. 3. Centralised reset_global_state() in robot/helpers_common.py clears Settings singleton, DI container, provider registry, and engine cache between chained CLI invocations in helper processes. Also: - Setup Test Environment now accepts optional mock_ai and auto_apply_migrations arguments (default TRUE) for backward compatibility while allowing suites to opt out. - Added Suite Teardown to cli_plan_context_commands.robot. - Fixed _COMMANDS typing in two helpers to eliminate type: ignore. - Updated docs/development/testing.md to reflect helpers_common delegation pattern. - Added timeout=30s to all Run Process calls in m4_e2e_verification.robot. Fixes: #563	2026-03-10 19:04:57 +00:00
hurui200320	09d92ac67b	test(e2e): validate M4 acceptance criteria for v3.3.0 milestone closure (#560) ... ## Summary Validates all M4 acceptance criteria for v3.3.0 milestone closure. Adds CLI-exercising integration tests for the three subplan commands that lacked actual CLI invocation, fixes a pre-existing unit test failure, and corrects CONTRIBUTORS.md ordering. ## Changes ### New CLI Integration Tests (Robot Framework) Three new test cases in `robot/m4_e2e_verification.robot` with helper functions in `robot/helper_m4_e2e_verification.py`: | Test | CLI Command | Mocking Approach | |------|-------------|-----------------| | `CLI Plan Use Creates Plan With Subplan Config` | `plan use local/refactor-action local/monorepo` | Patches `_get_lifecycle_service`; mocks `get_action_by_name` + `use_action` | | `CLI Plan Execute Transitions With Subplans` | `plan execute <plan_id>` | Patches `_get_lifecycle_service`; mocks `get_plan` + `execute_plan` | | `CLI Plan Tree Displays Subplan Hierarchy` | `plan tree <plan_id> --format json` | Patches `get_container`; real `Decision` objects with `SUBPLAN_SPAWN`/`SUBPLAN_PARALLEL_SPAWN` types | All three follow the same pattern as the existing `plan-diff` test: Typer `CliRunner.invoke()` with mocked services. ### Pre-existing Unit Test Fix **File:** `features/steps/repositories_uncovered_branches_steps.py` **Scenario:** `repo branch cov upsert profile with schema version mismatch` (line 110) **Root cause:** Plain `sessionmaker` returns a new session per call. The `Given` step inserted via session A and committed session B (different session), so the `When` step's session C couldn't see the uncommitted data. **Fix:** Changed to `scoped_session(sessionmaker(...))` so all factory calls return the same thread-local session. ### Other Fixes - **CONTRIBUTORS.md**: Moved "Rui Hu" to correct alphabetical position (between Freeman and Khyari) - **CHANGELOG.md**: Updated #495 entry to document CLI test additions ## M4 Acceptance Criteria Verification All 7 criteria exercised by 10 E2E tests (7 existing + 3 new) + 8 smoke tests: | # | Criterion | Test(s) | Status | |---|-----------|---------|--------| | 1 | Subplans spawned during Execute via SubplanConfig | `spawn-subplans`, **`cli-plan-use`**, **`cli-plan-execute`** | PASS | | 2 | Parallel execution with max_parallel bounds | `parallel-exec` | PASS | | 3 | Three-way merge combines non-conflicting changes | `merge-clean` | PASS | | 4 | Merge conflicts surfaced with git markers | `merge-conflict` | PASS | | 5 | Parent plan tracks subplan statuses | `parent-tracking` | PASS | | 6 | Plan tree displays subplan hierarchy | `plan-tree`, **`cli-plan-tree`** | PASS | | 7 | Plan diff shows merged results | `plan-diff` (already uses CLI) | PASS | ## Quality Gates | Stage | Result | |-------|--------| | lint | pass | | format | pass (1074 files unchanged) | | typecheck | pass (0 errors) | | unit_tests | 8524 scenarios, 0 failures | | integration_tests | 1110/1118 pass (8 pre-existing failures in `cli_plan_context_commands.robot`, identical on master) | | coverage_report | 97% (threshold: 97%) | | security_scan | pass | | dead_code | pass | | docs | pass | | build | pass | | benchmark | pass | ## Files Changed - `CHANGELOG.md` — Updated #495 entry - `CONTRIBUTORS.md` — Fixed alphabetical ordering - `features/steps/repositories_uncovered_branches_steps.py` — `scoped_session` fix - `robot/helper_m4_e2e_verification.py` — 3 new helper functions + imports - `robot/m4_e2e_verification.robot` — 3 new test cases ISSUES CLOSED: #495 Reviewed-on: #560 Reviewed-by: Luis Mendes <luis.mendes@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-03-06 03:22:45 +00:00
brent.edwards	cfc319ad27	test(e2e): verify M4 success criteria — subplans and parallel execution	2026-02-25 21:46:58 +00:00