test/cli-docstring-example-validation
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
80bc9c552d |
test: restore and enhance e2e test coverage
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 33s
CI / helm (pull_request) Successful in 45s
CI / build (pull_request) Successful in 1m4s
CI / lint (pull_request) Successful in 1m13s
CI / typecheck (pull_request) Successful in 1m31s
CI / quality (pull_request) Successful in 1m37s
CI / security (pull_request) Successful in 1m49s
CI / integration_tests (pull_request) Successful in 3m34s
CI / e2e_tests (pull_request) Successful in 5m6s
CI / unit_tests (pull_request) Successful in 6m14s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 12m27s
CI / status-check (push) Blocked by required conditions
CI / coverage (push) Blocked by required conditions
CI / docker (push) Blocked by required conditions
CI / unit_tests (push) Has started running
CI / status-check (pull_request) Successful in 4s
CI / helm (push) Successful in 31s
CI / build (push) Successful in 53s
CI / lint (push) Successful in 1m3s
CI / quality (push) Successful in 1m13s
CI / typecheck (push) Successful in 1m46s
CI / security (push) Successful in 1m46s
CI / push-validation (push) Successful in 20s
CI / benchmark-publish (push) Failing after 43s
CI / e2e_tests (push) Successful in 4m9s
CI / integration_tests (push) Successful in 6m57s
Fix malformed imports in helper_m1_e2e_verification.py and helper_m4_e2e_cli.py where 'from helpers_common import reset_global_state' was incorrectly inserted inside a parenthesized import block, causing Python syntax errors and breaking the lint and unit_tests CI gates. Also fix import ordering in helper_m2_e2e_verification.py, helper_m5_e2e_context.py, helper_m5_e2e_support.py, and helper_m5_e2e_verification.py to satisfy ruff I001 import-sort rules by placing helpers_common imports in the correct group alongside other local robot/ helper imports. ISSUES CLOSED: #8459 |
||
|
|
7aa50ac489 |
fix(e2e): restore M5/WF14 tests broken by JSON envelope and stale config
- Add NO_COLOR=1 to E2E Suite Setup and Test Environment to prevent Rich from injecting ANSI escape codes into JSON output, which breaks Extract JSON From Stdout and JSON assertions in M5/WF14 acceptance tests. - Add reset_global_state() calls to all M1/M2/M4/M5 E2E verification helpers to clear Settings singleton, DI container, provider registry, and SQLAlchemy engine cache between parallel pabot worker runs. - Propagate NO_COLOR=1 at suite-setup level in E2E and integration test resource files so all Run Process subprocesses inherit clean terminal output mode. Root causes: 1. Missing NO_COLOR=1 caused Rich ANSI codes in CLI JSON output, making json.loads() failures in robot Extract JSON From Stdout 2. Stale Settings singleton carried provider keys and DB paths between pabot workers, causing UNIQUE constraint violations and stale config lookups in context policy and server mode E2E tests. Related: PR #9912 |
||
|
|
5f07316641 |
fix: wire DI persistence and plan execute/apply for M1 lifecycle
CI / lint (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 42s
CI / security (pull_request) Successful in 46s
CI / quality (pull_request) Successful in 27s
CI / unit_tests (pull_request) Successful in 3m16s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 15s
CI / e2e_tests (pull_request) Successful in 1m12s
CI / integration_tests (pull_request) Successful in 4m20s
CI / docker (pull_request) Successful in 59s
CI / coverage (pull_request) Successful in 6m34s
CI / lint (push) Successful in 20s
CI / typecheck (push) Successful in 45s
CI / security (push) Successful in 46s
CI / quality (push) Successful in 37s
CI / build (push) Successful in 17s
CI / e2e_tests (push) Successful in 52s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 5m9s
CI / integration_tests (push) Successful in 5m32s
CI / docker (push) Successful in 57s
CI / coverage (push) Successful in 6m10s
CI / benchmark-publish (push) Successful in 20m11s
CI / benchmark-regression (pull_request) Successful in 38m29s
Fixed 5 bugs preventing the M1 E2E acceptance test from passing: 1. _get_lifecycle_service() in action.py and plan.py bypassed the DI container, creating PlanLifecycleService without UnitOfWork. All plan/action data was in-memory only and lost between subprocess calls. Now uses container.plan_lifecycle_service() for DB persistence. 2. `plan execute` CLI only called service.execute_plan() (a pure state transition) without running PlanExecutor phase processing. Rewrote to detect the plan's current phase/state and dispatch synchronously: Strategize/queued → run_strategize(), Strategize/complete → transition + run_execute(), Execute/queued → run_execute(). 3. `plan apply` CLI had no plan_id argument. Added optional positional plan_id with _lifecycle_apply_with_id() that drives the plan through Apply/queued → Apply/processing → Apply/applied. 4. Preflight guardrail in start_strategize() built action_registry from the in-memory _actions dict only. Added get_action(plan.action_name) call to load the action from DB into cache before the guardrail check. 5. Robot Framework Create File syntax used continuation lines producing 9 arguments instead of 1. Fixed to use Catenate SEPARATOR=\n then pass single variable to Create File. Also fixed --branch main to --branch master (git init default). update mocks for execute_plan CLI changes across unit and integration tests The new execute_plan() command calls _get_plan_executor() and service.get_plan(plan_id) for phase/state detection. Existing tests only mocked _get_lifecycle_service, so MagicMock defaults caused phase/state comparisons to fail. Changes across 14 files: - Patch _get_plan_executor in all test setups that invoke the CLI execute command (Behave step files + Robot helper scripts) - Set service.get_plan.return_value to real Plan objects with correct phase/state so the execute_plan dispatch logic works - Fix error-path tests to use STRATEGIZE/COMPLETE plans so the error side_effects are actually reached - Fix "Multiple plans eligible" → "Multiple plans ready" message text to match existing test expectations increase Robot Framework subprocess timeouts for CI resource contention Three integration tests were timing out in CI due to resource contention when pabot runs multiple test suites in parallel. All three pass locally and the timeouts were simply too tight for constrained CI environments. - tdd_session_create_di.robot: 30s → 90s (DI container init + DB setup) - database_integration.robot: 60s → 120s (Run Python Script keyword) - m3_e2e_verification.robot: 60s → 120s (correction-live-revert spawns 3 sequential CLI subprocesses with full container initialization) ISSUES CLOSED: #789 |
||
|
|
317d015260
|
test(e2e): validate M4 acceptance criteria for v3.3.0 milestone closure
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 18s
CI / e2e_tests (pull_request) Successful in 27s
CI / security (pull_request) Successful in 38s
CI / typecheck (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 3m0s
CI / docker (pull_request) Successful in 10s
CI / integration_tests (pull_request) Successful in 3m29s
CI / coverage (pull_request) Successful in 5m54s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 17s
CI / security (push) Successful in 38s
CI / e2e_tests (push) Successful in 36s
CI / typecheck (push) Successful in 39s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m59s
CI / integration_tests (push) Successful in 3m24s
CI / docker (push) Successful in 38s
CI / coverage (push) Successful in 5m54s
CI / benchmark-publish (push) Successful in 19m34s
CI / benchmark-regression (pull_request) Successful in 36m21s
Strengthen M4 E2E CLI acceptance tests and address all review findings from the self-review on PR #681 (3 blocking, 7 non-blocking issues). Blocking fixes: - Remove tautological domain assertions from cli_plan_execute() that verified values the test itself constructed via _mock_parent_plan and could never fail. Replace with a comment documenting that subplan info cannot be verified from CLI output alone. - Fix unguarded positional arg access in plan_diff() that would raise IndexError if the production CLI passes plan_id as a keyword argument. Now uses kwargs-with-positional-fallback pattern. - Split the 1074-line helper_m4_e2e_verification.py into four focused modules under the CONTRIBUTING.md 500-line limit: helper_m4_e2e_common.py (237 lines) — constants, helpers, factories helper_m4_e2e_domain.py (495 lines) — domain model tests helper_m4_e2e_cli.py (426 lines) — CLI integration tests helper_m4_e2e_verification.py (81 lines) — thin dispatcher Non-blocking improvements: - Add ProjectLink isinstance type guard before iterating project_links. - Wrap all assert_called_once*() calls in try/except with _fail() pattern to produce standardised FAIL: diagnostics instead of raw AssertionError. - Make project_links extraction symmetric with action_name (kwargs + positional fallback at index 1). - Replace broad "execute" substring match with word-boundary regex r"\bexecute\b" to avoid false positives from incidental substrings. - Extract _make_subplan_status() factory, _assert_exit_code() and _assert_mock_called_once*() wrappers to eliminate DRY violations (SubplanStatus construction repeated 3x, exit-code checks 4x). - Replace non-deterministic datetime.now() calls with frozen constant _FROZEN_NOW = datetime(2026, 3, 1, 12, 0, 0) for deterministic fixtures. - Rename generic _DECISION_ULID_N constants to descriptive names (_DECISION_PROMPT_DEF, _DECISION_STRATEGY, _DECISION_PARALLEL_SPAWN, _DECISION_SPAWN_API, _DECISION_SPAWN_UI). Quality gates: - lint: PASS - format: PASS (1404 files unchanged) - typecheck: PASS (0 errors) - unit_tests: 10,431 scenarios, 0 failures - integration_tests: 1,452 tests, 0 failures - coverage_report: PASS (98%, threshold 97%) - security_scan: PASS - dead_code: PASS - docs: PASS - build: PASS ISSUES CLOSED: #495 |