Brent E. Edwards
5e625b22e1
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 18s
CI / quality (pull_request) Successful in 18s
CI / typecheck (pull_request) Successful in 39s
CI / security (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 3m15s
CI / integration_tests (pull_request) Failing after 3m33s
CI / docker (pull_request) Successful in 2m6s
CI / coverage (pull_request) Successful in 5m20s
CI / benchmark-regression (pull_request) Successful in 36m19s
Replace CliRunner + unittest.mock.patch with subprocess.run for all 21 CLI-facing test functions across the M1-M6 E2E verification helpers. Application code fixes: - action.py: _get_lifecycle_service() uses container.plan_lifecycle_service() - plan.py: _get_lifecycle_service() uses container.plan_lifecycle_service() - plan.py: three container.resolve(DecisionService) → container.decision_service() Test infrastructure: - New robot/helper_e2e_common.py with shared subprocess utilities (run_cli, setup_workspace with DB migrations, cleanup_workspace) - M1-M4, M6 helpers refactored to use run_cli() with real SQLite DB - M5 unchanged (0 CLI tests, all domain-level) - TDD detection updated to recognise run_cli() as subprocess invocation - Remove @tdd_expected_fail from TDD feature + robot tags - Update 8 Behave step files that mocked container.resolve() to use container.decision_service() / container.plan_lifecycle_service()
38 lines
1.9 KiB
Gherkin
38 lines
1.9 KiB
Gherkin
@tdd_bug @tdd_bug_658
|
|
Feature: TDD Bug #658 — E2E verification suites use mocks instead of real system
|
|
As a developer
|
|
I want to verify that M1-M6 E2E verification suites exercise at least one
|
|
real (non-mocked) CLI code path via subprocess invocation
|
|
So that integration failures in DI wiring, database, and process-level
|
|
behavior are detectable by the E2E test suites
|
|
|
|
The root cause is that all 21 CLI-facing tests in the M1-M6 E2E helper
|
|
files use ``unittest.mock.patch`` to replace service factories with
|
|
``MagicMock`` objects and invoke the CLI via Typer's in-process
|
|
``CliRunner`` instead of ``subprocess.run``. This means:
|
|
|
|
- DI wiring bugs (e.g. the ``container.db()`` AttributeError in #554,
|
|
#570) are invisible to the E2E suites.
|
|
- Database schema/migration issues are invisible.
|
|
- Process-level behavior (exit codes, environment variables, Rich
|
|
console routing) is never tested.
|
|
|
|
The 35 remaining tests are pure domain/service-level tests that never
|
|
invoke the CLI at all — they are legitimate unit/integration tests but
|
|
should not be labeled "E2E verification."
|
|
|
|
Background:
|
|
Given the M1-M6 E2E verification helper files exist for e2e-mock-audit
|
|
|
|
Scenario: At least one CLI-facing E2E test invokes the real CLI via subprocess
|
|
When I analyze mock and subprocess usage in M1-M6 helpers for e2e-mock-audit
|
|
Then at least one CLI-facing test should use subprocess instead of CliRunner for e2e-mock-audit
|
|
|
|
Scenario: At least one CLI-facing E2E test exercises the real service layer
|
|
When I analyze mock and subprocess usage in M1-M6 helpers for e2e-mock-audit
|
|
Then at least one CLI-facing test should not mock the service factory for e2e-mock-audit
|
|
|
|
Scenario: No E2E suite has 100% mocked CLI tests
|
|
When I analyze mock and subprocess usage per suite for e2e-mock-audit
|
|
Then every suite with CLI tests should have at least one unmocked test for e2e-mock-audit
|