freemo
8ea00f5185
CI / unit_tests (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / security (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / push-validation (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me> Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
38 lines
1.9 KiB
Gherkin
38 lines
1.9 KiB
Gherkin
@tdd_issue @tdd_issue_658 @tdd_issue_4178
|
|
Feature: TDD Issue #658 — E2E verification suites use mocks instead of real system
|
|
As a developer
|
|
I want to verify that M1-M6 E2E verification suites exercise at least one
|
|
real (non-mocked) CLI code path via subprocess invocation
|
|
So that integration failures in DI wiring, database, and process-level
|
|
behavior are detectable by the E2E test suites
|
|
|
|
The root cause is that all 21 CLI-facing tests in the M1-M6 E2E helper
|
|
files use ``unittest.mock.patch`` to replace service factories with
|
|
``MagicMock`` objects and invoke the CLI via Typer's in-process
|
|
``CliRunner`` instead of ``subprocess.run``. This means:
|
|
|
|
- DI wiring bugs (e.g. the ``container.db()`` AttributeError in #554,
|
|
#570) are invisible to the E2E suites.
|
|
- Database schema/migration issues are invisible.
|
|
- Process-level behavior (exit codes, environment variables, Rich
|
|
console routing) is never tested.
|
|
|
|
The 35 remaining tests are pure domain/service-level tests that never
|
|
invoke the CLI at all — they are legitimate unit/integration tests but
|
|
should not be labeled "E2E verification."
|
|
|
|
Background:
|
|
Given the M1-M6 E2E verification helper files exist for e2e-mock-audit
|
|
|
|
Scenario: At least one CLI-facing E2E test invokes the real CLI via subprocess
|
|
When I analyze mock and subprocess usage in M1-M6 helpers for e2e-mock-audit
|
|
Then at least one CLI-facing test should use subprocess instead of CliRunner for e2e-mock-audit
|
|
|
|
Scenario: At least one CLI-facing E2E test exercises the real service layer
|
|
When I analyze mock and subprocess usage in M1-M6 helpers for e2e-mock-audit
|
|
Then at least one CLI-facing test should not mock the service factory for e2e-mock-audit
|
|
|
|
Scenario: No E2E suite has 100% mocked CLI tests
|
|
When I analyze mock and subprocess usage per suite for e2e-mock-audit
|
|
Then every suite with CLI tests should have at least one unmocked test for e2e-mock-audit
|