test(e2e): TDD failing tests for E2E mock-only coverage (bug #658) #738

Merged
brent.edwards merged 2 commits from tdd/m6-e2e-mock-only-coverage into master 2026-03-12 22:02:02 +00:00

2 Commits

Author SHA1 Message Date
brent.edwards 703511a28f Merge branch 'master' into tdd/m6-e2e-mock-only-coverage
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 18s
CI / e2e_tests (pull_request) Successful in 34s
CI / security (pull_request) Successful in 37s
CI / typecheck (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 4m46s
CI / integration_tests (pull_request) Successful in 5m11s
CI / docker (pull_request) Successful in 42s
CI / coverage (pull_request) Successful in 5m35s
CI / benchmark-regression (pull_request) Successful in 36m16s
2026-03-12 21:43:23 +00:00
Brent E. Edwards 6806ef3620 test(e2e): TDD failing tests for E2E mock-only coverage (bug #658)
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / lint (pull_request) Successful in 20s
CI / quality (pull_request) Successful in 20s
CI / typecheck (pull_request) Successful in 38s
CI / security (pull_request) Successful in 47s
CI / unit_tests (pull_request) Successful in 3m19s
CI / docker (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 5m17s
CI / coverage (pull_request) Successful in 5m38s
CI / benchmark-regression (pull_request) Successful in 35m56s
Add Behave and Robot Framework TDD tests that use AST analysis to
inspect the M1-M6 E2E verification helper files and prove that all
21 CLI-facing test functions use unittest.mock.patch + CliRunner
instead of exercising the real CLI via subprocess invocation.

The tests verify three properties that currently fail (proving #658):
- At least one CLI-facing test uses subprocess instead of CliRunner
- At least one CLI-facing test exercises the real service layer
- Every suite with CLI tests has at least one unmocked test

Tagged @tdd_expected_fail so the tests pass CI while the bug is
present.  The @tdd_expected_fail tag inverts the result: the tests
pass because the underlying assertions fail (confirming the bug).
Once #658 is fixed, the tag is removed and the tests run normally.

Files added:
- features/tdd_e2e_mock_only_coverage.feature (3 scenarios)
- features/steps/tdd_e2e_mock_only_coverage_steps.py
- robot/tdd_e2e_mock_only_coverage.robot (3 test cases)
- robot/helper_tdd_e2e_mock_only_coverage.py

ISSUES CLOSED: #697
2026-03-12 18:35:54 +00:00