feat(testing): implement @tdd_expected_fail tag handling in Robot Framework #673

Merged
brent.edwards merged 3 commits from feature/m5-robot-tdd-tags into master 2026-03-16 23:13:31 +00:00

3 Commits

Author SHA1 Message Date
brent.edwards c13a62a2f2 Merge branch 'master' into feature/m5-robot-tdd-tags
CI / lint (pull_request) Successful in 25s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 40s
CI / typecheck (pull_request) Successful in 47s
CI / security (pull_request) Successful in 45s
CI / e2e_tests (pull_request) Successful in 2m4s
CI / benchmark-regression (pull_request) Failing after 1m41s
CI / unit_tests (pull_request) Successful in 5m2s
CI / integration_tests (pull_request) Successful in 5m30s
CI / docker (pull_request) Successful in 55s
CI / coverage (pull_request) Successful in 6m9s
2026-03-16 23:06:19 +00:00
brent.edwards d0ca129d90 Merge branch 'master' into feature/m5-robot-tdd-tags
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / lint (pull_request) Successful in 20s
CI / quality (pull_request) Successful in 29s
CI / unit_tests (pull_request) Has been cancelled
CI / e2e_tests (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / security (pull_request) Has been cancelled
CI / integration_tests (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
CI / benchmark-regression (pull_request) Has been cancelled
2026-03-16 23:05:43 +00:00
hurui200320 a5de448856 feat(testing): implement @tdd_expected_fail tag handling in Robot Framework
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 20s
CI / quality (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 51s
CI / security (pull_request) Successful in 53s
CI / e2e_tests (pull_request) Successful in 1m20s
CI / unit_tests (pull_request) Successful in 3m24s
CI / benchmark-regression (pull_request) Failing after 2m32s
CI / integration_tests (pull_request) Successful in 3m33s
CI / docker (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 6m52s
Implements the three-tag TDD bug-capture system in Robot Framework via a
Listener v3 module, paralleling the Behave implementation. Tests tagged
tdd_expected_fail that fail have their result inverted to PASS (bug still
exists); tests that unexpectedly pass are inverted to FAIL with guidance.

Addresses all 15 findings from code review (PR !673, reviewer hamza.khyari):

P2 fixes:
- Added idempotency guard (_processed_tests set) to prevent double-inversion
  when the listener is loaded twice in the same process.
- Rewrote normal-test-unaffected check to run alongside a tdd_expected_fail
  fixture in a single Robot invocation, proving the listener is loaded and
  selectively applies rather than being a tautological pass.

P3 fixes:
- Added output.xml existence guard with clear diagnostics in _run_fixture.
- Documented intentional use of data.tags (static definition) vs result.tags
  (runtime-modifiable) in end_test docstring.
- Added SKIP status test fixture and integration test case.
- Added message content assertion in cmd_expected_fail_inverted.
- Tightened substring assertions to match specific error text.
- Added tdd_expected_fail-alone fixture (both companions missing).
- Added close() hook to clear _validation_errors and _processed_tests.
- Simplified _run_fixture return type to tuple[str, str].
- Changed listener path resolution from CWD-relative to __file__-relative
  in noxfile.py (integration_tests, slow_integration_tests, e2e_tests).

P4 fixes:
- Added __all__ declaration to helper module.
- Changed module docstring from "mirroring" to "paralleling".
- Added comment documenting accepted XML parsing risk (self-generated XML).

Additional fixes:
- Increased M4 E2E plan-tree test timeout from 30s to 120s (pre-existing
  timeout failure unrelated to this feature).

Quality gates (post-rebase onto latest master):
- nox -s lint: PASS
- nox -s typecheck: PASS (0 errors)
- nox -s unit_tests: PASS (10,700 scenarios)
- nox -s integration_tests: PASS (1,505 tests)
- nox -s coverage_report: PASS (97.9% >= 97% threshold)
- nox -s benchmark: PASS
- nox -s docs: PASS
- nox -s build: PASS
- nox -s security_scan: PASS
- nox -s dead_code: PASS

ISSUES CLOSED: #628
2026-03-16 22:45:55 +00:00