fix(testing): document and harden non-AssertionError guard in apply_tdd_inversion to reduce flaky CI
CI / lint (pull_request) Successful in 18s
CI / build (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 18s
CI / quality (pull_request) Successful in 53s
CI / typecheck (pull_request) Successful in 56s
CI / security (pull_request) Successful in 57s
CI / push-validation (pull_request) Successful in 40s
CI / unit_tests (pull_request) Successful in 3m13s
CI / integration_tests (pull_request) Successful in 4m23s
CI / e2e_tests (pull_request) Successful in 4m37s
CI / docker (pull_request) Successful in 1m34s
CI / coverage (pull_request) Successful in 10m46s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 17s
CI / quality (push) Successful in 17s
CI / build (push) Successful in 24s
CI / helm (push) Successful in 24s
CI / push-validation (push) Successful in 37s
CI / typecheck (push) Successful in 52s
CI / security (push) Successful in 52s
CI / e2e_tests (push) Successful in 3m13s
CI / unit_tests (push) Successful in 6m37s
CI / integration_tests (push) Successful in 6m39s
CI / docker (push) Successful in 1m35s
CI / coverage (push) Successful in 11m13s
CI / status-check (push) Successful in 1s

Surface the non-AssertionError guard warning in standard Behave output by emitting to stderr in addition to the structured logger, and add infrastructure coverage that asserts this guard path is visible during test runs. Document the @tdd_expected_fail expectation that bug-signaling failures must use AssertionError so infrastructure exceptions are not accidentally treated as expected bug failures.

ISSUES CLOSED: #8294
This commit was merged in pull request #8325.
This commit is contained in:
2026-04-13 08:45:14 +00:00
parent 237e776951
commit f67e8a2e07
5 changed files with 102 additions and 5 deletions
@@ -10,6 +10,9 @@ exceptions).
from __future__ import annotations
from contextlib import redirect_stderr
from io import StringIO
from behave import given, then, when
from behave.model import Scenario, Status, Step
from behave.runner import Context
@@ -125,6 +128,19 @@ def step_run_apply_inversion_failed(context: Context) -> None:
)
@when(
"apply_tdd_inversion processes the scenario with failed True while capturing stderr"
)
def step_run_apply_inversion_failed_capture_stderr(context: Context) -> None:
"""Invoke ``apply_tdd_inversion`` and capture stderr output."""
stderr_buffer = StringIO()
with redirect_stderr(stderr_buffer):
context.apply_inversion_result = apply_tdd_inversion(
context.mock_scenario, failed=True
)
context.captured_stderr = stderr_buffer.getvalue()
@when("apply_tdd_inversion processes the scenario with failed False")
def step_run_apply_inversion_not_failed(context: Context) -> None:
"""Invoke ``apply_tdd_inversion`` with ``failed=False``."""
@@ -151,6 +167,32 @@ def step_check_inversion_result_true(context: Context) -> None:
)
@then('stderr output should contain "{expected_text}"')
def step_check_stderr_contains(context: Context, expected_text: str) -> None:
"""Assert captured stderr output contains the expected text."""
assert hasattr(context, "captured_stderr"), (
"context.captured_stderr was not set — use the "
"'while capturing stderr' When step to capture stderr before asserting."
)
stderr_output: str = context.captured_stderr
assert expected_text in stderr_output, (
f"Expected stderr to contain {expected_text!r}, got: {stderr_output!r}"
)
@then('stderr output should not contain "{expected_text}"')
def step_check_stderr_not_contains(context: Context, expected_text: str) -> None:
"""Assert captured stderr output does not contain the given text."""
assert hasattr(context, "captured_stderr"), (
"context.captured_stderr was not set — use the "
"'while capturing stderr' When step to capture stderr before asserting."
)
stderr_output: str = context.captured_stderr
assert expected_text not in stderr_output, (
f"Expected stderr NOT to contain {expected_text!r}, got: {stderr_output!r}"
)
# ---------------------------------------------------------------------------
# handle_tdd_expected_fail steps (standalone entry point)
# ---------------------------------------------------------------------------