hurui200320
1878998b7a
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 25s
CI / lint (pull_request) Successful in 3m19s
CI / typecheck (pull_request) Successful in 3m56s
CI / security (pull_request) Successful in 4m6s
CI / quality (pull_request) Successful in 4m10s
CI / integration_tests (pull_request) Successful in 7m20s
CI / unit_tests (pull_request) Successful in 7m44s
CI / docker (pull_request) Successful in 1m8s
CI / e2e_tests (pull_request) Successful in 13m1s
CI / coverage (pull_request) Successful in 12m25s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 26s
CI / lint (push) Successful in 3m31s
CI / quality (push) Successful in 3m40s
CI / typecheck (push) Successful in 3m57s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m1s
CI / integration_tests (push) Successful in 8m55s
CI / unit_tests (push) Successful in 9m14s
CI / docker (push) Successful in 9s
CI / e2e_tests (push) Successful in 11m42s
CI / coverage (push) Successful in 11m37s
CI / status-check (push) Successful in 4s
CI / benchmark-publish (push) Successful in 29m25s
CI / benchmark-regression (pull_request) Successful in 54m8s
Rename the TDD tag system from tdd_bug/tdd_bug_<N> to tdd_issue/tdd_issue_<N> across the entire codebase. The tdd_expected_fail tag is unchanged. The TDD expected-failure workflow is not limited to bug fixes — it applies equally to any issue type (features, tasks, refactors). The _bug suffix was misleading and narrowed the perceived scope. The new _issue suffix accurately reflects that the TDD tagging system applies to any Forgejo issue. Changes span 92 files: - features/environment.py: validate_tdd_tags(), should_invert_result(), and apply_tdd_inversion() updated — regex, variables, error messages - robot/tdd_expected_fail_listener.py: _validate_tdd_tags(), _should_invert_result(), start_test(), end_test() updated consistently - 33 Behave .feature files: all @tdd_bug/@tdd_bug_<N> tags renamed - 29 Robot .robot files: all tdd_bug/tdd_bug_<N> tags renamed - 3 Robot fixture files renamed (tdd_bug_alone, tdd_missing_tdd_bug, tdd_expected_fail_missing_bug_n) with content and references updated - Tag validation tests and helpers updated (function names, command dispatch keys, output strings, fixture references) - CONTRIBUTING.md: section renamed from 'TDD Bug Test Tags' to 'TDD Issue Test Tags', all tag references and examples updated - noxfile.py: comment references updated - Step definition files, mock helpers, and benchmark files: docstring references updated ISSUES CLOSED: #965
166 lines
9.3 KiB
Gherkin
166 lines
9.3 KiB
Gherkin
@infrastructure
|
|
Feature: TDD expected-fail handler infrastructure
|
|
Verify that ``apply_tdd_inversion`` (the production code path called by the
|
|
``Scenario.run()`` monkey-patch) and ``handle_tdd_expected_fail`` (the
|
|
standalone entry point) correctly invert scenario AND step-level status for
|
|
TDD issue-capture tests.
|
|
|
|
# --- apply_tdd_inversion (production path) ---
|
|
|
|
Scenario: apply_tdd_inversion inverts a failed scenario with failed and skipped steps to passed
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
And the mock scenario has a step "skipped step" with status "skipped"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "passed"
|
|
And the step "broken step" should have status "passed"
|
|
And the step "skipped step" should have status "passed"
|
|
And the step "broken step" should have error_message cleared
|
|
And the step "skipped step" should have error_message cleared
|
|
|
|
Scenario: apply_tdd_inversion fails a passed scenario that still carries @tdd_expected_fail
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "passed"
|
|
And the mock scenario has a step "last step" with status "passed"
|
|
When apply_tdd_inversion processes the scenario with failed False
|
|
Then the scenario status should be "failed"
|
|
And the step "last step" should have a synthetic error_message
|
|
|
|
Scenario: apply_tdd_inversion does not invert when hook_failed is set
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed" and hook_failed
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "failed"
|
|
And the step "broken step" should have status "failed"
|
|
|
|
Scenario: apply_tdd_inversion does not invert during dry-run mode
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "passed" and was_dry_run
|
|
When apply_tdd_inversion processes the scenario with failed False
|
|
Then the apply_tdd_inversion result should be False
|
|
|
|
Scenario: apply_tdd_inversion does not invert non-AssertionError exceptions
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has an infrastructure-error step "infra step" with exception "RuntimeError"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "failed"
|
|
And the step "infra step" should have status "failed"
|
|
|
|
Scenario: apply_tdd_inversion ignores scenarios without @tdd_expected_fail
|
|
Given a mock scenario tagged "@tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "failed"
|
|
And the step "broken step" should have status "failed"
|
|
|
|
# --- handle_tdd_expected_fail (standalone entry point) ---
|
|
|
|
Scenario: Handler inverts a failed scenario with failed and skipped steps to passed
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
And the mock scenario has a step "skipped step" with status "skipped"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "passed"
|
|
And the step "broken step" should have status "passed"
|
|
And the step "skipped step" should have status "passed"
|
|
And the step "broken step" should have error_message cleared
|
|
And the step "skipped step" should have error_message cleared
|
|
|
|
Scenario: Handler fails a passed scenario that still carries @tdd_expected_fail
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "passed"
|
|
And the mock scenario has a step "last step" with status "passed"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
And the step "last step" should have a synthetic error_message
|
|
|
|
Scenario: Handler rejects @tdd_expected_fail without @tdd_issue
|
|
Given a mock scenario tagged "@tdd_expected_fail" with status "failed"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
|
|
Scenario: Handler rejects @tdd_expected_fail without @tdd_issue_N
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue" with status "failed"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
|
|
Scenario: Handler rejects @tdd_issue_N without @tdd_issue unconditionally
|
|
Given a mock scenario tagged "@tdd_issue_999" with status "passed"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
|
|
Scenario: Handler ignores scenarios without @tdd_expected_fail
|
|
Given a mock scenario tagged "@tdd_issue @tdd_issue_999" with status "failed"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
|
|
Scenario: Handler does not invert non-AssertionError exceptions
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has an infrastructure-error step "infra step" with exception "RuntimeError"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
And the step "infra step" should have status "failed"
|
|
|
|
# --- apply_tdd_inversion: mixed exception types (S3) ---
|
|
|
|
Scenario: apply_tdd_inversion does not invert when mixed exceptions present
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "bug step" with status "failed"
|
|
And the mock scenario has an infrastructure-error step "infra step" with exception "RuntimeError"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "failed"
|
|
And the step "bug step" should have status "failed"
|
|
And the step "infra step" should have status "failed"
|
|
|
|
Scenario: Handler does not invert when mixed exceptions present
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "bug step" with status "failed"
|
|
And the mock scenario has an infrastructure-error step "infra step" with exception "RuntimeError"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "failed"
|
|
And the step "bug step" should have status "failed"
|
|
And the step "infra step" should have status "failed"
|
|
|
|
# --- selective step reset preserves already-passed steps (TC-3) ---
|
|
|
|
Scenario: apply_tdd_inversion preserves already-passed steps during inversion
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "ok step" with status "passed"
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
And the mock scenario has a step "skipped step" with status "skipped"
|
|
When apply_tdd_inversion processes the scenario with failed True
|
|
Then the scenario status should be "passed"
|
|
And the step "ok step" should have status "passed"
|
|
And the step "broken step" should have status "passed"
|
|
And the step "skipped step" should have status "passed"
|
|
And the step "broken step" should have error_message cleared
|
|
And the step "skipped step" should have error_message cleared
|
|
|
|
Scenario: Handler preserves already-passed steps during inversion
|
|
Given a mock scenario tagged "@tdd_expected_fail @tdd_issue @tdd_issue_999" with status "failed"
|
|
And the mock scenario has a step "ok step" with status "passed"
|
|
And the mock scenario has a step "broken step" with status "failed"
|
|
And the mock scenario has a step "skipped step" with status "skipped"
|
|
When the TDD expected-fail handler processes the scenario
|
|
Then the scenario status should be "passed"
|
|
And the step "ok step" should have status "passed"
|
|
And the step "broken step" should have status "passed"
|
|
And the step "skipped step" should have status "passed"
|
|
And the step "broken step" should have error_message cleared
|
|
And the step "skipped step" should have error_message cleared
|
|
|
|
# --- before_scenario hook_failed regression test (TC-2) ---
|
|
|
|
Scenario: before_scenario sets hook_failed on invalid TDD tags
|
|
Given a mock scenario tagged "@tdd_expected_fail" with status "untested"
|
|
When before_scenario is called with the mock scenario
|
|
Then the scenario hook_failed flag should be True
|
|
And the scenario status should be "failed"
|
|
|
|
# --- _install_tdd_expected_fail_patch (S2) ---
|
|
|
|
Scenario: _install_tdd_expected_fail_patch sets the patched flag
|
|
When the TDD expected-fail patch installation status is checked
|
|
Then the Scenario class should have _tdd_run_patched set to True
|
|
|
|
Scenario: _install_tdd_expected_fail_patch is idempotent
|
|
When _install_tdd_expected_fail_patch is called twice
|
|
Then the Scenario.run method should not be double-wrapped
|