cleveragents-core/robot/tdd_quality_gate.robot

*** Settings ***
Documentation    Integration tests for the TDD quality gate script.
...
...              Exercises ``scripts/tdd_quality_gate.py`` end-to-end by
...              invoking it as a subprocess with various PR_DESCRIPTION
...              values and temporary file trees, then verifying exit codes
...              and output messages.
...
...              See CONTRIBUTING.md > Bug Fix Workflow for the full
...              specification.
Resource         ${CURDIR}/common.resource
Suite Setup      Setup Test Environment
Suite Teardown   Cleanup Test Environment

*** Variables ***
${HELPER}    ${CURDIR}/helper_tdd_quality_gate.py

*** Test Cases ***
# ===========================================================================
# PR description parsing
# ===========================================================================

Parse Single Fixes Reference
    [Documentation]    Verify the script extracts a single Fixes #N reference
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    parse_fixes_single
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    parse-fixes-single-ok

Parse Multiple Mixed References
    [Documentation]    Verify the script extracts multiple closing keywords
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    parse_mixed_refs
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    parse-mixed-refs-ok

Parse Issues Closed Block
    [Documentation]    Verify the script extracts ISSUES CLOSED: #N references
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    parse_issues_closed
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    parse-issues-closed-ok

Parse Non Closing Words
    [Documentation]    Verify parser ignores embedded keyword substrings
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    parse_non_closing_words
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    parse-non-closing-words-ok

No Bug References Passes Trivially
    [Documentation]    Verify the quality gate passes when no bug refs in PR
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    no_bug_refs_pass
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    no-bug-refs-pass-ok

# ===========================================================================
# TDD test search and tag verification
# ===========================================================================

Find TDD Test In Feature File
    [Documentation]    Verify the script finds @tdd_bug_N in .feature files
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    find_feature_test
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    find-feature-test-ok

Find TDD Test In Robot File
    [Documentation]    Verify the script finds tdd_bug_N in .robot files
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    find_robot_test
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    find-robot-test-ok

Find TDD Test Uses Exact Tag Match
    [Documentation]    Verify partial tags are not matched as exact bug tags
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    find_exact_tag_match
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    find-exact-tag-match-ok

No TDD Test Found Fails Gate
    [Documentation]    Verify the gate fails when no TDD test exists
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    no_tdd_test_fails
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    no-tdd-test-fails-ok

Expected Fail Tag Still Present Fails Gate
    [Documentation]    Verify the gate fails when @tdd_expected_fail is still present
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    expected_fail_present
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    expected-fail-present-ok

Expected Fail Tag Removed Passes Gate
    [Documentation]    Verify the gate passes when @tdd_expected_fail is removed
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    expected_fail_removed
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    expected-fail-removed-ok

# ===========================================================================
# Full quality gate integration
# ===========================================================================

Full Gate Multiple Bugs Mixed Results
    [Documentation]    Verify the gate handles multiple bugs with mixed outcomes
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    multi_bug_mixed
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    multi-bug-mixed-ok

Full Gate All Clean Passes
    [Documentation]    Verify the gate passes when all referenced bugs have clean tests
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    all_clean_passes
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    all-clean-passes-ok

TDD Tests In Both Behave And Robot
    [Documentation]    Verify the gate checks tests in both .feature and .robot files
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    both_behave_and_robot
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    both-behave-and-robot-ok

Diff Removal Is Required
    [Documentation]    Verify the gate fails when PR diff has no expected-fail removal
    [Tags]    ci    quality    tdd
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    diff_removal_required
    ...    cwd=${WORKSPACE}
    Should Be Equal As Integers    ${result.rc}    0    msg=${result.stderr}
    Should Contain    ${result.stdout}    diff-removal-required-ok