brent.edwards
a5cc81354d
CI / security (pull_request) Successful in 51s
CI / build (pull_request) Successful in 14s
CI / helm (pull_request) Successful in 22s
CI / lint (pull_request) Successful in 3m26s
CI / typecheck (pull_request) Successful in 3m54s
CI / quality (pull_request) Successful in 3m39s
CI / integration_tests (pull_request) Successful in 6m34s
CI / unit_tests (pull_request) Successful in 6m41s
CI / docker (pull_request) Successful in 1m21s
CI / coverage (pull_request) Successful in 9m10s
CI / e2e_tests (pull_request) Successful in 25m30s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 55m10s
Add --required and --informational as mutually exclusive boolean options to the `agents validation add` CLI command. When specified, they override the `mode` field from the YAML config file. This aligns the CLI with the specification (specification.md line 22339) which states the validation mode can be set "via --required/--informational on agents validation add". The fix resolves the spec contradiction by: - Implementing the flags in the CLI (per Core Concepts > Validation Mode) - Updating the formal CLI reference to include the new flags - Adding an exception to Design Principle #3 for validation add - Removing the spurious positional name argument from the walkthrough TDD tests from #1102 now run normally with @tdd_expected_fail removed. New edge-case tests cover mutual exclusivity (both flags rejected). Also excludes tool/wrapping.py from semgrep exec/compile rules since that module intentionally uses exec() in a controlled sandbox for the validation transform feature. ISSUES CLOSED: #1038
75 lines
4.1 KiB
Plaintext
75 lines
4.1 KiB
Plaintext
*** Settings ***
|
|
Documentation TDD Bug #1038 — validation add --required/--informational flags
|
|
... Integration smoke tests verifying that the ``agents validation add``
|
|
... command accepts ``--required`` and ``--informational`` flags as described
|
|
... in the specification (specification.md line 22339).
|
|
...
|
|
... Bug #1038 reported that these flags were missing from the CLI,
|
|
... causing a ``NoSuchOption`` error. The fix adds them as mutually
|
|
... exclusive boolean options that override the YAML config mode.
|
|
...
|
|
... These tests were originally tagged tdd_expected_fail while the bug
|
|
... was unfixed. Now that the fix is in place, the tag has been removed
|
|
... and the tests run normally as permanent regression guards.
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_tdd_validation_required_flag.py
|
|
|
|
*** Test Cases ***
|
|
TDD Validation Add Required Flag Accepted
|
|
[Documentation] Verify that ``validation add --config <file> --required``
|
|
... is accepted by the CLI and sets the validation mode to
|
|
... ``required``.
|
|
[Tags] tdd_issue tdd_issue_1038
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-required cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tdd-validation-required-flag-ok
|
|
|
|
TDD Validation Add Informational Flag Accepted
|
|
[Documentation] Verify that ``validation add --config <file> --informational``
|
|
... is accepted by the CLI and sets the validation mode to
|
|
... ``informational``.
|
|
[Tags] tdd_issue tdd_issue_1038
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-informational cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tdd-validation-informational-flag-ok
|
|
|
|
TDD Validation Add Required Flag Overrides YAML Config
|
|
[Documentation] Verify that ``--required`` overrides a YAML config that
|
|
... specifies ``mode: informational``. Both CLI output and
|
|
... the service layer should reflect ``mode: required``.
|
|
[Tags] tdd_issue tdd_issue_1038
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-required-overrides-config cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tdd-validation-required-overrides-config-ok
|
|
|
|
TDD Validation Add Informational Flag Overrides YAML Config
|
|
[Documentation] Verify that ``--informational`` overrides a YAML config
|
|
... that specifies ``mode: required``. Both CLI output and
|
|
... the service layer should reflect ``mode: informational``.
|
|
[Tags] tdd_issue tdd_issue_1038
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-informational-overrides-config cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tdd-validation-informational-overrides-config-ok
|
|
|
|
TDD Validation Add Both Flags Rejected
|
|
[Documentation] Verify that passing both ``--required`` and ``--informational``
|
|
... simultaneously is rejected with a mutually-exclusive error.
|
|
[Tags] tdd_issue tdd_issue_1038
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-both-flags-rejected cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tdd-validation-both-flags-rejected-ok
|