fix(test): tighten diagnostics check exit code assertion in CLI core test #900

New issue

Closed

opened 2026-03-13 23:47:42 +00:00 by freemo · 0 comments

freemo commented

2026-03-13 23:47:42 +00:00

Owner

Metadata

Commit Message: fix(test): tighten diagnostics check exit code assertion in CLI core test
Branch: fix/integration-cli-core-tolerant-rc

Background

The CLI core test (robot/cli_core.robot, 119 lines) contains a tolerant exit code check on line 107:

Should Be True    ${result.rc} == 0 or ${result.rc} == 1

This is for the diagnostics --check command. The intent may have been to accept both "no issues found" (rc=0) and "issues found" (rc=1), since --check semantics in many tools use rc=1 to indicate "check found problems." However, the test does not distinguish between:

rc=1 meaning "diagnostics found issues" (expected behavior, test should verify the issues)
rc=1 meaning "command crashed" (bug that should fail the test)

By accepting both codes, the test cannot detect a broken diagnostics command.

Acceptance Criteria

Line 107: Exit code check is tightened — either:
- (a) Assert rc == 0 if diagnostics should report no issues in the test environment, OR
- (b) If rc == 1 is expected (issues found), add assertions that verify the specific diagnostic output matches expected issues, so the test validates the content rather than just accepting any non-crash exit
The test clearly distinguishes between "diagnostics found issues" and "command failed"
nox -s integration_tests passes

Subtasks

Determine the expected behavior of diagnostics --check in the test environment (should it find issues or not?)
If rc=0 is expected: change to Should Be Equal As Integers ${result.rc} 0
If rc=1 is expected: keep rc=1 but add output content assertions that verify the specific diagnostics output
Run nox -s integration_tests and verify the test suite passes
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `fix(test): tighten diagnostics check exit code assertion in CLI core test` - **Branch**: `fix/integration-cli-core-tolerant-rc` ## Background The CLI core test (`robot/cli_core.robot`, 119 lines) contains a tolerant exit code check on line 107: ```robotframework Should Be True ${result.rc} == 0 or ${result.rc} == 1 ``` This is for the `diagnostics --check` command. The intent may have been to accept both "no issues found" (rc=0) and "issues found" (rc=1), since `--check` semantics in many tools use rc=1 to indicate "check found problems." However, the test does not distinguish between: - rc=1 meaning "diagnostics found issues" (expected behavior, test should verify the issues) - rc=1 meaning "command crashed" (bug that should fail the test) By accepting both codes, the test cannot detect a broken diagnostics command. ## Acceptance Criteria - [ ] Line 107: Exit code check is tightened — either: - (a) Assert `rc == 0` if diagnostics should report no issues in the test environment, OR - (b) If `rc == 1` is expected (issues found), add assertions that verify the specific diagnostic output matches expected issues, so the test validates the content rather than just accepting any non-crash exit - [ ] The test clearly distinguishes between "diagnostics found issues" and "command failed" - [ ] `nox -s integration_tests` passes ## Subtasks - [ ] Determine the expected behavior of `diagnostics --check` in the test environment (should it find issues or not?) - [ ] If rc=0 is expected: change to `Should Be Equal As Integers ${result.rc} 0` - [ ] If rc=1 is expected: keep rc=1 but add output content assertions that verify the specific diagnostics output - [ ] Run `nox -s integration_tests` and verify the test suite passes - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

Rows
Columns