test: restore and enhance e2e test coverage #11142

Open
freemo wants to merge 1 commits from restore-e2e-cleanup into master
4 changed files with 134 additions and 0 deletions
+1
View File
@@ -4,6 +4,7 @@
## Unreleased
- Restored deleted E2E test files (`features/cli_main_cov3.feature`, `robot/plan_diff_artifacts.robot`) removed by commit 8ea00f51 which covered CLI main line coverage and plan diff/artifacts workflows. Step definitions and helper utilities were already present in the codebase. Resolves AUTO-WDOG announcement #8490 and blocks milestone acceptance criteria validation for v3.2.0.
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
requires whole-word closing keywords (avoids false positives like
"prefixes #12"), TDD bug tag discovery now uses exact token matching
+1
View File
@@ -5,6 +5,7 @@
* HAL 9000 <hal9000@cleverthis.com>
* Hamza Khyari <hamza.khyari@cleverthis.com>
* Jeffrey Phillips Freeman <jeffrey.freeman@syncleus.com>
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
* Luis Mendes <luis.p.mendes@gmail.com>
* Rui Hu <rui.hu@cleverthis.com>
+65
View File
@@ -0,0 +1,65 @@
Feature: CLI main.py coverage round 3
Review

SUGGESTION — Consider tagging for CI classification: The feature file and its scenarios are well-formed. Consider whether any scenarios should be tagged (e.g. @slow, @wip) to support selective test execution. Minor point — not blocking.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

**SUGGESTION — Consider tagging for CI classification**: The feature file and its scenarios are well-formed. Consider whether any scenarios should be tagged (e.g. `@slow`, `@wip`) to support selective test execution. Minor point — not blocking. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
As a developer
I want to cover remaining uncovered lines in cleveragents.cli.main
So that code coverage gaps on lines 104-110, 582-590, and 593-594 are closed
# -------------------------------------------------------------------
# Lines 104-110: _register_subcommands exception handler
# -------------------------------------------------------------------
Scenario: _register_subcommands handles import failure gracefully
Given cmcov3 the _subcommands_registered flag is reset to False
When cmcov3 _register_subcommands is called and an import raises an error
Then cmcov3 the error console should have printed the failure message
And cmcov3 the _subcommands_registered flag should still be False
# -------------------------------------------------------------------
# Lines 582-586: completion command with subprocess returning stdout
# -------------------------------------------------------------------
Scenario: completion command outputs subprocess stdout when present
Given cmcov3 subprocess.run is mocked to return stdout "echo hello"
When cmcov3 the completion command is invoked with shell "bash"
Then cmcov3 the output should contain "echo hello"
# -------------------------------------------------------------------
# Lines 587-590: completion command with empty subprocess stdout
# -------------------------------------------------------------------
Scenario: completion command outputs placeholder when subprocess has no stdout
Given cmcov3 subprocess.run is mocked to return empty stdout
When cmcov3 the completion command is invoked with shell "zsh"
Then cmcov3 the output should contain "# Completion script for zsh"
And cmcov3 the output should contain "# Shell: zsh"
# -------------------------------------------------------------------
# Lines 593-594+: convert_exit_code function coverage
# -------------------------------------------------------------------
Scenario: convert_exit_code returns 0 for None input
When cmcov3 convert_exit_code is called with None
Then cmcov3 the exit code result should be 0
Scenario: convert_exit_code extracts exit_code attribute from object
When cmcov3 convert_exit_code is called with an object having exit_code 42
Then cmcov3 the exit code result should be 42
Scenario: convert_exit_code clamps large positive code to 255
When cmcov3 convert_exit_code is called with integer 999
Then cmcov3 the exit code result should be 255
Scenario: convert_exit_code passes through negative codes
When cmcov3 convert_exit_code is called with integer -1
Then cmcov3 the exit code result should be -1
Scenario: convert_exit_code returns 1 for non-integer string
When cmcov3 convert_exit_code is called with a non-convertible value
Then cmcov3 the exit code result should be 1
Scenario: convert_exit_code handles zero correctly
When cmcov3 convert_exit_code is called with integer 0
Then cmcov3 the exit code result should be 0
Scenario: convert_exit_code handles normal positive code
When cmcov3 convert_exit_code is called with integer 7
Then cmcov3 the exit code result should be 7
+67
View File
@@ -0,0 +1,67 @@
*** Settings ***
Review

BLOCKING — Undisclosed ci.yml scope creep: This file is correctly placed and well-structured. However, the overall PR diff vs master includes a large undisclosed re-introduction of CI jobs (tdd_quality_gate, e2e_tests) in ci.yml that is not mentioned in the PR description. This creates a misleading review surface where a reviewer is told to evaluate a 4-file diff but is actually reviewing 17+ file changes. Please update the PR description to accurately reflect all changes, including the ci.yml modifications.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

**BLOCKING — Undisclosed ci.yml scope creep**: This file is correctly placed and well-structured. However, the overall PR diff vs `master` includes a large undisclosed re-introduction of CI jobs (`tdd_quality_gate`, `e2e_tests`) in `ci.yml` that is not mentioned in the PR description. This creates a misleading review surface where a reviewer is told to evaluate a 4-file diff but is actually reviewing 17+ file changes. Please update the PR description to accurately reflect all changes, including the ci.yml modifications. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Documentation Integration tests for plan diff and artifacts output.
... Tests D0b.apply features: diff rendering, artifacts summary,
... empty changeset guard, apply summary persistence, and merge failure handling.
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER_SCRIPT} robot/helper_plan_diff_artifacts.py
*** Test Cases ***
Plan Diff Renders Changeset Summary
[Documentation] Verify plan diff shows file changes grouped by path
[Tags] diff plan changeset
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} diff-output cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} diff-output-ok
Plan Artifacts Shows Changeset Metadata
[Documentation] Verify plan artifacts shows changeset ID, sandbox refs, and file list
[Tags] artifacts plan changeset
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} artifacts-output cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} artifacts-output-ok
Empty Changeset Guard Blocks Apply
[Documentation] Verify apply is blocked when changeset is empty
[Tags] guard plan apply
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} empty-guard cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} empty-guard-ok
Empty Changeset Guard Allows With Flag
[Documentation] Verify apply proceeds when --allow-empty is set
[Tags] guard plan apply
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} empty-guard-allow cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} empty-guard-allow-ok
Apply Summary Persistence
[Documentation] Verify apply summary metadata is persisted into plan
[Tags] apply plan metadata
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} apply-summary cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} apply-summary-ok
Merge Failure Handling
[Documentation] Verify merge failure sets plan to errored with conflict details
[Tags] merge plan error
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} merge-failure cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} merge-failure-ok
Diff JSON Format Output
[Documentation] Verify diff JSON format includes entries and summary
[Tags] diff plan json
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} diff-json cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} diff-json-ok
Diff YAML Format Output
[Documentation] Verify diff YAML format includes changeset metadata
[Tags] diff plan yaml
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} diff-yaml cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} diff-yaml-ok