191 lines
8.8 KiB
Gherkin
191 lines
8.8 KiB
Gherkin
Feature: Auto Debug CLI Command Coverage
|
|
As a developer
|
|
I want reliable coverage for the auto_debug CLI entry points
|
|
So that build automation paths remain stable
|
|
|
|
Background:
|
|
Given I have a clean auto_debug test environment
|
|
|
|
# auto_debug_command() coverage
|
|
Scenario: Auto debug command succeeds on first attempt
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service auto_debug_build will succeed
|
|
When I call the auto_debug_command programmatic interface
|
|
Then the auto_debug_command should return success True
|
|
And the attempts made should be 1
|
|
|
|
Scenario: Auto debug command fails after max attempts
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service auto_debug_build will fail
|
|
When I call the auto_debug_command with max_attempts 3
|
|
Then the auto_debug_command should return success False
|
|
And the attempts made should be 3
|
|
|
|
Scenario: Auto debug command with no project raises error
|
|
Given I have no project initialized
|
|
When I call the auto_debug_command programmatic interface expecting error
|
|
Then a CleverAgentsError should be raised with message "No project found"
|
|
|
|
# _get_current_project() coverage
|
|
Scenario: Get current project when project exists
|
|
Given I have an initialized project for auto_debug
|
|
When I call _get_current_project
|
|
Then the project should be returned successfully
|
|
|
|
Scenario: Get current project when no project exists
|
|
Given I have no project initialized
|
|
When I call _get_current_project expecting abort
|
|
Then typer.Abort should be raised
|
|
|
|
# run command happy path
|
|
Scenario: Auto debug run command with successful build
|
|
Given I have an initialized project for auto_debug
|
|
And the build will succeed with 5 changes
|
|
When I run the auto_debug run command
|
|
Then the command should exit with code 0
|
|
And the output should contain "Generated 5 change(s)"
|
|
And the output should contain "AutoDebug Complete"
|
|
|
|
Scenario: Auto debug run command with build failure then success
|
|
Given I have an initialized project for auto_debug
|
|
And the build will fail then succeed after fix
|
|
When I run the auto_debug run command with max_attempts 3
|
|
Then the command should exit with code 0
|
|
And the output should contain "Build failed"
|
|
And the output should contain "Fix generated"
|
|
And the output should contain "Build succeeded"
|
|
|
|
Scenario: Auto debug run command exceeds max attempts
|
|
Given I have an initialized project for auto_debug
|
|
And the build will always fail with error "Final failure"
|
|
When I run the auto_debug run command with max_attempts 2
|
|
Then the command should exit with code 1
|
|
And the output should contain "Build failed after"
|
|
And the output should contain "Final error"
|
|
And the output should contain "Final failure"
|
|
|
|
Scenario: Auto debug run command fails with empty error message
|
|
Given I have an initialized project for auto_debug
|
|
And the build will always fail with empty error message
|
|
When I run the auto_debug run command with max_attempts 2
|
|
Then the command should exit with code 1
|
|
And the output should contain "Build failed after"
|
|
And the output should not contain "Final error"
|
|
|
|
Scenario: Auto debug run command handles PlanError with details
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service will raise a PlanError with details
|
|
When I run the auto_debug run command
|
|
Then the command should be aborted
|
|
And the output should contain "Plan Error"
|
|
And the output should contain "phase:"
|
|
|
|
Scenario: Auto debug run command handles PlanError without details
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service will raise a PlanError without details
|
|
When I run the auto_debug run command
|
|
Then the command should be aborted
|
|
And the output should contain "Plan Error"
|
|
And the output should not contain "phase:"
|
|
|
|
Scenario: Auto debug run command handles CleverAgentsError
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service will raise a CleverAgentsError
|
|
When I run the auto_debug run command
|
|
Then the command should be aborted
|
|
And the output should contain "Error:"
|
|
|
|
Scenario: Auto debug run command handles fix generation error
|
|
Given I have an initialized project for auto_debug
|
|
And the build will fail with fix generation error
|
|
When I run the auto_debug run command with max_attempts 2
|
|
Then the command should exit with code 0
|
|
And the output should contain "Could not generate fix"
|
|
|
|
# Edge case: Long error message truncation
|
|
Scenario: Auto debug run command truncates long error messages
|
|
Given I have an initialized project for auto_debug
|
|
And the build will always fail with error "This is a very long error message that exceeds eighty characters and should be truncated in the display output"
|
|
When I run the auto_debug run command with max_attempts 1
|
|
Then the command should exit with code 1
|
|
And the output should contain "..."
|
|
And the output should contain "Build failed after"
|
|
|
|
# Edge case: Minimum max_attempts boundary
|
|
Scenario: Auto debug run command with minimum max_attempts of 1
|
|
Given I have an initialized project for auto_debug
|
|
And the build will always fail with error "Single attempt failure"
|
|
When I run the auto_debug run command with max_attempts 1
|
|
Then the command should exit with code 1
|
|
And the output should contain "Attempt 1/1"
|
|
And the output should contain "Build failed after 1 attempt"
|
|
|
|
# Edge case: Multiple sequential failures before success
|
|
Scenario: Auto debug run command succeeds after multiple failures
|
|
Given I have an initialized project for auto_debug
|
|
And the build will fail 2 times then succeed with 3 changes
|
|
When I run the auto_debug run command with max_attempts 5
|
|
Then the command should exit with code 0
|
|
And the output should contain "Build failed"
|
|
And the output should contain "Build successful"
|
|
And the output should contain "Generated 3 change(s)"
|
|
|
|
# Edge case: Short error message (no truncation)
|
|
Scenario: Auto debug run command shows short error without truncation
|
|
Given I have an initialized project for auto_debug
|
|
And the build will always fail with error "Short error"
|
|
When I run the auto_debug run command with max_attempts 1
|
|
Then the command should exit with code 1
|
|
And the output should contain "Short error"
|
|
And the output should not contain "Short error..."
|
|
|
|
# Edge case: PlanError re-raised from within build loop
|
|
Scenario: Auto debug run command re-raises PlanError from build
|
|
Given I have an initialized project for auto_debug
|
|
And the build will raise PlanError after one attempt
|
|
When I run the auto_debug run command with max_attempts 3
|
|
Then the command should be aborted
|
|
And the output should contain "Plan Error"
|
|
|
|
# Edge case: CleverAgentsError re-raised from within build loop
|
|
Scenario: Auto debug run command re-raises CleverAgentsError from build
|
|
Given I have an initialized project for auto_debug
|
|
And the build will raise CleverAgentsError after one attempt
|
|
When I run the auto_debug run command with max_attempts 3
|
|
Then the command should be aborted
|
|
And the output should contain "Error:"
|
|
|
|
# Edge case: auto_debug_command with custom max_attempts
|
|
Scenario: Auto debug command with custom max_attempts of 5
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service auto_debug_build will fail
|
|
When I call the auto_debug_command with max_attempts 5
|
|
Then the auto_debug_command should return success False
|
|
And the attempts made should be 5
|
|
|
|
# Edge case: auto_debug_command with max_attempts of 1
|
|
Scenario: Auto debug command with minimum max_attempts of 1
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service auto_debug_build will fail
|
|
When I call the auto_debug_command with max_attempts 1
|
|
Then the auto_debug_command should return success False
|
|
And the attempts made should be 1
|
|
|
|
# Edge case: Successful build on exact last attempt
|
|
Scenario: Auto debug run command succeeds on the last attempt
|
|
Given I have an initialized project for auto_debug
|
|
And the build will fail 2 times then succeed with 1 changes
|
|
When I run the auto_debug run command with max_attempts 3
|
|
Then the command should exit with code 0
|
|
And the output should contain "Build succeeded after 3 attempt"
|
|
|
|
# Edge case: Multiple PlanError details
|
|
Scenario: Auto debug run command handles PlanError with multiple details
|
|
Given I have an initialized project for auto_debug
|
|
And the plan service will raise a PlanError with multiple details
|
|
When I run the auto_debug run command
|
|
Then the command should be aborted
|
|
And the output should contain "Plan Error"
|
|
And the output should contain "phase:"
|
|
And the output should contain "step:"
|