Files
cleveragents-core/features/error_recovery_coverage_boost.feature
2026-02-25 10:05:10 +00:00

176 lines
8.8 KiB
Gherkin

Feature: Error recovery coverage boost
As a developer
I want to cover uncovered lines in plan CLI errors/diff/artifacts commands
and plan executor edge cases introduced by the error recovery feature
So that overall code coverage exceeds the 97% threshold
Background:
Given an errcov test environment
# -- plan errors CLI (plan.py:1621-1720) ---------------------------------
Scenario: Plan errors command rich output with full error details
Given an errcov plan with error details category "transient" phase "execute"
And the errcov plan has error message "Connection timed out after 30s"
And the errcov plan error details include actor "test-actor" and tool "test-tool"
And the errcov plan error details include stack summary "File test.py line 10"
When I invoke errcov plan errors in "rich" format
Then the errcov errors output should contain "Plan Errors"
And the errcov errors output should contain "transient"
And the errcov errors output should contain "execute"
And the errcov errors output should contain "test-actor"
And the errcov errors output should contain "test-tool"
And the errcov errors output should contain "File test.py"
And the errcov errors output should contain "Recovery Suggestions"
Scenario: Plan errors command plain output with error details
Given an errcov plan with error details category "validation" phase "apply"
And the errcov plan has error message "Schema validation failed"
When I invoke errcov plan errors in "plain" format
Then the errcov errors output should contain "validation"
And the errcov errors output should contain "apply"
And the errcov errors output should contain "Schema validation failed"
Scenario: Plan errors command JSON output
Given an errcov plan with error details category "transient" phase "execute"
And the errcov plan has error message "Timeout"
When I invoke errcov plan errors in "json" format
Then the errcov errors output should contain "transient"
And the errcov errors output should contain "plan_id"
Scenario: Plan errors command rich output with no errors and no message
Given an errcov plan with no error details and no error message
When I invoke errcov plan errors in "rich" format
Then the errcov errors output should contain "No error recovery records"
Scenario: Plan errors command rich output with error message but no recovery
Given an errcov plan with no error details but has error message "Something broke"
When I invoke errcov plan errors in "rich" format
Then the errcov errors output should contain "Something broke"
Scenario: Plan errors command handles CleverAgentsError
Given an errcov lifecycle service that raises CleverAgentsError on get_plan
When I invoke errcov plan errors expecting abort
Then the errcov errors output should contain "Error:"
Scenario: Plan errors command plain output with actor and tool and stack
Given an errcov plan with error details category "provider" phase "execute"
And the errcov plan error details include actor "gpt4-actor" and tool "code-gen"
And the errcov plan error details include stack summary "traceback info"
And the errcov plan error details include retry count "2" max "3" retriable "true"
When I invoke errcov plan errors in "plain" format
Then the errcov errors output should contain "gpt4-actor"
And the errcov errors output should contain "code-gen"
And the errcov errors output should contain "traceback info"
And the errcov errors output should contain "recovery_hints"
# -- plan diff CLI (plan.py:1969-1979) -----------------------------------
Scenario: Plan diff command calls apply service diff
Given an errcov apply service returning diff text "--- a/file.py"
When I invoke errcov plan diff in "rich" format
Then the errcov output should contain "--- a/file.py"
Scenario: Plan diff command handles PlanError
Given an errcov apply service that raises PlanError on diff
When I invoke errcov plan diff expecting abort
Then the errcov output should contain "Diff Error:"
Scenario: Plan diff command handles CleverAgentsError
Given an errcov apply service that raises CleverAgentsError on diff
When I invoke errcov plan diff expecting abort
Then the errcov output should contain "Error:"
# -- plan artifacts CLI (plan.py:2002-2012) ------------------------------
Scenario: Plan artifacts command calls apply service artifacts
Given an errcov apply service returning artifacts text "ChangeSet: cs_001"
When I invoke errcov plan artifacts in "rich" format
Then the errcov output should contain "ChangeSet: cs_001"
Scenario: Plan artifacts command handles PlanError
Given an errcov apply service that raises PlanError on artifacts
When I invoke errcov plan artifacts expecting abort
Then the errcov output should contain "Artifacts Error:"
Scenario: Plan artifacts command handles CleverAgentsError
Given an errcov apply service that raises CleverAgentsError on artifacts
When I invoke errcov plan artifacts expecting abort
Then the errcov output should contain "Error:"
# -- _get_apply_service (plan.py:1941-1946) ------------------------------
Scenario: Get apply service constructs PlanApplyService
When I invoke errcov _get_apply_service
Then the errcov apply service should be a PlanApplyService instance
# -- PlanExecutor edge cases (plan_executor.py) --------------------------
Scenario: Strategize with empty plan_id raises ValidationError
When I invoke errcov strategize with empty plan_id
Then an errcov ValidationError should be raised with "plan_id must not be empty"
Scenario: Execute stub actor with empty plan_id raises ValidationError
When I invoke errcov execute stub with empty plan_id
Then an errcov ValidationError should be raised with "plan_id must not be empty"
Scenario: Run execute with wrong phase raises PlanError
Given an errcov plan in strategize phase
When I invoke errcov run_execute on wrong-phase plan
Then an errcov PlanError should be raised containing "not in Execute phase"
Scenario: Run execute with wrong state raises PlanError
Given an errcov plan in execute phase but processing state
When I invoke errcov run_execute on wrong-state plan
Then an errcov PlanError should be raised containing "not queued"
Scenario: Parse definition of done skips blank lines
When I parse errcov definition of done with blank lines
Then the parsed steps should not contain empty entries
Scenario: Parse definition of done handles numbered prefixes
When I parse errcov definition of done with numbered items "1. First\n2) Second\n3. Third"
Then the errcov parsed steps should be "First" and "Second" and "Third"
Scenario: Strategize error recording with recovery service
Given an errcov executor with error recovery service
And the errcov strategize actor raises RuntimeError "strategize failed"
When I invoke errcov run_strategize expecting failure
Then the errcov error recovery should have recorded a strategize error
Scenario: Execute retry loop exhaustion without recovery service
Given an errcov executor without error recovery service
And the errcov execute actor always fails with "total failure"
When I invoke errcov run_execute expecting failure
Then the errcov lifecycle should have called fail_execute
# -- error_recovery.py policy edge cases ---------------------------------
Scenario: Policy should_retry returns false for exhausted retriable error
Given an errcov policy with threshold 0.0 and max retries 3
And an errcov retriable error record with retry count 3 max 3
Then the errcov policy should not recommend retry
Scenario: Policy should_escalate returns true for exhausted retriable error
Given an errcov policy with threshold 0.0 and max retries 3
And an errcov retriable error record with retry count 3 max 3
Then the errcov policy should recommend escalation
Scenario: Policy format_recovery_output shows hint cli commands
Given an errcov policy with threshold 0.0 and max retries 3
And an errcov retriable error record with hints retry count 0 max 3
When I format errcov recovery output
Then the errcov recovery output should contain "agents plan"
# -- plan.py branch: actor_registry and testing_mode (647->649, 649->653)
Scenario: Plan use command with actor registry present
Given an errcov plan creation environment with actor registry
When I invoke errcov plan use with actor registry
Then the errcov actor registry ensure_built_in_actors should have been called
Scenario: Plan use command with testing mode enabled
Given an errcov plan creation environment with testing mode
When I invoke errcov plan use with testing mode
Then the errcov container actor_service should have been called