Feature: Auto Debug Agent Coverage As a developer I want comprehensive test coverage for the AutoDebugAgent So that I can ensure the auto-debug workflow works correctly Background: Given the auto debug agent module is importable And I have a mock LLM provider configured for auto debug And logging is enabled at INFO level for auto debug Scenario: AutoDebugAgent can be instantiated with default parameters When I create an AutoDebugAgent with default parameters Then the agent should be initialized successfully for auto debug And the agent should have a max_fix_attempts attribute set to 3 And the agent should have an llm provider configured for auto debug Scenario: AutoDebugAgent can be instantiated with custom parameters When I create an AutoDebugAgent with parameters: | parameter | value | | provider | openai | | model | gpt-4 | | temperature | 0.3 | | max_fix_attempts | 5 | Then the agent should be initialized successfully for auto debug And the agent max_fix_attempts should be 5 And the agent temperature should be 0.3 for auto debug Scenario: AutoDebugAgent builds a valid workflow graph Given I have an AutoDebugAgent instance When I build the workflow graph for auto debug Then the graph should contain node "analyze_error" for auto debug And the graph should contain node "generate_fix" for auto debug And the graph should contain node "validate_fix" for auto debug And the graph should contain node "finalize" for auto debug And the entry point should be "analyze_error" for auto debug Scenario: Analyze error step processes error message and code context Given I have an AutoDebugAgent instance And I have a state with error details: """ { "error_message": "NameError: name 'x' is not defined", "code_context": "def test():\n return x + 1" } """ When I execute the analyze_error step Then the state messages should contain an error_analysis message And the error_analysis should mention "Error analysis completed" Scenario: Generate fix step creates a fix suggestion Given I have an AutoDebugAgent instance And I have a state with error analysis completed When I execute the generate_fix step Then the state should contain current_fix And the current_fix should have a description field And the current_fix should have a code field Scenario: Validate fix step validates the proposed fix Given I have an AutoDebugAgent instance And I have a state with a current fix: """ { "description": "Define variable x before use", "code": "def test():\n x = 0\n return x + 1" } """ When I execute the validate_fix step Then the state should contain fix_validated And the fix_validated should be true Scenario: Should retry fix returns "retry" when validation fails and under max attempts Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with validation results for auto debug: """ { "fix_validated": false, "attempted_fixes": [] } """ When I check if retry is needed Then the decision should be "retry" for auto debug Scenario: Should retry fix returns "retry" on second attempt Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with validation results for auto debug: """ { "fix_validated": false, "attempted_fixes": [{"attempt": 1}] } """ When I check if retry is needed Then the decision should be "retry" for auto debug Scenario: Should retry fix returns "done" when validation succeeds Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with validation results for auto debug: """ { "fix_validated": true, "attempted_fixes": [] } """ When I check if retry is needed Then the decision should be "done" for auto debug Scenario: Should retry fix returns "done" when max attempts reached Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with validation results for auto debug: """ { "fix_validated": false, "attempted_fixes": [{"attempt": 1}, {"attempt": 2}, {"attempt": 3}] } """ When I check if retry is needed Then the decision should be "done" for auto debug Scenario: Finalize step creates final result structure with success Given I have an AutoDebugAgent instance And I have a state with successful fix: """ { "fix_validated": true, "current_fix": {"description": "Fixed", "code": "fixed code"}, "attempted_fixes": [] } """ When I execute the finalize step for auto debug Then the state should contain a result field for auto debug And the result should have a success field set to true for auto debug And the result should have a fix field And the result should have an attempts field Scenario: Finalize step marks failure when validation fails after max attempts Given I have an AutoDebugAgent instance And I have a state with failed validation after max attempts When I execute the finalize step for auto debug Then the result success field should be false for auto debug Scenario: AutoDebugState holds required workflow data Given I can create an AutoDebugState When I initialize it with all required fields for auto debug: | field | type | | error_message | str | | code_context | str | | attempted_fixes | list | | current_fix | dict | | fix_validated | bool | Then the state should store all fields correctly for auto debug Scenario: Workflow graph edges connect nodes correctly Given I have an AutoDebugAgent instance When I build the workflow graph for auto debug Then "analyze_error" should connect to "generate_fix" for auto debug And "generate_fix" should connect to "validate_fix" for auto debug And "validate_fix" should have conditional edges to "generate_fix" and "finalize" for auto debug And "finalize" should connect to END for auto debug Scenario: Full workflow with successful first fix Given I have an AutoDebugAgent instance And I have initial state with for auto debug: """ { "error_message": "TypeError: unsupported operand", "code_context": "x = '5' + 5", "messages": [], "attempted_fixes": [] } """ And the mock workflow returns valid fix on first attempt When I run the complete workflow for auto debug Then the workflow should complete successfully for auto debug And the final result should have success true for auto debug And the attempts should be 0 Scenario: Full workflow with one retry cycle Given I have an AutoDebugAgent instance And I have initial state with error details And the mock workflow returns invalid fix on first attempt And the mock workflow returns valid fix on second attempt When I run the complete workflow for auto debug Then the workflow should complete successfully for auto debug And the attempts should be 1 Scenario: Full workflow reaching max fix attempts Given I have an AutoDebugAgent instance with max_fix_attempts of 2 And I have initial state with error details And the mock workflow always returns invalid fix When I run the complete workflow for auto debug Then the workflow should complete for auto debug And the attempts should be 2 And the final result success should be false for auto debug Scenario: Analyze error logs appropriate messages Given I have an AutoDebugAgent instance And logging is enabled at INFO level for auto debug And I have a state with error message and code context When I execute the analyze_error step Then the log should contain "Analyzing error message" Scenario: Generate fix logs fix generation Given I have an AutoDebugAgent instance And logging is enabled at INFO level for auto debug And I have a state with error analysis When I execute the generate_fix step Then the log should contain "Generating fix suggestion" Scenario: Validate fix logs validation activity Given I have an AutoDebugAgent instance And logging is enabled at INFO level for auto debug And I have a state with current fix When I execute the validate_fix step Then the log should contain "Validating fix" Scenario: Finalize logs result summary Given I have an AutoDebugAgent instance And logging is enabled at INFO level for auto debug And I have a state with validated fix When I execute the finalize step for auto debug Then the log should contain "Finalizing auto-debug results" Scenario: Analyze error handles missing error message gracefully Given I have an AutoDebugAgent instance And I have a state with incomplete error details When I execute the analyze_error step Then the state messages should be updated Scenario: Generate fix handles empty attempted fixes list Given I have an AutoDebugAgent instance And I have a state with no attempted fixes When I execute the generate_fix step Then the state should contain current_fix Scenario: Validate fix handles missing current fix field Given I have an AutoDebugAgent instance And I have a state without current fix When I execute the validate_fix step Then the state should contain fix_validated Scenario: Should retry fix handles missing fix_validated field Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state without fix_validated field When I check if retry is needed Then the decision should be "retry" for auto debug Scenario: Should retry fix handles missing attempted_fixes field Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state without attempted_fixes field When I check if retry is needed Then the decision should be determined correctly Scenario: Finalize handles missing attempted_fixes gracefully Given I have an AutoDebugAgent instance And I have a state with minimal fields When I execute the finalize step for auto debug Then the result should have an attempts field with value 0 Scenario: State preserves messages through workflow Given I have an AutoDebugAgent instance And I have a state with existing messages When I execute the analyze_error step Then the state should preserve previous messages Scenario: Current fix updates between generate and validate Given I have an AutoDebugAgent instance And I have a state after error analysis When I execute the generate_fix step And I execute the validate_fix step Then the current_fix should still be present Scenario: AutoDebugAgent inherits from BaseAgent Given I have an AutoDebugAgent instance Then the agent should have provider attribute And the agent should have model attribute And the agent should have temperature attribute And the agent should have llm attribute And the agent should have graph attribute Scenario: Should retry handles exactly max attempts Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with exactly max attempted fixes When I check if retry is needed Then the decision should be "done" for auto debug Scenario: Analyze error appends to existing messages Given I have an AutoDebugAgent instance And I have a state with 2 existing messages for auto debug When I execute the analyze_error step Then the state should have 3 messages Scenario: Finalize result reflects validation status accurately Given I have an AutoDebugAgent instance And I have a state with fix_validated as true When I execute the finalize step for auto debug Then the result success should match fix_validated Scenario: Finalize result includes current fix details Given I have an AutoDebugAgent instance And I have a state with detailed current fix When I execute the finalize step for auto debug Then the result fix should contain description And the result fix should contain code Scenario: Generate fix can be called multiple times Given I have an AutoDebugAgent instance And I have a state after first fix attempt When I execute the generate_fix step And I execute the generate_fix step again Then both fix generations should complete Scenario: Workflow entry point is analyze_error Given I have an AutoDebugAgent instance When I inspect the workflow graph for auto debug Then the entry point should be "analyze_error" for auto debug Scenario: Validate fix always sets fix_validated field Given I have an AutoDebugAgent instance And I have a state with any current fix When I execute the validate_fix step Then the fix_validated field should be present Scenario: Should retry evaluates fix_validated correctly when false Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with fix_validated false and 1 attempt When I check if retry is needed Then the decision should be "retry" for auto debug Scenario: Should retry evaluates fix_validated correctly when true Given I have an AutoDebugAgent instance with max_fix_attempts of 3 And I have a state with fix_validated true and 1 attempt When I check if retry is needed Then the decision should be "done" for auto debug Scenario: Max fix attempts can be configured on initialization When I create an AutoDebugAgent with max_fix_attempts of 10 Then the agent max_fix_attempts should be 10 Scenario: Temperature defaults to 0.3 for deterministic debugging When I create an AutoDebugAgent with default parameters Then the agent temperature should be 0.3 for auto debug # Coverage for lines 133: analyze_error with real LLM response (not "Mock LLM response") Scenario: Analyze error with non-mock LLM response returns actual content Given I have an AutoDebugAgent instance And the LLM returns a non-mock analysis response "This is a type mismatch error" And I have a state with error details: """ { "error_message": "TypeError: expected int but got str", "code_context": "x = '5' + 5" } """ When I execute the analyze_error step Then the error_analysis content should be "This is a type mismatch error" # Coverage for lines 134-136: analyze_error exception handling Scenario: Analyze error handles LLM exception gracefully Given I have an AutoDebugAgent instance And the LLM raises an exception during analysis And I have a state with error details: """ { "error_message": "SyntaxError: invalid syntax", "code_context": "def foo(:" } """ When I execute the analyze_error step Then the error_analysis content should be "Error analysis completed" And a warning should be logged containing "LLM analysis failed" # Coverage for lines 223-225: generate_fix with valid JSON response Scenario: Generate fix parses valid JSON from non-mock LLM response Given I have an AutoDebugAgent instance And the LLM returns a valid JSON fix response And I have a state with error analysis completed When I execute the generate_fix step Then the current_fix description should be "Add variable definition" And the current_fix code should be "x = 0; return x + 1" And the current_fix files_to_modify should contain "test.py" # Coverage for lines 226-232: generate_fix with invalid JSON response (fallback) Scenario: Generate fix falls back when LLM returns non-JSON response Given I have an AutoDebugAgent instance And the LLM returns a non-JSON fix response "Just add x = 0 before the return" And I have a state with error analysis completed When I execute the generate_fix step Then the current_fix description should contain "Fix attempt" And the current_fix code should be "Just add x = 0 before the return" And the current_fix files_to_modify should be empty # Coverage for lines 233-239: generate_fix exception handling Scenario: Generate fix handles LLM exception gracefully Given I have an AutoDebugAgent instance And the LLM raises an exception during fix generation And I have a state with error analysis completed When I execute the generate_fix step Then the current_fix should have a description field And the current_fix should have a code field And a warning should be logged containing "LLM fix generation failed" # Coverage for lines 297-299: validate_fix with valid JSON response (is_valid true) Scenario: Validate fix parses valid JSON with is_valid true from non-mock response Given I have an AutoDebugAgent instance And the LLM returns a valid JSON validation response with is_valid true And I have a state with a current fix: """ { "description": "Add variable x before use", "code": "x = 0\nreturn x + 1" } """ When I execute the validate_fix step Then the fix_validated should be true # Coverage for lines 297-299: validate_fix with valid JSON response (is_valid false) Scenario: Validate fix parses valid JSON with is_valid false from non-mock response Given I have an AutoDebugAgent instance And the LLM returns a valid JSON validation response with is_valid false And I have a state with a current fix: """ { "description": "Incorrect fix attempt", "code": "return x" } """ When I execute the validate_fix step Then the fix_validated should be false # Coverage for lines 300-305: validate_fix JSON decode error with positive fallback Scenario: Validate fix falls back to text parsing with positive indicators Given I have an AutoDebugAgent instance And the LLM returns a non-JSON validation response "The fix is valid and correct" And I have a state with a current fix: """ { "description": "Add x = 0", "code": "x = 0; return x + 1" } """ When I execute the validate_fix step Then the fix_validated should be true # Coverage for lines 300-305: validate_fix JSON decode error without positive indicators Scenario: Validate fix falls back to text parsing without positive indicators Given I have an AutoDebugAgent instance And the LLM returns a non-JSON validation response "The code still has issues" And I have a state with a current fix: """ { "description": "Broken fix", "code": "return undefined_var" } """ When I execute the validate_fix step Then the fix_validated should be false # Coverage for lines 306-309: validate_fix exception handling Scenario: Validate fix handles LLM exception gracefully Given I have an AutoDebugAgent instance And the LLM raises an exception during validation And I have a state with a current fix: """ { "description": "Some fix", "code": "some code" } """ When I execute the validate_fix step Then the fix_validated should be true And a warning should be logged containing "LLM validation failed" # Coverage for lines 314-317: tracking failed fix attempts Scenario: Validate fix tracks failed attempts when validation is false Given I have an AutoDebugAgent instance And the LLM returns a valid JSON validation response with is_valid false And I have a state with current fix and empty attempted_fixes: """ { "description": "Failed fix", "code": "bad code" } """ When I execute the validate_fix step Then the fix_validated should be false And the attempted_fixes should have 1 entry And the attempted_fixes should contain the current fix # Coverage for lines 314-317: multiple failed attempts are tracked Scenario: Validate fix appends to existing attempted_fixes when validation fails Given I have an AutoDebugAgent instance And the LLM returns a valid JSON validation response with is_valid false And I have a state with current fix and one existing attempted fix When I execute the validate_fix step Then the fix_validated should be false And the attempted_fixes should have 2 entries # Coverage: validate_fix with "resolves" in response Scenario: Validate fix detects "resolves" keyword in fallback parsing Given I have an AutoDebugAgent instance And the LLM returns a non-JSON validation response "This change resolves the issue" And I have a state with a current fix: """ { "description": "Fix that resolves issue", "code": "fixed code" } """ When I execute the validate_fix step Then the fix_validated should be true # Coverage: validate_fix with "fixes" in response Scenario: Validate fix detects "fixes" keyword in fallback parsing Given I have an AutoDebugAgent instance And the LLM returns a non-JSON validation response "This fixes the problem" And I have a state with a current fix: """ { "description": "Fix that fixes problem", "code": "fixed code" } """ When I execute the validate_fix step Then the fix_validated should be true # Coverage: generate_fix includes previous attempts in prompt Scenario: Generate fix includes previous attempt descriptions in prompt Given I have an AutoDebugAgent instance And the LLM returns a valid JSON fix response And I have a state with two previous fix attempts When I execute the generate_fix step Then the current_fix should have a description field And the attempt number should be 3 # Edge case: Empty content from LLM in generate_fix Scenario: Generate fix handles empty non-mock response Given I have an AutoDebugAgent instance And the LLM returns an empty non-mock response for fix generation And I have a state with error analysis completed When I execute the generate_fix step Then the current_fix should have a description field And the current_fix code should be empty string # Edge case: Validation response with is_valid as string Scenario: Validate fix handles is_valid as string "true" in JSON Given I have an AutoDebugAgent instance And the LLM returns a JSON validation response with is_valid as string "true" And I have a state with a current fix: """ { "description": "Fix", "code": "code" } """ When I execute the validate_fix step Then the fix_validated should be true # Edge case: Validation response with is_valid missing Scenario: Validate fix handles missing is_valid field in JSON response Given I have an AutoDebugAgent instance And the LLM returns a JSON validation response without is_valid field And I have a state with a current fix: """ { "description": "Fix", "code": "code" } """ When I execute the validate_fix step Then the fix_validated should be false # Edge case: Case insensitive keyword detection in validation fallback Scenario: Validate fix keyword detection is case insensitive Given I have an AutoDebugAgent instance And the LLM returns a non-JSON validation response "VALID solution found" And I have a state with a current fix: """ { "description": "Fix", "code": "code" } """ When I execute the validate_fix step Then the fix_validated should be true