Add Robot Framework integration tests for agents/graphs/auto_debug graph #10733

New issue

Open

opened 2026-04-19 08:44:06 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-19 08:44:06 +00:00

Owner

Summary

The agents/graphs/auto_debug.py module implements the AutoDebug LangGraph workflow that automatically analyzes errors, generates fixes, and validates them. This module has Behave BDD coverage but no dedicated Robot Framework integration test.

Source file	Behave coverage	Robot coverage	Benchmark coverage
agents/graphs/auto_debug.py	features/auto_debug_graph.feature, features/auto_debug_coverage_boost.feature, features/auto_debug_integration.feature	none - no robot/auto_debug*.robot file exists	none

The existing Robot Framework tests robot/context_analysis_agent.robot and robot/plan_generation_graph.robot cover the context analysis and plan generation graphs respectively, but the auto_debug graph has no equivalent integration test.

Proposed Tests

Add robot/auto_debug_graph.robot with Robot Framework integration tests that:

Smoke test: Verify the auto_debug graph can be instantiated and invoked with a mock LLM provider.
Error analysis node: Verify _analyze_error node returns a structured analysis dict.
Fix generation node: Verify _generate_fix node produces a code fix.
Fix validation node: Verify _validate_fix node evaluates the generated fix.
Full workflow: Verify the complete auto_debug graph executes all three nodes in sequence.

All tests must use mock LLM providers (no real API calls).

Subtasks

Create robot/auto_debug_graph.robot with smoke test for graph instantiation
Add test for _analyze_error node returning update dict
Add test for _generate_fix node producing structured fix output
Add test for _validate_fix node evaluating fix correctness
Add full workflow integration test with mock LLM
Create corresponding helper module robot/scripts/helper_auto_debug_graph.py if needed
Verify tests pass under nox -s integration_tests
Confirm no regressions in existing tests via nox

Acceptance Criteria

robot/auto_debug_graph.robot exists and contains at least 3 test cases
All test cases use mock LLM providers - no real API calls
Tests are tagged with appropriate Robot Framework tags
nox -s integration_tests completes without errors for the new test file

Definition of Done

This issue is complete when all subtasks are completed, a commit is created with the exact commit message from Metadata, pushed to the branch from Metadata, and a PR is submitted, reviewed, and merged.

Duplicate Check

Searched open issues for auto_debug robot, auto_debug integration, auto_debug graph robot, agents/graphs/auto_debug integration - no matches found.
Reviewed existing issues: The bug report about _analyze_error mutating state is a bug report, not a missing integration test issue. The TDD issue for that bug is for a specific bug, not a comprehensive integration test suite.
Searched closed issues for auto_debug robot, auto_debug integration - no relevant matches.
Cross-area search: robot/context_analysis_agent.robot and robot/plan_generation_graph.robot exist for other agent graphs, but no equivalent exists for auto_debug.

Metadata

Commit message: test(robot): add Robot Framework integration tests for agents/graphs/auto_debug graph
Branch name: test/robot-auto-debug-graph

Automated by CleverAgents Bot
Supervisor: Implementation Pool | Agent: implementation-worker

## Summary The agents/graphs/auto_debug.py module implements the AutoDebug LangGraph workflow that automatically analyzes errors, generates fixes, and validates them. This module has Behave BDD coverage but no dedicated Robot Framework integration test. | Source file | Behave coverage | Robot coverage | Benchmark coverage | |---|---|---|---| | agents/graphs/auto_debug.py | features/auto_debug_graph.feature, features/auto_debug_coverage_boost.feature, features/auto_debug_integration.feature | none - no robot/auto_debug*.robot file exists | none | The existing Robot Framework tests robot/context_analysis_agent.robot and robot/plan_generation_graph.robot cover the context analysis and plan generation graphs respectively, but the auto_debug graph has no equivalent integration test. ## Proposed Tests Add robot/auto_debug_graph.robot with Robot Framework integration tests that: 1. Smoke test: Verify the auto_debug graph can be instantiated and invoked with a mock LLM provider. 2. Error analysis node: Verify _analyze_error node returns a structured analysis dict. 3. Fix generation node: Verify _generate_fix node produces a code fix. 4. Fix validation node: Verify _validate_fix node evaluates the generated fix. 5. Full workflow: Verify the complete auto_debug graph executes all three nodes in sequence. All tests must use mock LLM providers (no real API calls). ## Subtasks - [ ] Create robot/auto_debug_graph.robot with smoke test for graph instantiation - [ ] Add test for _analyze_error node returning update dict - [ ] Add test for _generate_fix node producing structured fix output - [ ] Add test for _validate_fix node evaluating fix correctness - [ ] Add full workflow integration test with mock LLM - [ ] Create corresponding helper module robot/scripts/helper_auto_debug_graph.py if needed - [ ] Verify tests pass under nox -s integration_tests - [ ] Confirm no regressions in existing tests via nox ## Acceptance Criteria - robot/auto_debug_graph.robot exists and contains at least 3 test cases - All test cases use mock LLM providers - no real API calls - Tests are tagged with appropriate Robot Framework tags - nox -s integration_tests completes without errors for the new test file ## Definition of Done This issue is complete when all subtasks are completed, a commit is created with the exact commit message from Metadata, pushed to the branch from Metadata, and a PR is submitted, reviewed, and merged. ## Duplicate Check - Searched open issues for auto_debug robot, auto_debug integration, auto_debug graph robot, agents/graphs/auto_debug integration - no matches found. - Reviewed existing issues: The bug report about _analyze_error mutating state is a bug report, not a missing integration test issue. The TDD issue for that bug is for a specific bug, not a comprehensive integration test suite. - Searched closed issues for auto_debug robot, auto_debug integration - no relevant matches. - Cross-area search: robot/context_analysis_agent.robot and robot/plan_generation_graph.robot exist for other agent graphs, but no equivalent exists for auto_debug. ## Metadata - **Commit message:** test(robot): add Robot Framework integration tests for agents/graphs/auto_debug graph - **Branch name:** test/robot-auto-debug-graph --- **Automated by CleverAgents Bot** Supervisor: Implementation Pool | Agent: implementation-worker

HAL9000 added this to the v3.9.0 milestone

2026-04-19 08:44:06 +00:00

HAL9000 added the

labels