TEST-INFRA: [test-data-quality] Use Scenario Outlines in auto_debug_integration.feature #2439

New issue

Open

opened 2026-04-03 18:21:25 +00:00 by freemo · 0 comments

freemo commented

2026-04-03 18:21:25 +00:00

Owner

Metadata

Branch: test/auto-debug-scenario-outlines
Commit Message: test(auto_debug): refactor auto_debug_integration.feature to use Scenario Outlines
Milestone: v3.8.0
Parent Epic: #1678

Description

The scenarios in features/auto_debug_integration.feature contain hardcoded values for max attempts and actor stub names. This makes the tests less flexible and harder to maintain.

Recommendation:

Use Scenario Outlines to provide these values as examples. This will reduce duplication and make it easier to add new test cases with different data.

Example:

Scenario Outline: AutoDebug gives up after N attempts
  Given the AI provider will generate code with an unfixable error
  And the AutoDebugAgent cannot fix the error
  When I run auto_debug_build with max attempts <max_attempts>
  Then the build should fail after <max_attempts> attempts
  And debug attempts should be saved to database
  And all <max_attempts> debug attempts should be marked as unsuccessful

  Examples:
    | max_attempts |
    | 1            |
    | 3            |
    | 5            |

Subtasks

Audit features/auto_debug_integration.feature for all hardcoded max_attempts values and actor stub names
Refactor affected scenarios into Scenario Outline blocks with Examples tables
Update step definitions in features/steps/ to accept outline parameters where needed
Verify no duplication remains across scenarios that can be collapsed into a single outline
Run nox -e unit_tests to confirm all scenarios pass with the new outline structure
Run nox -e coverage_report to confirm coverage remains >= 97%

Definition of Done

All hardcoded max_attempts values in auto_debug_integration.feature are replaced with <max_attempts> outline parameters
All hardcoded actor stub names are replaced with outline parameters where applicable
Examples tables cover at least the original hardcoded values plus meaningful boundary cases
Step definitions correctly handle parameterised outline values
No scenario duplication remains that could be expressed as a single outline
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/auto-debug-scenario-outlines` - **Commit Message**: `test(auto_debug): refactor auto_debug_integration.feature to use Scenario Outlines` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The scenarios in `features/auto_debug_integration.feature` contain hardcoded values for `max attempts` and actor stub names. This makes the tests less flexible and harder to maintain. **Recommendation:** Use Scenario Outlines to provide these values as examples. This will reduce duplication and make it easier to add new test cases with different data. **Example:** ```gherkin Scenario Outline: AutoDebug gives up after N attempts Given the AI provider will generate code with an unfixable error And the AutoDebugAgent cannot fix the error When I run auto_debug_build with max attempts <max_attempts> Then the build should fail after <max_attempts> attempts And debug attempts should be saved to database And all <max_attempts> debug attempts should be marked as unsuccessful Examples: | max_attempts | | 1 | | 3 | | 5 | ``` ## Subtasks - [ ] Audit `features/auto_debug_integration.feature` for all hardcoded `max_attempts` values and actor stub names - [ ] Refactor affected scenarios into `Scenario Outline` blocks with `Examples` tables - [ ] Update step definitions in `features/steps/` to accept outline parameters where needed - [ ] Verify no duplication remains across scenarios that can be collapsed into a single outline - [ ] Run `nox -e unit_tests` to confirm all scenarios pass with the new outline structure - [ ] Run `nox -e coverage_report` to confirm coverage remains >= 97% ## Definition of Done - [ ] All hardcoded `max_attempts` values in `auto_debug_integration.feature` are replaced with `<max_attempts>` outline parameters - [ ] All hardcoded actor stub names are replaced with outline parameters where applicable - [ ] `Examples` tables cover at least the original hardcoded values plus meaningful boundary cases - [ ] Step definitions correctly handle parameterised outline values - [ ] No scenario duplication remains that could be expressed as a single outline - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

Rows
Columns