test(coverage): add Behave BDD tests to improve coverage across 52 source files
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 31s
CI / typecheck (pull_request) Successful in 47s
CI / security (pull_request) Successful in 52s
CI / build (pull_request) Successful in 56s
CI / e2e_tests (pull_request) Successful in 5m1s
CI / integration_tests (pull_request) Successful in 5m30s
CI / unit_tests (pull_request) Successful in 5m42s
CI / docker (pull_request) Successful in 58s
CI / coverage (pull_request) Successful in 7m35s
CI / build (push) Successful in 21s
CI / docker (push) Has been skipped
CI / benchmark-regression (pull_request) Failing after 49m24s
CI / lint (push) Successful in 22s
CI / quality (push) Successful in 39s
CI / security (push) Successful in 48s
CI / typecheck (push) Successful in 1m26s
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 5m53s
CI / coverage (push) Successful in 9m4s
CI / benchmark-publish (push) Successful in 19m10s
CI / integration_tests (push) Failing after 19m18s
CI / unit_tests (push) Failing after 19m20s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 31s
CI / typecheck (pull_request) Successful in 47s
CI / security (pull_request) Successful in 52s
CI / build (pull_request) Successful in 56s
CI / e2e_tests (pull_request) Successful in 5m1s
CI / integration_tests (pull_request) Successful in 5m30s
CI / unit_tests (pull_request) Successful in 5m42s
CI / docker (pull_request) Successful in 58s
CI / coverage (pull_request) Successful in 7m35s
CI / build (push) Successful in 21s
CI / docker (push) Has been skipped
CI / benchmark-regression (pull_request) Failing after 49m24s
CI / lint (push) Successful in 22s
CI / quality (push) Successful in 39s
CI / security (push) Successful in 48s
CI / typecheck (push) Successful in 1m26s
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 5m53s
CI / coverage (push) Successful in 9m4s
CI / benchmark-publish (push) Successful in 19m10s
CI / integration_tests (push) Failing after 19m18s
CI / unit_tests (push) Failing after 19m20s
Added 52 new .feature files and corresponding _steps.py files targeting previously uncovered code paths in the following areas: - TUI layer: app, commands, persona (state/schema/registry), widgets, input (shell_exec, reference_parser) - Application services: plan lifecycle/service/executor, session, project, repo indexing, correction, checkpoint, actor, llm_actors, strategy coordinator, resource file watcher, service retry wiring - CLI commands: session, resource, repl, plan, db, automation_profile - Domain models: retry_policy, resource_type, cost_budget, docker_compose_analyzer, detail_level, _sql_string_aware, _postgresql_helpers - Core: circuit_breaker, retry_service_patterns - Infrastructure: repositories, transaction_sandbox, strategy_registry, plugins/loader, container - Config: settings - Agents: plan_generation, context_analysis, auto_debug - A2A: facade All new tests follow the Behave/Gherkin BDD standard. Resolved step definition collisions with unique prefixes. Fixed Alembic fileConfig logger disabling issue (disable_existing_loggers=False). ISSUES CLOSED: #1068
This commit was merged in pull request #1069.
This commit is contained in:
@@ -0,0 +1,145 @@
|
||||
Feature: AutoDebug graph node coverage boost
|
||||
Additional scenarios that exercise previously uncovered code paths
|
||||
in the auto_debug.py graph module, targeting individual node methods
|
||||
and edge-case LLM response handling.
|
||||
|
||||
Background:
|
||||
Given the auto debug graph module is imported
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# _analyze_error node
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: Analyze error with Mock LLM response yields default analysis
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
When I call _analyze_error with an error state
|
||||
Then the analysis message should be "Error analysis completed"
|
||||
|
||||
Scenario: Analyze error with real content passes through LLM output
|
||||
Given an auto debug agent with a mock LLM returning "Root cause is a null pointer"
|
||||
When I call _analyze_error with an error state
|
||||
Then the analysis message should be "Root cause is a null pointer"
|
||||
|
||||
Scenario: Analyze error handles LLM invocation failure gracefully
|
||||
Given an auto debug agent with a failing LLM
|
||||
When I call _analyze_error with an error state
|
||||
Then the analysis message should be "Error analysis completed"
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# _generate_fix node
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: Generate fix with Mock LLM response yields default fix data
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
And a state with an existing error analysis message
|
||||
When I call _generate_fix with the prepared state
|
||||
Then the current fix description should be "Fix suggestion"
|
||||
And the current fix code should be "# Fixed code"
|
||||
|
||||
Scenario: Generate fix parses valid JSON from LLM
|
||||
Given an auto debug agent with a mock LLM returning valid fix JSON
|
||||
And a state with an existing error analysis message
|
||||
When I call _generate_fix with the prepared state
|
||||
Then the current fix description should be "Parsed JSON fix"
|
||||
And the current fix code should be "print('fixed')"
|
||||
|
||||
Scenario: Generate fix falls back on invalid JSON from LLM
|
||||
Given an auto debug agent with a mock LLM returning "not valid json at all"
|
||||
And a state with an existing error analysis message
|
||||
When I call _generate_fix with the prepared state
|
||||
Then the current fix description should be "Fix attempt 1"
|
||||
And the current fix code should be "not valid json at all"
|
||||
|
||||
Scenario: Generate fix handles LLM invocation failure gracefully
|
||||
Given an auto debug agent with a failing LLM
|
||||
And a state with an existing error analysis message
|
||||
When I call _generate_fix with the prepared state
|
||||
Then the current fix description should be "Fix suggestion"
|
||||
And the current fix code should be "# Fixed code"
|
||||
|
||||
Scenario: Generate fix includes previous attempt count in fallback description
|
||||
Given an auto debug agent with a mock LLM returning "still broken"
|
||||
And a state with two previous attempted fixes
|
||||
When I call _generate_fix with the prepared state
|
||||
Then the current fix description should be "Fix attempt 3"
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# _validate_fix node
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: Validate fix with Mock LLM response marks fix as valid
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should be marked as validated
|
||||
|
||||
Scenario: Validate fix parses valid JSON with is_valid true
|
||||
Given an auto debug agent with a mock LLM returning valid validation JSON true
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should be marked as validated
|
||||
|
||||
Scenario: Validate fix parses valid JSON with is_valid false
|
||||
Given an auto debug agent with a mock LLM returning valid validation JSON false
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should not be marked as validated
|
||||
And the current fix should be appended to attempted fixes
|
||||
|
||||
Scenario: Validate fix handles invalid JSON containing positive keywords
|
||||
Given an auto debug agent with a mock LLM returning "The fix is valid and correct"
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should be marked as validated
|
||||
|
||||
Scenario: Validate fix handles invalid JSON without positive keywords
|
||||
Given an auto debug agent with a mock LLM returning "The fix does not work"
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should not be marked as validated
|
||||
And the current fix should be appended to attempted fixes
|
||||
|
||||
Scenario: Validate fix handles LLM invocation failure gracefully
|
||||
Given an auto debug agent with a failing LLM
|
||||
And a state with a current fix to validate
|
||||
When I call _validate_fix with the prepared state
|
||||
Then the fix should be marked as validated
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# _should_retry_fix routing
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: Should retry fix returns retry when validation failed and under max attempts
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
When I check should_retry_fix with validated false and 1 attempt
|
||||
Then the routing decision should be "retry"
|
||||
|
||||
Scenario: Should retry fix returns done when validation succeeded
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
When I check should_retry_fix with validated true and 0 attempts
|
||||
Then the routing decision should be "done"
|
||||
|
||||
Scenario: Should retry fix returns done when max attempts reached
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response" and max 2 fix attempts
|
||||
When I check should_retry_fix with validated false and 2 attempts
|
||||
Then the routing decision should be "done"
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# _finalize node
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: Finalize produces result dict from successful state
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
And a finalize state with fix_validated true and 0 attempted fixes
|
||||
When I call _finalize with the prepared state
|
||||
Then the result should indicate success true
|
||||
And the result attempts count should be 0
|
||||
|
||||
Scenario: Finalize produces result dict from failed state
|
||||
Given an auto debug agent with a mock LLM returning "Mock LLM response"
|
||||
And a finalize state with fix_validated false and 3 attempted fixes
|
||||
When I call _finalize with the prepared state
|
||||
Then the result should indicate success false
|
||||
And the result attempts count should be 3
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Constructor edge case
|
||||
# -----------------------------------------------------------------------
|
||||
Scenario: AutoDebugAgent constructor raises ValueError when no LLM is provided
|
||||
When I try to create an AutoDebugAgent with no LLM
|
||||
Then a ValueError should be raised with the missing provider message
|
||||
Reference in New Issue
Block a user