158 lines
8.6 KiB
Gherkin
158 lines
8.6 KiB
Gherkin
Feature: Plan Service Uncovered Lines Coverage
|
|
As a developer working on the CleverAgents system
|
|
I want to achieve complete test coverage for the plan service
|
|
So that all edge cases and error conditions are properly tested
|
|
|
|
Background:
|
|
Given I have a temporary test directory for plan service
|
|
And I have a Unit of Work instance for plan testing
|
|
And I have a PlanService instance
|
|
|
|
Scenario: Apply MOVE operation with nested relative new path
|
|
Given I have a saved project with current plan
|
|
And the plan has a pending MOVE change to a nested relative path
|
|
When I apply the plan changes
|
|
Then the file should be moved to the nested relative location
|
|
And the change should be marked as applied in plan service
|
|
|
|
Scenario: Get current plan returns None for unsaved project
|
|
Given I have an unsaved project without ID
|
|
When I get the current plan for the project
|
|
Then the current plan result should be None
|
|
|
|
Scenario: List plans returns empty list for unsaved project
|
|
Given I have an unsaved project without ID
|
|
When I list the plans for the project
|
|
Then the plans result should be an empty list
|
|
|
|
Scenario: Switch to plan raises error for unsaved project
|
|
Given I have an unsaved project without ID
|
|
When I attempt to switch to plan "some-plan"
|
|
Then a ValidationError should be raised with message "Project not initialized"
|
|
And the validation error details should hint to initialize the project
|
|
|
|
Scenario: Switch to non-existent plan raises ValidationError
|
|
Given I have a saved project with current plan
|
|
And the project has plans named "plan-a" and "plan-b"
|
|
When I attempt to switch to plan "non-existent-plan"
|
|
Then a ValidationError should be raised with message "Plan 'non-existent-plan' not found"
|
|
And the validation error details should list available plans "plan-a" and "plan-b"
|
|
|
|
Scenario: Add to plan raises error when no current plan exists
|
|
Given I have a saved project with no current plan
|
|
When I attempt to add prompt "Additional instructions" to the current plan
|
|
Then a PlanError should be raised with message "No current plan to add to"
|
|
And the plan error details should suggest creating a plan first
|
|
|
|
Scenario: Add to plan sets prompt when current plan prompt is None
|
|
Given I have a saved project with current plan
|
|
And the current plan prompt is set to None
|
|
When I add prompt "New instructions" to the current plan
|
|
Then the current plan prompt should be "New instructions"
|
|
And the current plan status should be PENDING
|
|
And the current plan updated_at should be refreshed
|
|
|
|
Scenario: LangSmith config includes project and plan metadata when enabled
|
|
Given LangSmith integration is enabled for plan service
|
|
When I prepare a LangSmith config for project "Docs" and plan "Improve docs"
|
|
Then the LangSmith builder should receive metadata for project "Docs" and plan "Improve docs"
|
|
And the prepared LangSmith config should include a generated thread id
|
|
|
|
Scenario: Build plan surfaces provider errors
|
|
Given I have a plan service with a failing AI provider
|
|
And I have a saved project with current plan
|
|
When I attempt to build the plan and the provider returns an error
|
|
Then a PlanError should be raised with message "mock provider failure"
|
|
|
|
Scenario: Auto debug build requires a current plan
|
|
Given I have a saved project with no current plan
|
|
When I run the auto debug build for the project
|
|
Then a PlanError should be raised with message "No current plan to build"
|
|
|
|
Scenario: Auto debug build reports failure after retries
|
|
Given I have a saved project with current plan
|
|
And auto debug retries should run with a failing build pipeline
|
|
When I run the auto debug build with max attempts 2
|
|
Then the auto debug result should indicate failure with error message "Simulated auto debug failure"
|
|
|
|
Scenario: Streaming generation requires an AI provider
|
|
Given I have a saved project
|
|
When I try to stream plan generation without an AI provider
|
|
Then a PlanError should be raised with message "No AI provider configured"
|
|
|
|
Scenario: Streaming provider that never completes raises PlanError
|
|
Given I have a plan service with an incomplete streaming provider
|
|
When I try to stream plan generation with prompt "incomplete stream"
|
|
Then a PlanError should be raised with message "Provider streaming ended before completion"
|
|
|
|
Scenario: Streaming provider surfaces string error payload
|
|
Given I have a plan service with a string error streaming provider
|
|
When I try to stream plan generation with prompt "string error stream"
|
|
Then the streaming failure events should include an error with message "stub failure"
|
|
And a PlanError should be raised with message "stub failure"
|
|
|
|
Scenario: Streaming provider returns non-list changes payload
|
|
Given I have a plan service with a non-list change streaming provider
|
|
When I try to stream plan generation with prompt "non list change payload"
|
|
Then a PlanError should be raised with message "Provider stream returned an invalid change payload"
|
|
|
|
Scenario: Streaming provider returns invalid change entry
|
|
Given I have a plan service with an invalid change entry streaming provider
|
|
When I try to stream plan generation with prompt "invalid change entry"
|
|
Then a PlanError should be raised with message "Provider stream returned a non-change entry"
|
|
|
|
Scenario: Streaming provider returns non-python change without validation errors
|
|
Given I have a plan service with a non-python streaming provider
|
|
When I stream plan generation with prompt "non python change"
|
|
Then the streaming result should include the non-python change without validation errors
|
|
|
|
Scenario: Streaming provider returns syntax error python change
|
|
Given I have a plan service with a syntax error streaming provider
|
|
When I try to stream plan generation with prompt "syntax error change"
|
|
Then a PlanError should be raised containing "bad.py contained invalid Python"
|
|
|
|
Scenario: Streaming provider dict payload coerces token count
|
|
Given I have a plan service with a dict streaming provider
|
|
When I stream plan generation with prompt "dict payload change"
|
|
Then the streaming events should include nodes "generate_plan, __end__"
|
|
And the streamed plan should persist token count 0
|
|
|
|
Scenario: Resolving injected provider without identifiers falls back to defaults
|
|
Given the plan service AI provider has no identifier attributes
|
|
When I try to resolve a provider for the plan service
|
|
Then the provider resolution should use fallback names
|
|
|
|
Scenario: Resolving provider uses injected provider model when actor model missing
|
|
Given the plan service AI provider is "override-provider" with model "provider-model"
|
|
And the plan service actor lookup returns actor "override/actor" with provider "override-provider" and no model
|
|
When I resolve provider for actor "override/actor"
|
|
Then the provider resolution should return provider "override-provider" and model "provider-model"
|
|
|
|
Scenario: Resolving provider uses provider model attribute when actor model missing
|
|
Given the plan service AI provider is "attr-provider" with model "attr-model"
|
|
And the plan service actor lookup returns actor "attr/actor" with provider "attr-provider" and no model
|
|
When I resolve provider for actor "attr/actor"
|
|
Then the provider resolution should return provider "attr-provider" and model "attr-model"
|
|
|
|
Scenario: Resolving provider falls back to provider name when model id missing
|
|
Given the plan service AI provider has name "name-only-provider" without model id
|
|
And the plan service actor lookup returns actor "name-only/actor" with provider "name-only-provider" and no model
|
|
When I resolve provider for actor "name-only/actor"
|
|
Then the provider resolution should return provider "name-only-provider" and model "name-only-provider"
|
|
|
|
Scenario: Provider registry getter reuses cached instance
|
|
Given the plan service has a cached provider registry instance
|
|
When I fetch the provider registry from the plan service
|
|
Then the cached provider registry should be reused
|
|
|
|
Scenario: Resolving actor provider without configured AI in mock mode raises error
|
|
Given mock provider mode is forced
|
|
And the plan service actor lookup returns actor "openai/retry-actor" with provider "openai" and model "gpt-4"
|
|
When I resolve the AI provider for actor "openai/retry-actor"
|
|
Then a PlanError should be raised with message "No AI provider configured"
|
|
|
|
Scenario: Auto debug marks previous attempt successful after retry
|
|
Given I have a saved project with current plan
|
|
When I run the auto debug build with a retrying success
|
|
Then the auto debug attempt should be marked successful after retry
|