freemo
0bbec4b77e
Optimized the 8 features accounting for 64% of total BDD test runtime (1,505s of 2,352s): - features/environment.py: Added _ensure_template_db() for direct behave invocations, expanded _SCENARIO_DB_PREFIXES to include "test_" for step-file-created DBs, added @mock_only tag support to skip unnecessary DB setup. - features/plan_commands_coverage.feature: Added @mock_only tag (fully mocked, no DB needed). - features/plan_service.feature: 14 actor-resolution scenarios now use lightweight in-memory plan service instead of heavyweight file-based DB + project init. - features/steps/plan_service_steps.py: New step_create_lightweight_plan_service for actor-resolution tests. - features/steps/services_coverage_steps.py: Extracted 3 helper functions to consolidate 8 near-identical Given steps (~200 lines of duplicated boilerplate removed). Per-feature results: services_coverage 245s->2.8s (99%), context_service 215s->2.1s (99%), project_service 140s->0.7s (99%), plan_service 215s->4.6s (98%), core_cli_commands 113s->4.2s (96%), cli_streaming 213s->6.4s (97%), plan_commands_coverage 116s->20s (83%), cli_plan_context_commands 248s->31.6s (87%). ISSUES CLOSED: #479
231 lines
12 KiB
Gherkin
231 lines
12 KiB
Gherkin
Feature: Plan Service
|
|
As a developer
|
|
I want to manage plans
|
|
So that I can generate and apply code changes
|
|
|
|
Scenario: Create a new plan with prompt
|
|
Given I have a plan service
|
|
When I create a plan with prompt "Add error handling"
|
|
Then the plan should be created
|
|
And the plan prompt should be "Add error handling"
|
|
And the plan should be the current plan
|
|
|
|
Scenario: Create empty plan with name
|
|
Given I have a plan service
|
|
When I create an empty plan named "feature-x"
|
|
Then the plan should be created
|
|
And the plan name should be "feature-x"
|
|
And the plan prompt should be empty
|
|
|
|
Scenario: Build a plan to generate changes
|
|
Given I have a plan service
|
|
And I have created a plan with "Generate example code"
|
|
When I build the plan
|
|
Then changes should be generated
|
|
And the changes should include file operations
|
|
|
|
Scenario: Build plan uses actor selection
|
|
Given I have a plan service with stub provider registry
|
|
And I have created a plan with "Generate example code"
|
|
When I build the plan with actor "anthropic/claude-dev"
|
|
Then changes should be generated
|
|
And the stub provider registry should record provider "anthropic" and model "claude-dev"
|
|
|
|
Scenario: Build plan fails when no provider configured
|
|
Given I have a plan service without providers configured
|
|
And I have created a plan with "Generate example code"
|
|
When I try to build the plan
|
|
Then a PlanError should be raised with message "No AI provider configured"
|
|
And the PlanError details should include provider diagnostics
|
|
|
|
Scenario: Apply generated changes
|
|
|
|
Given I have a plan service
|
|
And I have created and built a plan
|
|
When I apply the changes
|
|
Then the changes should be applied
|
|
And files should be created or modified
|
|
|
|
Scenario: List all plans
|
|
Given I have a plan service
|
|
And I have created multiple plans
|
|
When I list all plans
|
|
Then I should see all created plans
|
|
And each plan should have a name and status
|
|
|
|
Scenario: Switch between plans
|
|
Given I have a plan service
|
|
And I have created plan "plan-a"
|
|
And I have created plan "plan-b"
|
|
When I switch to plan "plan-a"
|
|
Then "plan-a" should be the current service plan
|
|
And "plan-b" should not be current
|
|
|
|
Scenario: Continue adding to existing plan
|
|
Given I have a plan service
|
|
And I have created a plan with "Initial instructions"
|
|
When I add "Additional instructions" to the plan
|
|
Then the plan should contain both instructions
|
|
|
|
Scenario: Auto-debug build uses empty context when plan ID disappears
|
|
Given I have a temporary test directory for plan service
|
|
And I configured an auto-debug plan with a disappearing plan ID
|
|
When I run auto-debug build with missing plan context
|
|
Then the auto-debug build should fall back to an empty context
|
|
|
|
Scenario: Apply changes moves files to absolute destinations
|
|
Given I have a temporary test directory for plan service
|
|
And I have a Unit of Work instance for plan testing
|
|
And I have a PlanService instance
|
|
And I have a saved project with current plan
|
|
And the plan has a pending MOVE change to an absolute path
|
|
When I apply the plan changes
|
|
Then the absolute destination should exist and the source should be removed
|
|
|
|
Scenario: Stream plan generation and persist usage
|
|
Given I have a plan service with a streaming stub provider
|
|
When I stream plan generation with prompt "Streamed instructions"
|
|
Then the streaming events should include nodes "load_context, analyze_requirements, generate_plan, validate, __end__"
|
|
And the streamed plan should persist token count 321
|
|
|
|
Scenario: Stream plan generation fails when provider never completes
|
|
Given I have a plan service with an incomplete streaming provider
|
|
When I try to stream plan generation with prompt "Incomplete stream"
|
|
Then a PlanError should be raised with message "Provider streaming ended before completion"
|
|
|
|
Scenario: Stream plan generation coerces dict payloads
|
|
Given I have a plan service with a dict streaming provider
|
|
When I stream plan generation with prompt "Dict payload"
|
|
Then the streaming events should include nodes "generate_plan, __end__"
|
|
And the streamed plan should persist token count 0
|
|
|
|
Scenario: Stream plan generation surfaces provider error strings
|
|
Given I have a plan service with a string error streaming provider
|
|
When I try to stream plan generation with prompt "String error"
|
|
Then the streaming failure events should include an error with message "stub failure"
|
|
And a PlanError should be raised with message "stub failure"
|
|
|
|
Scenario: Stream plan generation rejects non-list change payloads
|
|
Given I have a plan service with a non-list change streaming provider
|
|
When I try to stream plan generation with prompt "Non list payload"
|
|
Then a PlanError should be raised with message "Provider stream returned an invalid change payload"
|
|
|
|
Scenario: Stream plan generation rejects invalid change entries
|
|
Given I have a plan service with an invalid change entry streaming provider
|
|
When I try to stream plan generation with prompt "Invalid change entry"
|
|
Then a PlanError should be raised with message "Provider stream returned a non-change entry"
|
|
|
|
Scenario: Mock provider resolution keeps nameless provider metadata
|
|
Given I have a lightweight plan service for actor testing
|
|
And mock provider mode is forced
|
|
And the plan service uses a nameless AI provider
|
|
And the plan service actor lookup returns actor "anthropic/claude-mock" with provider "anthropic" and model "claude-mock"
|
|
When I resolve provider for actor "anthropic/claude-mock"
|
|
Then the nameless provider resolution should return provider "anthropic" and model "claude-mock"
|
|
|
|
Scenario: Actor resolution triggers lazy registry creation
|
|
Given I have a lightweight plan service for actor testing
|
|
And I stub the provider registry factory for lazy initialization
|
|
And the plan service actor lookup returns actor "anthropic/claude-dev" with provider "anthropic" and model "claude-dev"
|
|
When I resolve provider for actor "anthropic/claude-dev"
|
|
Then the lazy registry should memoize provider "anthropic" and model "claude-dev"
|
|
|
|
Scenario: LangSmith config omits missing metadata
|
|
Given I have a temporary test directory for plan service
|
|
And LangSmith integration is enabled for plan service
|
|
When I prepare a LangSmith config without plan metadata for project "ephemeral-project"
|
|
Then the LangSmith builder should receive only base metadata for project "ephemeral-project"
|
|
And the prepared LangSmith config should include a generated thread id
|
|
|
|
Scenario: Build fails when plan ID disappears mid-transaction
|
|
Given I have a temporary test directory for plan service
|
|
And I configure a stub plan service whose current plan loses its ID after the transaction
|
|
When I try to build the plan with the stubbed service
|
|
Then a PlanError should be raised with message "Plan does not have a valid ID"
|
|
|
|
Scenario: Clearing memory without forgetting history keeps cached messages
|
|
Given I have a lightweight plan service for actor testing
|
|
And I stored a chat message in session "ephemeral-branch"
|
|
When I clear the session "ephemeral-branch" memory without forgetting history
|
|
Then the session "ephemeral-branch" memory should retain its stored messages
|
|
|
|
Scenario: Actor resolution fails when actor service is missing
|
|
Given I have a lightweight plan service for actor testing
|
|
When I try to resolve actor "ghost/actor"
|
|
Then a PlanError should be raised with message "Actor support is not configured"
|
|
|
|
Scenario: Actor lookup validation errors surface as plan errors
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup raises ValidationError "invalid actor selection"
|
|
When I try to resolve actor "invalid/actor"
|
|
Then a PlanError should be raised with message "invalid actor selection"
|
|
And the PlanError should include actor detail "invalid/actor"
|
|
|
|
Scenario: Actor fallback fails when mock provisioning returns nothing
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns no actor and mock provisioning fails
|
|
When I try to resolve actor "openai/gpt-4o"
|
|
Then a PlanError should be raised with message "No actor configured"
|
|
|
|
Scenario: Actor resolution returns explicit actors
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns actor "openai/gpt-4o" with provider "openai" and model "gpt-4o"
|
|
When I try to resolve actor "openai/gpt-4o"
|
|
Then the actor resolution should return actor "openai/gpt-4o" from source "explicit"
|
|
|
|
|
|
Scenario: Mock actor provider resolution requires configured AI provider
|
|
Given I have a lightweight plan service for actor testing
|
|
And mock provider mode is forced
|
|
And the plan service actor lookup returns actor "mock/provider" with provider "mock-provider" and model "mock-model"
|
|
When I resolve provider for actor "mock/provider"
|
|
Then a PlanError should be raised with message "No AI provider configured"
|
|
|
|
Scenario: Mock actor provider resolution uses injected provider defaults
|
|
Given I have a lightweight plan service for actor testing
|
|
And mock provider mode is forced
|
|
And the plan service AI provider is "injected-provider" with model "injected-model"
|
|
And the plan service actor lookup returns actor "mock/provider" with provider "mock-provider" and model "mock-model"
|
|
When I resolve provider for actor "mock/provider"
|
|
Then the provider resolution should return provider "mock-provider" and model "mock-model"
|
|
|
|
Scenario: Mock actor provider appears after initial check
|
|
Given I have a Unit of Work instance for plan testing
|
|
And the plan service uses a delayed mock provider
|
|
And the plan service actor lookup returns actor "mock/provider" with provider "mock-provider" and model "mock-model"
|
|
When I resolve provider for actor "mock/provider"
|
|
Then the provider resolution should return provider "mock-provider" and model "mock-model"
|
|
|
|
Scenario: Actor provider registry factory failure surfaces PlanError
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns actor "broken/provider" with provider "broken-provider" and model "broken-model"
|
|
And the provider registry factory raises ValueError "actor registry missing" for actor providers
|
|
When I resolve provider for actor "broken/provider"
|
|
Then a PlanError should be raised with message "actor registry missing"
|
|
|
|
Scenario: Actor provider registry errors surface as PlanError
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns actor "anthropic/haiku" with provider "anthropic" and model "haiku"
|
|
And the provider registry raises ValueError "actor registry unavailable" for actor providers
|
|
When I resolve provider for actor "anthropic/haiku"
|
|
Then a PlanError should be raised with message "actor registry unavailable"
|
|
|
|
Scenario: Actor provider registry resolves provider and model
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns actor "openai/gpt-4o" with provider "openai" and model "gpt-4o"
|
|
And the provider registry returns provider "openai" with model "gpt-4o"
|
|
When I resolve provider for actor "openai/gpt-4o"
|
|
Then the provider resolution should return provider "openai" and model "gpt-4o"
|
|
|
|
Scenario: Actor provider resolution uses lazy registry fallback
|
|
Given I have a lightweight plan service for actor testing
|
|
And the plan service actor lookup returns actor "openai/gpt-4o" with provider "openai" and model "gpt-4o"
|
|
And I stub the provider registry factory for lazy initialization
|
|
When I resolve provider for actor "openai/gpt-4o"
|
|
Then the lazy registry should record actor provider "openai" and model "gpt-4o"
|
|
|
|
Scenario: Streaming sanitization reports unrecoverable Python syntax
|
|
Given I have a plan service with an unrecoverable streaming provider
|
|
When I try to stream plan generation with prompt "Fatal stream"
|
|
Then a PlanError should be raised containing "Validation failed: Syntax error"
|