forked from HAL9000/cleveragents-core
freemo
a321fb3b37
- What was implemented
- Added PLAN_ROLLED_BACK event type to the EventType enum at src/cleveragents/infrastructure/events/types.py to properly represent successful rollbacks in the domain model.
- Implemented rollback_plan(plan_id: str, checkpoint_id: str) -> RollbackResult in PlanLifecycleService (src/cleveragents/application/services/plan_lifecycle_service.py) with:
- Plan state validation: rejects rollback when the plan is in terminal APPLIED or CANCELLED states.
- Delegation to CheckpointService.selective_rollback() to perform the actual rollback logic and obtain a RollbackResult.
- Emission of PLAN_ROLLED_BACK as a domain event to reflect the completed rollback.
- checkpoint_service is accepted as an optional constructor parameter; if not provided, a PlanError is raised to preserve backward compatibility.
- Updated CLI behavior in src/cleveragents/cli/commands/plan.py so agents plan rollback routes through PlanLifecycleService.rollback_plan() rather than calling CheckpointService.selective_rollback() directly.
- Updated PlanLifecycleService module docstring to include rollback_plan in the documented API.
- Added Behave feature file features/plan_lifecycle_rollback.feature with 11 scenarios covering state validation, domain events, and delegation.
- Added step implementations in features/steps/plan_lifecycle_rollback_steps.py to support the new scenarios.
- Key design decisions
- rollback_plan returns RollbackResult (the same result type produced by CheckpointService.selective_rollback) so the CLI can display rollback details consistently.
- Terminal states APPLIED and CANCELLED are disallowed for rollback to prevent inconsistent or invalid state transitions.
- checkpoint_service is optional in the PlanLifecycleService constructor; when omitted (None), a PlanError is raised to retain backward compatibility while signaling explicit dependency requirements.
- CLI UI remains powered by CheckpointService for metadata enrichment (e.g., confirmation prompts), but the actual rollback action is performed via PlanLifecycleService to ensure proper domain workflow and event emission.
- Technical implications
- All rollback logic now flows through the domain service layer (PlanLifecycleService) to preserve invariants and emit domain events, rather than allowing ad-hoc UI routes to bypass service validation.
- The UI can still retrieve checkpoint metadata for user confirmation, but the operation that modifies state uses the new rollback_plan pathway.
- Tests and behavior coverage were expanded via the new Behave feature and step implementations to validate state handling, events, and delegation.
- Affected modules/components
- src/cleveragents/infrastructure/events/types.py
- src/cleveragents/application/services/plan_lifecycle_service.py
- src/cleveragents/cli/commands/plan.py
- PlanLifecycleService module docstring
- features/plan_lifecycle_rollback.feature
- features/steps/plan_lifecycle_rollback_steps.py
ISSUES CLOSED: #3677
307 lines
13 KiB
Gherkin
307 lines
13 KiB
Gherkin
Feature: Tool and Validation CLI commands
|
|
As a developer
|
|
I want to manage tools and validations via CLI commands
|
|
So that I can register, list, show, and remove tools and validations
|
|
|
|
Background:
|
|
Given a tool CLI runner with mocks
|
|
And a mocked tool registry service
|
|
|
|
# Tool add command tests
|
|
Scenario: Add tool via config file
|
|
Given a valid tool config YAML file
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Registered"
|
|
|
|
Scenario: Add tool with --update flag when tool exists
|
|
Given a valid tool config YAML file
|
|
And the tool already exists in the registry
|
|
When I run tool CLI add with --config and --update
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Updated"
|
|
|
|
Scenario: Add tool with --update flag when tool does not exist
|
|
Given a valid tool config YAML file
|
|
And the tool does not exist in the registry
|
|
When I run tool CLI add with --config and --update
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Registered"
|
|
|
|
Scenario: Add tool with missing config file fails
|
|
When I run tool CLI add with --config pointing to a missing file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with invalid YAML fails
|
|
Given an invalid YAML tool config file
|
|
When I run tool CLI add with --config pointing to the invalid YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with duplicate name fails
|
|
Given a valid tool config YAML file
|
|
And the tool registry raises a duplicate error on register
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with json format output
|
|
Given a valid tool config YAML file
|
|
When I run tool CLI add with --config and --format json
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "local/test-tool"
|
|
|
|
# Tool list command tests
|
|
Scenario: List all tools
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list
|
|
Then the tool CLI list should show all tools
|
|
|
|
Scenario: List tools with namespace filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with namespace filter "local"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with source filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with source filter "custom"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with type filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with type filter "tool"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with type filter validation
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with type filter "validation"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with invalid type filter
|
|
When I run tool CLI list with invalid type "unknown"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: List tools with regex filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with regex filter "line-.*"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with invalid regex
|
|
When I run tool CLI list with invalid regex "[invalid"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: List tools when empty
|
|
Given there are no mocked tools
|
|
When I run tool CLI list
|
|
Then the tool CLI should show no tools message
|
|
|
|
Scenario: List tools with json format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "json"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with yaml format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "yaml"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with plain format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "plain"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with table format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "table"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
# Tool show command tests
|
|
Scenario: Show tool by name
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show with name "local/test-tool"
|
|
Then the tool CLI should show the tool details
|
|
|
|
Scenario: Show tool not found
|
|
Given there is no tool with name "local/missing"
|
|
When I run tool CLI show with name "local/missing"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Show tool with json format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "json"
|
|
Then the tool CLI show should succeed
|
|
|
|
Scenario: Show tool with yaml format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "yaml"
|
|
Then the tool CLI show should succeed
|
|
|
|
Scenario: Show tool with plain format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "plain"
|
|
Then the tool CLI show should succeed
|
|
|
|
# Tool remove command tests
|
|
Scenario: Remove tool by name with confirmation
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI remove with name "local/test-tool" and --yes
|
|
Then the tool CLI remove should succeed
|
|
And the tool CLI output should contain "Removed tool"
|
|
|
|
Scenario: Remove tool not found
|
|
Given there is no tool with name "local/missing"
|
|
When I run tool CLI remove with name "local/missing" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Remove tool with json format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI remove confirmed name "local/test-tool" with fmt "json"
|
|
Then the tool CLI remove should succeed
|
|
|
|
Scenario: Remove tool with service error
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
And the tool registry raises an error on remove
|
|
When I run tool CLI remove with name "local/test-tool" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add command tests
|
|
Scenario: Add validation via config file
|
|
Given a valid validation config YAML file
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Registered"
|
|
|
|
Scenario: Add validation with --update flag when validation exists
|
|
Given a valid validation config YAML file
|
|
And the validation already exists in the registry
|
|
When I run validation CLI add with --config and --update
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Updated"
|
|
|
|
Scenario: Add validation with missing config file fails
|
|
When I run validation CLI add with --config pointing to a missing file
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Add validation with invalid YAML fails
|
|
Given an invalid YAML validation config file
|
|
When I run validation CLI add with --config pointing to the invalid YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Add validation with json format output
|
|
Given a valid validation config YAML file
|
|
When I run validation CLI add with --config and --format json
|
|
Then the validation CLI add should succeed
|
|
|
|
# Validation attach command tests
|
|
Scenario: Attach validation to resource
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI attach should succeed
|
|
And the validation CLI output should contain "Attached validation"
|
|
|
|
Scenario: Attach validation with project scope
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach scoped "resource/r1" "local/test-val" with project "myproj"
|
|
Then the validation CLI attach should succeed
|
|
|
|
Scenario: Attach validation with extra args using named option format
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach with named option "resource/r1" "local/test-val" option "--threshold" value "80"
|
|
Then the validation CLI attach should succeed
|
|
|
|
Scenario: Attach validation with invalid arg format (not a named option)
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach with extra args "resource/r1" "local/test-val" arg is "badarg"
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Attach validation not found
|
|
Given the validation is not found for attaching
|
|
When I run validation CLI attach "resource/r1" "local/missing-val"
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Attach validation with json format
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach formatted "resource/r1" "local/test-val" fmt is "json"
|
|
Then the validation CLI attach should succeed
|
|
|
|
# Validation detach command tests
|
|
Scenario: Detach validation with confirmation
|
|
Given a mocked attachment exists for detaching
|
|
When I run validation CLI detach "att-123" with --yes
|
|
Then the validation CLI detach should succeed
|
|
And the validation CLI output should contain "Detached validation"
|
|
|
|
Scenario: Detach validation not found
|
|
Given the attachment is not found for detaching
|
|
When I run validation CLI detach "att-missing" with --yes
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Detach validation with json format
|
|
Given a mocked attachment exists for detaching
|
|
When I run validation CLI detach formatted "att-123" confirmed fmt "json"
|
|
Then the validation CLI detach should succeed
|
|
|
|
# Tool add with CleverAgentsError
|
|
Scenario: Add tool with CleverAgentsError from service
|
|
Given a valid tool config YAML file
|
|
And the tool registry raises a CleverAgentsError on register
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
# Tool list with CleverAgentsError
|
|
Scenario: List tools with CleverAgentsError from service
|
|
Given the tool registry raises a CleverAgentsError on list
|
|
When I run tool CLI list
|
|
Then the tool CLI command should abort
|
|
|
|
# Tool show with CleverAgentsError
|
|
Scenario: Show tool with CleverAgentsError from service
|
|
Given the tool registry raises a CleverAgentsError on get
|
|
When I run tool CLI show with name "local/test-tool"
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add with CleverAgentsError
|
|
Scenario: Add validation with CleverAgentsError from service
|
|
Given a valid validation config YAML file
|
|
And the validation registry raises a CleverAgentsError on register
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation attach with CleverAgentsError
|
|
Scenario: Attach validation with CleverAgentsError from service
|
|
Given the validation registry raises a CleverAgentsError on attach
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation detach with CleverAgentsError
|
|
Scenario: Detach validation with CleverAgentsError from service
|
|
Given the validation registry raises a CleverAgentsError on detach
|
|
When I run validation CLI detach "att-123" with --yes
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation add with update and non-existent
|
|
Scenario: Add validation with --update flag when validation does not exist
|
|
Given a valid validation config YAML file
|
|
And the validation does not exist in the registry
|
|
When I run validation CLI add with --config and --update
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Registered"
|
|
|
|
# Tool remove with remove returning false
|
|
Scenario: Remove tool with remove returning false
|
|
Given there is a mocked tool with name "local/fail-remove"
|
|
And the tool registry returns false on remove
|
|
When I run tool CLI remove with name "local/fail-remove" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add with validation error
|
|
Scenario: Add validation with validation error from service
|
|
Given a valid validation config YAML file
|
|
And the validation registry raises a validation error on register
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation attach with validation error
|
|
Scenario: Attach validation with validation error from service
|
|
Given the validation registry raises a validation error on attach
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI command should abort
|