freemo
1d36449a98
CI / build (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / security (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
Remove the --mode/-m CLI flag from the validation attach command to align with the specification, which defines validation mode as an inherent property of the validation definition set at registration time via validation add, not as a per-attachment override. The service layer (ToolRegistryService.attach_validation) no longer accepts mode as a parameter; instead it reads the mode from the validation's registered definition. The ToolRegistryRepository _to_legacy_domain now exposes the mode field so the service can access it. All tests that passed --mode to the CLI or service have been updated or removed. The invalid-mode validation scenarios were removed since the mode is no longer caller-supplied. Coverage remains at 98.7%. ISSUES CLOSED: #913 Co-authored-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-committed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
307 lines
13 KiB
Gherkin
307 lines
13 KiB
Gherkin
Feature: Tool and Validation CLI commands
|
|
As a developer
|
|
I want to manage tools and validations via CLI commands
|
|
So that I can register, list, show, and remove tools and validations
|
|
|
|
Background:
|
|
Given a tool CLI runner with mocks
|
|
And a mocked tool registry service
|
|
|
|
# Tool add command tests
|
|
Scenario: Add tool via config file
|
|
Given a valid tool config YAML file
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Registered"
|
|
|
|
Scenario: Add tool with --update flag when tool exists
|
|
Given a valid tool config YAML file
|
|
And the tool already exists in the registry
|
|
When I run tool CLI add with --config and --update
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Updated"
|
|
|
|
Scenario: Add tool with --update flag when tool does not exist
|
|
Given a valid tool config YAML file
|
|
And the tool does not exist in the registry
|
|
When I run tool CLI add with --config and --update
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "Tool Registered"
|
|
|
|
Scenario: Add tool with missing config file fails
|
|
When I run tool CLI add with --config pointing to a missing file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with invalid YAML fails
|
|
Given an invalid YAML tool config file
|
|
When I run tool CLI add with --config pointing to the invalid YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with duplicate name fails
|
|
Given a valid tool config YAML file
|
|
And the tool registry raises a duplicate error on register
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Add tool with json format output
|
|
Given a valid tool config YAML file
|
|
When I run tool CLI add with --config and --format json
|
|
Then the tool CLI add should succeed
|
|
And the tool CLI output should contain "local/test-tool"
|
|
|
|
# Tool list command tests
|
|
Scenario: List all tools
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list
|
|
Then the tool CLI list should show all tools
|
|
|
|
Scenario: List tools with namespace filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with namespace filter "local"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with source filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with source filter "custom"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with type filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with type filter "tool"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with type filter validation
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with type filter "validation"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with invalid type filter
|
|
When I run tool CLI list with invalid type "unknown"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: List tools with regex filter
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with regex filter "line-.*"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with invalid regex
|
|
When I run tool CLI list with invalid regex "[invalid"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: List tools when empty
|
|
Given there are no mocked tools
|
|
When I run tool CLI list
|
|
Then the tool CLI should show no tools message
|
|
|
|
Scenario: List tools with json format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "json"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with yaml format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "yaml"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with plain format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "plain"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
Scenario: List tools with table format
|
|
Given there are mocked tools in the registry
|
|
When I run tool CLI list with format "table"
|
|
Then the tool CLI list should succeed with results
|
|
|
|
# Tool show command tests
|
|
Scenario: Show tool by name
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show with name "local/test-tool"
|
|
Then the tool CLI should show the tool details
|
|
|
|
Scenario: Show tool not found
|
|
Given there is no tool with name "local/missing"
|
|
When I run tool CLI show with name "local/missing"
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Show tool with json format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "json"
|
|
Then the tool CLI show should succeed
|
|
|
|
Scenario: Show tool with yaml format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "yaml"
|
|
Then the tool CLI show should succeed
|
|
|
|
Scenario: Show tool with plain format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI show for name "local/test-tool" using format "plain"
|
|
Then the tool CLI show should succeed
|
|
|
|
# Tool remove command tests
|
|
Scenario: Remove tool by name with confirmation
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI remove with name "local/test-tool" and --yes
|
|
Then the tool CLI remove should succeed
|
|
And the tool CLI output should contain "Removed tool"
|
|
|
|
Scenario: Remove tool not found
|
|
Given there is no tool with name "local/missing"
|
|
When I run tool CLI remove with name "local/missing" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
Scenario: Remove tool with json format
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
When I run tool CLI remove confirmed name "local/test-tool" with fmt "json"
|
|
Then the tool CLI remove should succeed
|
|
|
|
Scenario: Remove tool with service error
|
|
Given there is a mocked tool with name "local/test-tool"
|
|
And the tool registry raises an error on remove
|
|
When I run tool CLI remove with name "local/test-tool" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add command tests
|
|
Scenario: Add validation via config file
|
|
Given a valid validation config YAML file
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Registered"
|
|
|
|
Scenario: Add validation with --update flag when validation exists
|
|
Given a valid validation config YAML file
|
|
And the validation already exists in the registry
|
|
When I run validation CLI add with --config and --update
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Updated"
|
|
|
|
Scenario: Add validation with missing config file fails
|
|
When I run validation CLI add with --config pointing to a missing file
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Add validation with invalid YAML fails
|
|
Given an invalid YAML validation config file
|
|
When I run validation CLI add with --config pointing to the invalid YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Add validation with json format output
|
|
Given a valid validation config YAML file
|
|
When I run validation CLI add with --config and --format json
|
|
Then the validation CLI add should succeed
|
|
|
|
# Validation attach command tests
|
|
Scenario: Attach validation to resource
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI attach should succeed
|
|
And the validation CLI output should contain "Attached validation"
|
|
|
|
Scenario: Attach validation with project scope
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach scoped "resource/r1" "local/test-val" with project "myproj"
|
|
Then the validation CLI attach should succeed
|
|
|
|
Scenario: Attach validation with extra args
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach with extra args "resource/r1" "local/test-val" arg is "threshold=80"
|
|
Then the validation CLI attach should succeed
|
|
|
|
Scenario: Attach validation with invalid arg format
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach with extra args "resource/r1" "local/test-val" arg is "badarg"
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Attach validation not found
|
|
Given the validation is not found for attaching
|
|
When I run validation CLI attach "resource/r1" "local/missing-val"
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Attach validation with json format
|
|
Given a mocked validation exists for attaching
|
|
When I run validation CLI attach formatted "resource/r1" "local/test-val" fmt is "json"
|
|
Then the validation CLI attach should succeed
|
|
|
|
# Validation detach command tests
|
|
Scenario: Detach validation with confirmation
|
|
Given a mocked attachment exists for detaching
|
|
When I run validation CLI detach "att-123" with --yes
|
|
Then the validation CLI detach should succeed
|
|
And the validation CLI output should contain "Detached validation"
|
|
|
|
Scenario: Detach validation not found
|
|
Given the attachment is not found for detaching
|
|
When I run validation CLI detach "att-missing" with --yes
|
|
Then the validation CLI command should abort
|
|
|
|
Scenario: Detach validation with json format
|
|
Given a mocked attachment exists for detaching
|
|
When I run validation CLI detach formatted "att-123" confirmed fmt "json"
|
|
Then the validation CLI detach should succeed
|
|
|
|
# Tool add with CleverAgentsError
|
|
Scenario: Add tool with CleverAgentsError from service
|
|
Given a valid tool config YAML file
|
|
And the tool registry raises a CleverAgentsError on register
|
|
When I run tool CLI add with --config pointing to the YAML file
|
|
Then the tool CLI command should abort
|
|
|
|
# Tool list with CleverAgentsError
|
|
Scenario: List tools with CleverAgentsError from service
|
|
Given the tool registry raises a CleverAgentsError on list
|
|
When I run tool CLI list
|
|
Then the tool CLI command should abort
|
|
|
|
# Tool show with CleverAgentsError
|
|
Scenario: Show tool with CleverAgentsError from service
|
|
Given the tool registry raises a CleverAgentsError on get
|
|
When I run tool CLI show with name "local/test-tool"
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add with CleverAgentsError
|
|
Scenario: Add validation with CleverAgentsError from service
|
|
Given a valid validation config YAML file
|
|
And the validation registry raises a CleverAgentsError on register
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation attach with CleverAgentsError
|
|
Scenario: Attach validation with CleverAgentsError from service
|
|
Given the validation registry raises a CleverAgentsError on attach
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation detach with CleverAgentsError
|
|
Scenario: Detach validation with CleverAgentsError from service
|
|
Given the validation registry raises a CleverAgentsError on detach
|
|
When I run validation CLI detach "att-123" with --yes
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation add with update and non-existent
|
|
Scenario: Add validation with --update flag when validation does not exist
|
|
Given a valid validation config YAML file
|
|
And the validation does not exist in the registry
|
|
When I run validation CLI add with --config and --update
|
|
Then the validation CLI add should succeed
|
|
And the validation CLI output should contain "Validation Registered"
|
|
|
|
# Tool remove with remove returning false
|
|
Scenario: Remove tool with remove returning false
|
|
Given there is a mocked tool with name "local/fail-remove"
|
|
And the tool registry returns false on remove
|
|
When I run tool CLI remove with name "local/fail-remove" and --yes
|
|
Then the tool CLI command should abort
|
|
|
|
# Validation add with validation error
|
|
Scenario: Add validation with validation error from service
|
|
Given a valid validation config YAML file
|
|
And the validation registry raises a validation error on register
|
|
When I run validation CLI add with --config pointing to the YAML file
|
|
Then the validation CLI command should abort
|
|
|
|
# Validation attach with validation error
|
|
Scenario: Attach validation with validation error from service
|
|
Given the validation registry raises a validation error on attach
|
|
When I run validation CLI attach "resource/r1" "local/test-val"
|
|
Then the validation CLI command should abort
|