freemo
1d36449a98
CI / build (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / security (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
Remove the --mode/-m CLI flag from the validation attach command to align with the specification, which defines validation mode as an inherent property of the validation definition set at registration time via validation add, not as a per-attachment override. The service layer (ToolRegistryService.attach_validation) no longer accepts mode as a parameter; instead it reads the mode from the validation's registered definition. The ToolRegistryRepository _to_legacy_domain now exposes the mode field so the service can access it. All tests that passed --mode to the CLI or service have been updated or removed. The invalid-mode validation scenarios were removed since the mode is no longer caller-supplied. Coverage remains at 98.7%. ISSUES CLOSED: #913 Co-authored-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-committed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
975 lines
39 KiB
Gherkin
975 lines
39 KiB
Gherkin
Feature: Consolidated Tool
|
||
Combined scenarios from: tool_lifecycle_coverage_boost, tool_model, tool_registry_service_coverage, tool_registry_service_fallback_coverage, tool_registry_service_uncovered_branches, tool_runtime
|
||
|
||
# ============================================================
|
||
# Originally from: tool_lifecycle_coverage_boost.feature
|
||
# Feature: Tool lifecycle coverage boost
|
||
# ============================================================
|
||
|
||
Scenario: Removing the last tool from a plan deletes the plan entry
|
||
Given I create a coverage-boost lifecycle cache
|
||
And I have a coverage-boost mock tool instance named "builtin/only-tool"
|
||
When I put the coverage-boost instance in cache for plan "plan-rm-1" and tool "builtin/only-tool"
|
||
Then the coverage-boost cache should have 1 plans
|
||
When I remove tool "builtin/only-tool" from coverage-boost cache plan "plan-rm-1"
|
||
Then the coverage-boost cache should have 0 plans
|
||
And the coverage-boost cache internal dict should not contain plan "plan-rm-1"
|
||
|
||
|
||
Scenario: Removing a tool that does not exist in cache returns None
|
||
Given I create a coverage-boost lifecycle cache
|
||
When I remove tool "builtin/ghost" from coverage-boost cache plan "plan-rm-2"
|
||
Then the coverage-boost remove result should be None
|
||
|
||
# ── ToolLifecycleCache.get_plan_tools with empty cache ──────────────────
|
||
|
||
|
||
Scenario: Getting plan tools for a plan with no activated tools returns empty list
|
||
Given I create a coverage-boost lifecycle cache
|
||
Then the coverage-boost plan tools for plan "plan-empty" should be an empty list
|
||
|
||
# ── ToolRuntime.activate with ToolCancelledError ────────────────────────
|
||
|
||
|
||
Scenario: Activating a tool that raises ToolCancelledError propagates the error
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost cancelling-activate tool "builtin/cancel-act"
|
||
And I create a coverage-boost execution context with plan_id "plan-cancel-act"
|
||
When I try to activate the cancelling tool "builtin/cancel-act" in coverage-boost runtime
|
||
Then a ToolCancelledError should have been raised in coverage-boost activation
|
||
|
||
# ── ToolRuntime.execute with None cached instance ──────────────────────
|
||
|
||
|
||
Scenario: Executing a tool when cached instance is None raises ToolNotActivatedError
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost tool "builtin/null-cache" with a mock that nullifies cache
|
||
And I create a coverage-boost execution context with plan_id "plan-null-cache"
|
||
When I try to execute the null-cache tool "builtin/null-cache" in coverage-boost runtime
|
||
Then a ToolNotActivatedError should have been raised in coverage-boost execution
|
||
|
||
# ── ToolRuntime.execute with ToolCancelledError during execution ────────
|
||
|
||
|
||
Scenario: Executing a tool that raises ToolCancelledError records trace and re-raises
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost cancelling-execute tool "builtin/cancel-exec"
|
||
And I create a coverage-boost execution context with plan_id "plan-cancel-exec"
|
||
When I try to execute the cancelling-execute tool "builtin/cancel-exec" in coverage-boost runtime
|
||
Then a ToolCancelledError should have been raised in coverage-boost execution
|
||
And the coverage-boost context should have 1 traces
|
||
And the coverage-boost last trace should show success False
|
||
And the coverage-boost last trace error should be "Cancelled"
|
||
|
||
# ── ToolRuntime.deactivate with tool not in cache ──────────────────────
|
||
|
||
|
||
Scenario: Deactivating a tool that was never activated is a silent no-op
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost read-only tool "builtin/never-active" with a mock
|
||
And I create a coverage-boost execution context with plan_id "plan-deact-none"
|
||
When I deactivate tool "builtin/never-active" in the coverage-boost runtime
|
||
Then no coverage-boost exception should have been raised
|
||
And the coverage-boost mock "builtin/never-active" should not have been deactivated
|
||
|
||
# ── ToolRuntime.deactivate_plan with empty plan ────────────────────────
|
||
|
||
|
||
Scenario: Deactivating a plan with no activated tools is a silent no-op
|
||
Given I create a coverage-boost tool runtime
|
||
And I create a coverage-boost execution context with plan_id "plan-deact-empty"
|
||
When I deactivate plan in the coverage-boost runtime
|
||
Then no coverage-boost exception should have been raised
|
||
|
||
# ── ToolLifecycleCache.remove with multiple tools leaves remaining ─────
|
||
|
||
|
||
Scenario: Removing one of two tools keeps the plan entry with the remaining tool
|
||
Given I create a coverage-boost lifecycle cache
|
||
And I have a coverage-boost mock tool instance named "builtin/tool-a"
|
||
And I have a second coverage-boost mock tool instance named "builtin/tool-b"
|
||
When I put the coverage-boost instance in cache for plan "plan-rm-multi" and tool "builtin/tool-a"
|
||
And I put the second coverage-boost instance in cache for plan "plan-rm-multi" and tool "builtin/tool-b"
|
||
Then the coverage-boost cache should have 1 plans
|
||
When I remove tool "builtin/tool-a" from coverage-boost cache plan "plan-rm-multi"
|
||
Then the coverage-boost cache should have 1 plans
|
||
And the coverage-boost cache get for plan "plan-rm-multi" and tool "builtin/tool-b" should not be None
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_model.feature
|
||
# Feature: Tool and Validation Domain Models
|
||
# ============================================================
|
||
|
||
Scenario: Create a custom tool with valid fields
|
||
When I create a tool with name "local/line-counter" source "custom" and code "print('hello')"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/line-counter"
|
||
And the tool model source should be "custom"
|
||
And the tool model type should be "tool"
|
||
And the tool model namespace should be "local"
|
||
And the tool model short name should be "line-counter"
|
||
|
||
|
||
Scenario: Create a builtin tool
|
||
When I create a tool with name "builtin/file-read" and source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model source should be "builtin"
|
||
|
||
|
||
Scenario: Create an MCP tool with required fields
|
||
When I create an MCP tool with name "devops/docker-build" server "devops-mcp" and tool name "docker_build"
|
||
Then the tool model should be created
|
||
And the tool model source should be "mcp"
|
||
|
||
|
||
Scenario: Create an agent_skill tool
|
||
When I create an agent skill tool with name "skills/formatter" and path "/skills/formatter"
|
||
Then the tool model should be created
|
||
And the tool model source should be "agent_skill"
|
||
|
||
# ---- Tool Name Validation ----
|
||
|
||
|
||
Scenario: Tool name must match namespace/name pattern
|
||
When I try to create a tool with invalid name "no-namespace"
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "namespace/name"
|
||
|
||
|
||
Scenario: Tool name rejects spaces
|
||
When I try to create a tool with invalid name "local/bad name"
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool name rejects double slashes
|
||
When I try to create a tool with invalid name "local//bad"
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool name allows hyphens and underscores
|
||
When I create a tool with name "my-ns/my_tool" and source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model namespace should be "my-ns"
|
||
And the tool model short name should be "my_tool"
|
||
|
||
# ---- Source-conditional field requirements ----
|
||
|
||
|
||
Scenario: Custom source requires code
|
||
When I try to create a tool with name "local/test" source "custom" without code
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "code is required"
|
||
|
||
|
||
Scenario: MCP source requires mcp_server
|
||
When I try to create an MCP tool without mcp_server
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "mcp_server is required"
|
||
|
||
|
||
Scenario: MCP source requires mcp_tool_name
|
||
When I try to create an MCP tool without mcp_tool_name
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "mcp_tool_name is required"
|
||
|
||
|
||
Scenario: Agent skill source requires agent_skill_path
|
||
When I try to create an agent skill tool without path
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "agent_skill_path is required"
|
||
|
||
# ---- Capability constraint enforcement ----
|
||
|
||
|
||
Scenario: Read-only capability enforces writes=false
|
||
When I try to create a tool capability with read_only true and writes true
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "read_only tool cannot have writes"
|
||
|
||
|
||
Scenario: Read-only capability enforces checkpointable=false
|
||
When I try to create a tool capability with read_only true and checkpointable true
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "read_only tool cannot have checkpointable"
|
||
|
||
|
||
Scenario: Non-read-only capability allows writes
|
||
When I create a tool capability with writes true
|
||
Then the tool capability should have writes true
|
||
|
||
# ---- ResourceSlot binding validation ----
|
||
|
||
|
||
Scenario: Static binding requires static_resource
|
||
When I try to create a resource slot with static binding and no static resource
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "static_resource is required"
|
||
|
||
|
||
Scenario: Non-static binding forbids static_resource
|
||
When I try to create a resource slot with contextual binding and static resource set
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "static_resource must not be set"
|
||
|
||
|
||
Scenario: Static binding with static_resource succeeds
|
||
When I create a resource slot with static binding and static resource "my-repo"
|
||
Then the tool resource slot should be created
|
||
|
||
|
||
Scenario: ResourceSlot name must be valid identifier
|
||
When I try to create a resource slot with invalid name "123bad"
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "valid Python identifier"
|
||
|
||
# ---- Validation model creation ----
|
||
|
||
|
||
Scenario: Create a custom validation with forced read-only
|
||
When I create a validation with name "qa/coverage-check" source "custom" and code "return True"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
And the tool validation capability should not be checkpointable
|
||
|
||
|
||
Scenario: Validation forces tool_type to validation
|
||
When I create a validation with name "qa/test-check" source "builtin"
|
||
Then the tool model type should be "validation"
|
||
|
||
|
||
Scenario: Validation mode defaults to required
|
||
When I create a validation with name "qa/check" source "builtin"
|
||
Then the tool validation mode should be "required"
|
||
|
||
|
||
Scenario: Validation can be informational
|
||
When I create an informational validation with name "qa/info-check" source "builtin"
|
||
Then the tool validation mode should be "informational"
|
||
|
||
# ---- Validation wraps/transform requirements ----
|
||
|
||
|
||
Scenario: Wrapped validation requires transform
|
||
When I try to create a wrapped validation without transform
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "transform is required"
|
||
|
||
|
||
Scenario: Wrapped validation forbids code
|
||
When I try to create a wrapped validation with code
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "code must not be set"
|
||
|
||
|
||
Scenario: Wrapped validation requires source=wrapped
|
||
When I try to create a validation with wraps and wrong source
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "source must be 'wrapped'"
|
||
|
||
|
||
Scenario: argument_mapping only valid with wraps
|
||
When I try to create a validation with argument_mapping but no wraps
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "argument_mapping is only valid"
|
||
|
||
|
||
Scenario: Valid wrapped validation
|
||
When I create a valid wrapped validation
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool validation wraps should be "devops/run-linter"
|
||
|
||
# ---- from_config loading ----
|
||
|
||
|
||
Scenario: Load tool from config dict
|
||
When I load a tool from config with name "local/my-tool" source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/my-tool"
|
||
|
||
|
||
Scenario: Load tool from config missing name raises error
|
||
When I try to load a tool from config missing "name"
|
||
Then a tool model config error should be raised with "name"
|
||
|
||
|
||
Scenario: Load tool from config missing description raises error
|
||
When I try to load a tool from config missing "description"
|
||
Then a tool model config error should be raised with "description"
|
||
|
||
|
||
Scenario: Load tool from config missing source raises error
|
||
When I try to load a tool from config missing "source"
|
||
Then a tool model config error should be raised with "source"
|
||
|
||
|
||
Scenario: Load validation from config with wraps
|
||
When I load a validation from config with wraps "devops/linter"
|
||
Then the tool model should be created
|
||
And the tool model source should be "wrapped"
|
||
And the tool validation wraps should be "devops/linter"
|
||
|
||
|
||
Scenario: Load validation from config missing name raises error
|
||
When I try to load a validation config missing "name"
|
||
Then a tool model config error should be raised with "name"
|
||
|
||
# ---- as_cli_dict output stability ----
|
||
|
||
|
||
Scenario: Tool as_cli_dict has required keys
|
||
When I create a tool and call as_cli_dict
|
||
Then the tool cli dict should have key "name"
|
||
And the tool cli dict should have key "description"
|
||
And the tool cli dict should have key "source"
|
||
And the tool cli dict should have key "tool_type"
|
||
And the tool cli dict should have key "namespace"
|
||
And the tool cli dict should have key "short_name"
|
||
And the tool cli dict should have key "capability"
|
||
And the tool cli dict should have key "timeout"
|
||
|
||
|
||
Scenario: Validation as_cli_dict includes mode
|
||
When I create a validation and call as_cli_dict
|
||
Then the tool cli dict should have key "mode"
|
||
|
||
# ---- Tool timeout validation ----
|
||
|
||
|
||
Scenario: Tool timeout must be >= 1
|
||
When I try to create a tool with timeout 0
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool timeout default is 300
|
||
When I create a tool with name "local/default-timeout" and source "builtin"
|
||
Then the tool model timeout should be 300
|
||
|
||
# ---- ToolLifecycle ----
|
||
|
||
|
||
Scenario: ToolLifecycle accepts optional hooks
|
||
When I create a tool lifecycle with discover "my.discover"
|
||
Then the tool lifecycle discover should be "my.discover"
|
||
|
||
# ---- Enum values ----
|
||
|
||
|
||
Scenario: ToolSource enum has all expected values
|
||
Then the ToolSource enum should have values "mcp,agent_skill,builtin,custom,wrapped"
|
||
|
||
|
||
Scenario: ToolType enum has all expected values
|
||
Then the ToolType enum should have values "tool,validation"
|
||
|
||
|
||
Scenario: ValidationMode enum has all expected values
|
||
Then the ValidationMode enum should have values "required,informational"
|
||
|
||
|
||
Scenario: CheckpointScope enum has all expected values
|
||
Then the CheckpointScope enum should have values "file,transaction,snapshot,composite"
|
||
|
||
|
||
Scenario: BindingMode enum has all expected values
|
||
Then the BindingMode enum should have values "contextual,static,parameter"
|
||
|
||
|
||
Scenario: ResourceAccessMode enum has all expected values
|
||
Then the ResourceAccessMode enum should have values "read_only,read_write"
|
||
|
||
# ---- Tool with resource slots via from_config ----
|
||
|
||
|
||
Scenario: Load tool from config with resource slots
|
||
When I load a tool from config with resource slots
|
||
Then the tool model should be created
|
||
And the tool model should have 1 resource slot
|
||
|
||
# ---- Validation from_config missing description ----
|
||
|
||
|
||
Scenario: Load validation from config missing description raises error
|
||
When I try to load a validation config missing "description"
|
||
Then a tool model config error should be raised with "description"
|
||
|
||
# ---- Tool description validation ----
|
||
|
||
|
||
Scenario: Tool description must not be empty
|
||
When I try to create a tool with empty description
|
||
Then a tool model validation error should be raised
|
||
|
||
# ---- Invalid scenarios ----
|
||
|
||
|
||
Scenario: Tool with conflicting capability flags
|
||
When I try to create a tool capability with read_only true and writes true
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Validation wraps with custom source is rejected
|
||
When I try to create a validation with wraps and wrong source
|
||
Then a tool model validation error should be raised
|
||
|
||
# ---- as_cli_dict coverage for optional fields ----
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes code when set
|
||
When I create a custom tool and call as_cli_dict
|
||
Then the tool cli dict should have key "code"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes mcp fields when set
|
||
When I create an MCP tool and call as_cli_dict
|
||
Then the tool cli dict should have key "mcp_server"
|
||
And the tool cli dict should have key "mcp_tool_name"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes agent_skill_path when set
|
||
When I create an agent skill tool and call as_cli_dict
|
||
Then the tool cli dict should have key "agent_skill_path"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes schemas when set
|
||
When I create a tool with schemas and call as_cli_dict
|
||
Then the tool cli dict should have key "input_schema"
|
||
And the tool cli dict should have key "output_schema"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes side_effects and cost_profile
|
||
When I create a tool with side effects and cost profile and call as_cli_dict
|
||
Then the tool cli dict capability should have key "side_effects"
|
||
And the tool cli dict capability should have key "cost_profile"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes resource slots when set
|
||
When I create a tool with resource slots and call as_cli_dict
|
||
Then the tool cli dict should have key "resource_slots"
|
||
|
||
# ---- Validation as_cli_dict wraps fields ----
|
||
|
||
|
||
Scenario: Validation as_cli_dict includes wraps and transform
|
||
When I create a wrapped validation and call as_cli_dict
|
||
Then the tool cli dict should have key "wraps"
|
||
And the tool cli dict should have key "transform"
|
||
And the tool cli dict should have key "argument_mapping"
|
||
|
||
# ---- Validation from_config without wraps ----
|
||
|
||
|
||
Scenario: Load validation from config without wraps needs source
|
||
When I try to load a validation config without source or wraps
|
||
Then a tool model config error should be raised with "source"
|
||
|
||
|
||
Scenario: Load validation from config with explicit source
|
||
When I load a validation from config with source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model source should be "builtin"
|
||
|
||
# ---- Validation forces read_only on writable capability ----
|
||
|
||
|
||
Scenario: Validation forces read_only on writable capability
|
||
When I create a validation with writes capability
|
||
Then the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
And the tool validation capability should not be checkpointable
|
||
|
||
# ---- YAML loader output alignment (mirrors Robot smoke suite) ----
|
||
|
||
|
||
Scenario: Load custom tool from example YAML config
|
||
When I load a tool from YAML file "examples/tools/custom-tool.yaml"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/line-counter"
|
||
And the tool model source should be "custom"
|
||
And the tool model should have 1 resource slot
|
||
And the tool model timeout should be 60
|
||
|
||
|
||
Scenario: Load MCP tool from example YAML config
|
||
When I load a tool from YAML file "examples/tools/mcp-tool.yaml"
|
||
Then the tool model should be created
|
||
And the tool model name should be "devops/docker-build"
|
||
And the tool model source should be "mcp"
|
||
And the tool model timeout should be 600
|
||
|
||
|
||
Scenario: Load wrapped validation from example YAML config
|
||
When I load a validation from YAML file "examples/validations/wrapped-validation.yaml"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool model source should be "wrapped"
|
||
And the tool validation wraps should be "devops/run-linter"
|
||
And the tool validation mode should be "required"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
|
||
|
||
Scenario: Load required validation from example YAML config
|
||
When I load a validation from YAML file "examples/validations/required-validation.yaml"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool model source should be "custom"
|
||
And the tool validation mode should be "required"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_coverage.feature
|
||
# Feature: Tool Registry Service – uncovered error paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool propagates DuplicateToolError from repo.create
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo create method raises DuplicateToolError
|
||
When I attempt to register the tool via the coverage service
|
||
Then a coverage DuplicateToolError should be raised
|
||
|
||
# --- register_tool: DatabaseError propagation ------------------------------
|
||
|
||
|
||
Scenario: register_tool propagates DatabaseError from repo.create
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo create method raises DatabaseError
|
||
When I attempt to register the tool via the coverage service
|
||
Then a coverage DatabaseError should be raised with message containing "Failed to create"
|
||
|
||
# --- update_tool: ToolNotFoundError propagation ----------------------------
|
||
|
||
|
||
Scenario: update_tool propagates ToolNotFoundError from repo.update
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo update method raises ToolNotFoundError
|
||
When I attempt to update a tool via the coverage service
|
||
Then a coverage ToolNotFoundError should be raised
|
||
|
||
# --- update_tool: DatabaseError propagation --------------------------------
|
||
|
||
|
||
Scenario: update_tool propagates DatabaseError from repo.update
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo update method raises DatabaseError
|
||
When I attempt to update a tool via the coverage service
|
||
Then a coverage DatabaseError should be raised with message containing "Failed to update"
|
||
|
||
# --- remove_tool: returns False for missing tool ---------------------------
|
||
|
||
|
||
Scenario: remove_tool returns False when tool not found
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo delete method returns False
|
||
When I attempt to remove tool "local/ghost" via the coverage service
|
||
Then the removal result should be False
|
||
|
||
# --- remove_tool: ToolInUseError propagation -------------------------------
|
||
|
||
|
||
Scenario: remove_tool propagates ToolInUseError from repo.delete
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo delete method raises ToolInUseError
|
||
When I attempt to remove tool "local/in-use" via the coverage service
|
||
Then a coverage ToolInUseError should be raised
|
||
|
||
# --- attach_validation: validation not found -------------------------------
|
||
|
||
|
||
Scenario: attach_validation raises NotFoundError when validation does not exist
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo returns None for get_by_name
|
||
When I attempt to attach validation via the coverage service
|
||
Then a coverage NotFoundError should be raised with message containing "not found"
|
||
|
||
# --- attach_validation: successful delegation ------------------------------
|
||
|
||
|
||
Scenario: attach_validation delegates to attachment repo when validation exists
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo returns a sentinel for get_by_name
|
||
When I attempt to attach validation via the coverage service
|
||
Then no error should be raised by the coverage service
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_fallback_coverage.feature
|
||
# Feature: Tool Registry Service - fallback method paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool falls back to add when create does not exist
|
||
Given a tool registry service with a repo that only has add method
|
||
When I register a tool via the fallback service
|
||
Then the tool should be registered via the add method
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: register_tool falls back to create attribute when neither create nor add are callable
|
||
Given a tool registry service with a repo that has no create or add methods
|
||
When I register a tool via the fallback service and expect error
|
||
Then a fallback AttributeError should be raised
|
||
|
||
# --- update_tool with tool_config parameter (dict) ----------------------
|
||
|
||
|
||
Scenario: update_tool with string tool name and dict config merges them
|
||
Given a tool registry service with a standard mock repo
|
||
When I update tool "local/my-tool" with dict config via the fallback service
|
||
Then the updated tool should have name "local/my-tool"
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: update_tool with string tool name and Tool model config uses model
|
||
Given a tool registry service with a standard mock repo
|
||
When I update tool "local/my-tool" with Tool model config via the fallback service
|
||
Then the update should use the Tool model directly
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: update_tool with dict tool and dict config uses the dict tool
|
||
Given a tool registry service with a standard mock repo
|
||
When I update with dict tool and dict config via the fallback service
|
||
Then the updated tool should be the original dict
|
||
And no error should be raised by the fallback service
|
||
|
||
# --- remove_tool fallback: no delete, uses remove -----------------------
|
||
|
||
|
||
Scenario: remove_tool falls back to remove when delete does not exist
|
||
Given a tool registry service with a repo that only has remove method
|
||
When I remove tool "local/gone" via the fallback service
|
||
Then the tool should be removed via the remove method
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: remove_tool falls back to delete attribute when neither delete nor remove are callable
|
||
Given a tool registry service with a repo that has no delete or remove methods
|
||
When I remove tool "local/gone" via the fallback service and expect error
|
||
Then a fallback AttributeError should be raised
|
||
|
||
# --- list_tools delegates to list_all -----------------------------------
|
||
|
||
|
||
Scenario: list_tools delegates to repo list_all with filters
|
||
Given a tool registry service with a standard mock repo
|
||
When I list tools with namespace "local" and type "validation" via the fallback service
|
||
Then the list result should be returned from list_all
|
||
And no error should be raised by the fallback service
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_uncovered_branches.feature
|
||
# Feature: Tool Registry Service – uncovered branch paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool falls back to add when create is not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where create is not callable but add is callable
|
||
When I trs branch register a tool
|
||
Then the trs branch result should equal the registered tool
|
||
And the trs branch add method should have been called
|
||
|
||
# --- register_tool: create AND add both not callable (L54-55,57) -----------
|
||
|
||
|
||
Scenario: register_tool falls through to direct create call when both create and add are not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where neither create nor add is callable
|
||
When I trs branch register a tool expecting an error
|
||
Then a trs branch TypeError should be raised
|
||
|
||
# --- remove_tool: delete not callable, remove IS callable (L101-103) -------
|
||
|
||
|
||
Scenario: remove_tool falls back to remove when delete is not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where delete is not callable but remove is callable
|
||
When I trs branch remove tool "local/some-tool"
|
||
Then the trs branch removal result should be True
|
||
And the trs branch remove method should have been called
|
||
|
||
# --- remove_tool: delete AND remove both not callable (L101-102,105) -------
|
||
|
||
|
||
Scenario: remove_tool falls through to direct delete call when both delete and remove are not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where neither delete nor remove is callable
|
||
When I trs branch remove tool "local/some-tool" expecting an error
|
||
Then a trs branch TypeError should be raised
|
||
|
||
# --- update_tool: tool_config is a Tool instance (L77-78,82) ---------------
|
||
|
||
|
||
Scenario: update_tool uses Tool instance directly when tool_config is a Tool
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a Tool instance as tool_config
|
||
Then the trs branch updated result should be the Tool instance
|
||
|
||
# --- update_tool: tool_config is a dict and tool is a string (L80,82) ------
|
||
|
||
|
||
Scenario: update_tool merges name from string tool into dict tool_config
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a string name and dict tool_config
|
||
Then the trs branch updated result should contain the merged name and config
|
||
|
||
# --- update_tool: tool_config is a dict and tool is not a string (L80,82) --
|
||
|
||
|
||
Scenario: update_tool uses tool directly when tool_config is a dict but tool is not a string
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a dict tool and dict tool_config
|
||
Then the trs branch updated result should be the original tool dict
|
||
|
||
# --- list_tools: with filters (L123) ---------------------------------------
|
||
|
||
|
||
Scenario: list_tools passes filters through to repo list_all
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch list tools with namespace "local" type "tool" source "builtin"
|
||
Then the trs branch list result should come from the repo
|
||
|
||
# ============================================================
|
||
# Originally from: tool_runtime.feature
|
||
# Feature: Tool Runtime Core
|
||
# ============================================================
|
||
|
||
Scenario: Register a tool in the registry
|
||
Given a tool registry
|
||
And a tool spec named "test/echo" with a handler
|
||
When I register the tool spec
|
||
Then the tool should be in the registry
|
||
|
||
|
||
Scenario: Get a registered tool by name
|
||
Given a tool registry
|
||
And a registered tool spec named "test/echo"
|
||
When I get the tool by name "test/echo"
|
||
Then the retrieved tool spec name should be "test/echo"
|
||
|
||
|
||
Scenario: List all tools in the registry
|
||
Given a tool registry
|
||
And a registered tool spec named "test/alpha"
|
||
And a registered tool spec named "test/beta"
|
||
When I list all tools
|
||
Then the tool list should contain 2 tools
|
||
|
||
|
||
Scenario: List tools filtered by namespace
|
||
Given a tool registry
|
||
And a registered tool spec named "ns1/tool-a"
|
||
And a registered tool spec named "ns2/tool-b"
|
||
And a registered tool spec named "ns1/tool-c"
|
||
When I list tools with namespace "ns1"
|
||
Then the tool list should contain 2 tools
|
||
|
||
|
||
Scenario: Remove a tool from the registry
|
||
Given a tool registry
|
||
And a registered tool spec named "test/remove-me"
|
||
When I remove the tool "test/remove-me"
|
||
Then the removal should succeed
|
||
And getting "test/remove-me" should return None
|
||
|
||
# ---- Missing Tool Errors ----
|
||
|
||
|
||
Scenario: Get a missing tool returns None
|
||
Given a tool registry
|
||
When I get the tool by name "test/nonexistent"
|
||
Then the retrieved tool spec should be None
|
||
|
||
|
||
Scenario: Remove a missing tool returns False
|
||
Given a tool registry
|
||
When I remove the tool "test/nonexistent"
|
||
Then the removal should fail
|
||
|
||
# ---- Name Collision Detection ----
|
||
|
||
|
||
Scenario: Duplicate registration raises ToolError
|
||
Given a tool registry
|
||
And a registered tool spec named "test/duplicate"
|
||
When I try to register a duplicate tool "test/duplicate"
|
||
Then a ToolError should be raised with type "RegistrationError"
|
||
|
||
# ---- Capability Flag Validation ----
|
||
|
||
|
||
Scenario: Tool spec with read-only capability
|
||
Given a tool spec named "test/reader" with read_only capability
|
||
Then the tool spec capabilities read_only should be True
|
||
And the tool spec capabilities writes should be False
|
||
|
||
|
||
Scenario: Tool spec with write capability
|
||
Given a tool spec named "test/writer" with writes capability
|
||
Then the tool spec capabilities writes should be True
|
||
|
||
# ---- Tool Lifecycle Hook Ordering ----
|
||
|
||
|
||
Scenario: Full lifecycle discover activate execute deactivate
|
||
Given a tool registry
|
||
And a registered tool spec named "test/lifecycle"
|
||
And a tool runner for the registry
|
||
When I discover tools from the runner
|
||
And I activate tool "test/lifecycle"
|
||
And I execute tool "test/lifecycle" with inputs {"key": "value"}
|
||
And I deactivate tool "test/lifecycle"
|
||
Then the lifecycle should complete in order
|
||
|
||
# ---- Execute With Valid Inputs ----
|
||
|
||
|
||
Scenario: Execute with valid inputs returns success
|
||
Given a tool registry
|
||
And a registered tool spec named "test/add" with an adder handler
|
||
And a tool runner for the registry
|
||
When I activate tool "test/add"
|
||
And I execute tool "test/add" with inputs {"a": 1, "b": 2}
|
||
Then the tool result should be successful
|
||
And the tool result output should contain key "sum"
|
||
And the tool result duration_ms should be non-negative
|
||
|
||
# ---- Execute With Handler Failure ----
|
||
|
||
|
||
Scenario: Execute with handler failure returns failure result
|
||
Given a tool registry
|
||
And a registered tool spec named "test/fail" with a failing handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/fail" with inputs {}
|
||
Then the tool result should not be successful
|
||
And the tool result error should not be empty
|
||
|
||
# ---- Thread-Safe Concurrent Registration ----
|
||
|
||
|
||
Scenario: Concurrent registrations are thread-safe
|
||
Given a tool registry
|
||
When I concurrently register 50 tools
|
||
Then all 50 tools should be in the registry
|
||
|
||
# ---- JSON-Serialisable IO Validation ----
|
||
|
||
|
||
Scenario: Execute rejects non-JSON-serialisable inputs
|
||
Given a tool registry
|
||
And a registered tool spec named "test/json-check"
|
||
And a tool runner for the registry
|
||
When I execute tool "test/json-check" with non-serialisable inputs
|
||
Then the tool result should not be successful
|
||
And the tool result error should mention "JSON"
|
||
|
||
|
||
Scenario: Execute rejects non-JSON-serialisable outputs
|
||
Given a tool registry
|
||
And a registered tool spec named "test/bad-output" with a non-serialisable handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/bad-output" with inputs {}
|
||
Then the tool result should not be successful
|
||
And the tool result error should mention "JSON"
|
||
|
||
# ---- ToolError Exception ----
|
||
|
||
|
||
Scenario: ToolError carries structured context
|
||
When I create a ToolError with name "test/err" type "TestError" and details "something broke"
|
||
Then the ToolError tool_name should be "test/err"
|
||
And the ToolError error_type should be "TestError"
|
||
And the ToolError details should be "something broke"
|
||
And the ToolError message should contain "TestError"
|
||
|
||
# ---- ToolResult Model ----
|
||
|
||
|
||
Scenario: ToolResult with success and metadata
|
||
When I create a ToolResult with success True and metadata
|
||
Then the ToolResult success should be True
|
||
And the ToolResult metadata should not be empty
|
||
|
||
|
||
Scenario: ToolResult with failure
|
||
When I create a ToolResult with success False and error "handler crashed"
|
||
Then the ToolResult success should be False
|
||
And the ToolResult error should be "handler crashed"
|
||
|
||
# ---- Runner Activate Errors ----
|
||
|
||
|
||
Scenario: Activate missing tool raises ToolError
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I try to activate a missing tool "test/missing"
|
||
Then a ToolError should be raised with type "ActivationError"
|
||
|
||
# ---- Runner Execute Missing Tool ----
|
||
|
||
|
||
Scenario: Execute missing tool raises ToolError
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I try to execute a missing tool "test/missing" with inputs {}
|
||
Then a ToolError should be raised with type "ExecutionError"
|
||
|
||
# ---- Runner Deactivate ----
|
||
|
||
|
||
Scenario: Deactivate a non-active tool returns False
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I deactivate tool "test/nonexistent"
|
||
Then the deactivation should return False
|
||
|
||
# ---- Discover With Alternate Registry ----
|
||
|
||
|
||
Scenario: Discover from an alternate registry
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
And an alternate registry with 3 tools
|
||
When I discover tools from the alternate registry
|
||
Then the discovered list should contain 3 tools
|
||
|
||
# ---- Execute Without Prior Activate ----
|
||
|
||
|
||
Scenario: Execute without prior activate still works via registry lookup
|
||
Given a tool registry
|
||
And a registered tool spec named "test/no-activate" with an adder handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/no-activate" with inputs {"a": 5, "b": 3}
|
||
Then the tool result should be successful
|
||
|
||
# ---- Handler Returns Non-Dict ----
|
||
|
||
|
||
Scenario: Handler returning non-dict is wrapped in result dict
|
||
Given a tool registry
|
||
And a registered tool spec named "test/scalar" with a scalar handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/scalar" with inputs {}
|
||
Then the tool result should be successful
|
||
And the tool result output should contain key "result"
|
||
|
||
# ---- List Tools With tool_type Filter ----
|
||
|
||
|
||
Scenario: List tools with tool_type filter
|
||
Given a tool registry
|
||
And a registered tool spec named "test/filtered"
|
||
When I list tools with tool_type "tool"
|
||
Then the tool list should contain 1 tools
|
||
|