HAL9000
86525a4ba4
Add BDD test scenarios for server-qualified name acceptance in: - features/actor_schema.feature: Accept/reject scenarios for server:namespace/name format - features/skill_schema.feature: Accept scenarios for server:namespace/name format - features/consolidated_tool.feature: Accept scenarios for server:namespace/name format Also fix lint issues in the previous commit: - Remove trailing whitespace from actor/schema.py docstring - Split long NAMESPACED_NAME_RE line in skills/schema.py ISSUES CLOSED: #9074
1026 lines
41 KiB
Gherkin
1026 lines
41 KiB
Gherkin
Feature: Consolidated Tool
|
||
Combined scenarios from: tool_lifecycle_coverage_boost, tool_model, tool_registry_service_coverage, tool_registry_service_fallback_coverage, tool_registry_service_uncovered_branches, tool_runtime
|
||
|
||
# ============================================================
|
||
# Originally from: tool_lifecycle_coverage_boost.feature
|
||
# Feature: Tool lifecycle coverage boost
|
||
# ============================================================
|
||
|
||
Scenario: Removing the last tool from a plan deletes the plan entry
|
||
Given I create a coverage-boost lifecycle cache
|
||
And I have a coverage-boost mock tool instance named "builtin/only-tool"
|
||
When I put the coverage-boost instance in cache for plan "plan-rm-1" and tool "builtin/only-tool"
|
||
Then the coverage-boost cache should have 1 plans
|
||
When I remove tool "builtin/only-tool" from coverage-boost cache plan "plan-rm-1"
|
||
Then the coverage-boost cache should have 0 plans
|
||
And the coverage-boost cache internal dict should not contain plan "plan-rm-1"
|
||
|
||
|
||
Scenario: Removing a tool that does not exist in cache returns None
|
||
Given I create a coverage-boost lifecycle cache
|
||
When I remove tool "builtin/ghost" from coverage-boost cache plan "plan-rm-2"
|
||
Then the coverage-boost remove result should be None
|
||
|
||
# ── ToolLifecycleCache.get_plan_tools with empty cache ──────────────────
|
||
|
||
|
||
Scenario: Getting plan tools for a plan with no activated tools returns empty list
|
||
Given I create a coverage-boost lifecycle cache
|
||
Then the coverage-boost plan tools for plan "plan-empty" should be an empty list
|
||
|
||
# ── ToolRuntime.activate with ToolCancelledError ────────────────────────
|
||
|
||
|
||
Scenario: Activating a tool that raises ToolCancelledError propagates the error
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost cancelling-activate tool "builtin/cancel-act"
|
||
And I create a coverage-boost execution context with plan_id "plan-cancel-act"
|
||
When I try to activate the cancelling tool "builtin/cancel-act" in coverage-boost runtime
|
||
Then a ToolCancelledError should have been raised in coverage-boost activation
|
||
|
||
# ── ToolRuntime.execute with None cached instance ──────────────────────
|
||
|
||
|
||
Scenario: Executing a tool when cached instance is None raises ToolNotActivatedError
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost tool "builtin/null-cache" with a mock that nullifies cache
|
||
And I create a coverage-boost execution context with plan_id "plan-null-cache"
|
||
When I try to execute the null-cache tool "builtin/null-cache" in coverage-boost runtime
|
||
Then a ToolNotActivatedError should have been raised in coverage-boost execution
|
||
|
||
# ── ToolRuntime.execute with ToolCancelledError during execution ────────
|
||
|
||
|
||
Scenario: Executing a tool that raises ToolCancelledError records trace and re-raises
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost cancelling-execute tool "builtin/cancel-exec"
|
||
And I create a coverage-boost execution context with plan_id "plan-cancel-exec"
|
||
When I try to execute the cancelling-execute tool "builtin/cancel-exec" in coverage-boost runtime
|
||
Then a ToolCancelledError should have been raised in coverage-boost execution
|
||
And the coverage-boost context should have 1 traces
|
||
And the coverage-boost last trace should show success False
|
||
And the coverage-boost last trace error should be "Cancelled"
|
||
|
||
# ── ToolRuntime.deactivate with tool not in cache ──────────────────────
|
||
|
||
|
||
Scenario: Deactivating a tool that was never activated is a silent no-op
|
||
Given I create a coverage-boost tool runtime
|
||
And I register a coverage-boost read-only tool "builtin/never-active" with a mock
|
||
And I create a coverage-boost execution context with plan_id "plan-deact-none"
|
||
When I deactivate tool "builtin/never-active" in the coverage-boost runtime
|
||
Then no coverage-boost exception should have been raised
|
||
And the coverage-boost mock "builtin/never-active" should not have been deactivated
|
||
|
||
# ── ToolRuntime.deactivate_plan with empty plan ────────────────────────
|
||
|
||
|
||
Scenario: Deactivating a plan with no activated tools is a silent no-op
|
||
Given I create a coverage-boost tool runtime
|
||
And I create a coverage-boost execution context with plan_id "plan-deact-empty"
|
||
When I deactivate plan in the coverage-boost runtime
|
||
Then no coverage-boost exception should have been raised
|
||
|
||
# ── ToolLifecycleCache.remove with multiple tools leaves remaining ─────
|
||
|
||
|
||
Scenario: Removing one of two tools keeps the plan entry with the remaining tool
|
||
Given I create a coverage-boost lifecycle cache
|
||
And I have a coverage-boost mock tool instance named "builtin/tool-a"
|
||
And I have a second coverage-boost mock tool instance named "builtin/tool-b"
|
||
When I put the coverage-boost instance in cache for plan "plan-rm-multi" and tool "builtin/tool-a"
|
||
And I put the second coverage-boost instance in cache for plan "plan-rm-multi" and tool "builtin/tool-b"
|
||
Then the coverage-boost cache should have 1 plans
|
||
When I remove tool "builtin/tool-a" from coverage-boost cache plan "plan-rm-multi"
|
||
Then the coverage-boost cache should have 1 plans
|
||
And the coverage-boost cache get for plan "plan-rm-multi" and tool "builtin/tool-b" should not be None
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_model.feature
|
||
# Feature: Tool and Validation Domain Models
|
||
# ============================================================
|
||
|
||
Scenario: Create a custom tool with valid fields
|
||
When I create a tool with name "local/line-counter" source "custom" and code "print('hello')"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/line-counter"
|
||
And the tool model source should be "custom"
|
||
And the tool model type should be "tool"
|
||
And the tool model namespace should be "local"
|
||
And the tool model short name should be "line-counter"
|
||
|
||
|
||
Scenario: Create a builtin tool
|
||
When I create a tool with name "builtin/file-read" and source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model source should be "builtin"
|
||
|
||
|
||
Scenario: Create an MCP tool with required fields
|
||
When I create an MCP tool with name "devops/docker-build" server "devops-mcp" and tool name "docker_build"
|
||
Then the tool model should be created
|
||
And the tool model source should be "mcp"
|
||
|
||
|
||
Scenario: Create an agent_skill tool
|
||
When I create an agent skill tool with name "skills/formatter" and path "/skills/formatter"
|
||
Then the tool model should be created
|
||
And the tool model source should be "agent_skill"
|
||
|
||
# ---- Tool Name Validation ----
|
||
|
||
|
||
Scenario: Tool name must match namespace/name pattern
|
||
When I try to create a tool with invalid name "no-namespace"
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "namespace/name"
|
||
|
||
|
||
Scenario: Tool name rejects spaces
|
||
When I try to create a tool with invalid name "local/bad name"
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool name rejects double slashes
|
||
When I try to create a tool with invalid name "local//bad"
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool name allows hyphens and underscores
|
||
When I create a tool with name "my-ns/my_tool" and source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model namespace should be "my-ns"
|
||
And the tool model short name should be "my_tool"
|
||
|
||
|
||
Scenario: Tool name accepts server-qualified format
|
||
When I create a tool with name "dev:freemo/custom-analysis" and source "builtin"
|
||
Then the tool model should be created
|
||
|
||
|
||
Scenario: Tool name accepts server-qualified format with cleverthis server
|
||
When I create a tool with name "cleverthis:local/run-migrations" and source "builtin"
|
||
Then the tool model should be created
|
||
|
||
# ---- Source-conditional field requirements ----
|
||
|
||
|
||
Scenario: Custom source requires code
|
||
When I try to create a tool with name "local/test" source "custom" without code
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "code is required"
|
||
|
||
|
||
Scenario: MCP source requires mcp_server
|
||
When I try to create an MCP tool without mcp_server
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "mcp_server is required"
|
||
|
||
|
||
Scenario: MCP source requires mcp_tool_name
|
||
When I try to create an MCP tool without mcp_tool_name
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "mcp_tool_name is required"
|
||
|
||
|
||
Scenario: Agent skill source requires agent_skill_path
|
||
When I try to create an agent skill tool without path
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "agent_skill_path is required"
|
||
|
||
# ---- Capability constraint enforcement ----
|
||
|
||
|
||
Scenario: Read-only capability enforces writes=false
|
||
When I try to create a tool capability with read_only true and writes true
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "read_only tool cannot have writes"
|
||
|
||
|
||
Scenario: Read-only capability enforces checkpointable=false
|
||
When I try to create a tool capability with read_only true and checkpointable true
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "read_only tool cannot have checkpointable"
|
||
|
||
|
||
Scenario: Non-read-only capability allows writes
|
||
When I create a tool capability with writes true
|
||
Then the tool capability should have writes true
|
||
|
||
# ---- ResourceSlot binding validation ----
|
||
|
||
|
||
Scenario: Static binding requires static_resource
|
||
When I try to create a resource slot with static binding and no static resource
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "static_resource is required"
|
||
|
||
|
||
Scenario: Non-static binding forbids static_resource
|
||
When I try to create a resource slot with contextual binding and static resource set
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "static_resource must not be set"
|
||
|
||
|
||
Scenario: Static binding with static_resource succeeds
|
||
When I create a resource slot with static binding and static resource "my-repo"
|
||
Then the tool resource slot should be created
|
||
|
||
|
||
Scenario: ResourceSlot name must be valid identifier
|
||
When I try to create a resource slot with invalid name "123bad"
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "valid Python identifier"
|
||
|
||
# ---- Validation model creation ----
|
||
|
||
|
||
Scenario: Create a custom validation with forced read-only
|
||
When I create a validation with name "qa/coverage-check" source "custom" and code "return True"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
And the tool validation capability should not be checkpointable
|
||
|
||
|
||
Scenario: Validation forces tool_type to validation
|
||
When I create a validation with name "qa/test-check" source "builtin"
|
||
Then the tool model type should be "validation"
|
||
|
||
|
||
Scenario: Validation mode defaults to required
|
||
When I create a validation with name "qa/check" source "builtin"
|
||
Then the tool validation mode should be "required"
|
||
|
||
|
||
Scenario: Validation can be informational
|
||
When I create an informational validation with name "qa/info-check" source "builtin"
|
||
Then the tool validation mode should be "informational"
|
||
|
||
# ---- Validation wraps/transform requirements ----
|
||
|
||
|
||
Scenario: Wrapped validation requires transform
|
||
When I try to create a wrapped validation without transform
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "transform is required"
|
||
|
||
|
||
Scenario: Wrapped validation forbids code
|
||
When I try to create a wrapped validation with code
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "code must not be set"
|
||
|
||
|
||
Scenario: Wrapped validation requires source=wrapped
|
||
When I try to create a validation with wraps and wrong source
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "source must be 'wrapped'"
|
||
|
||
|
||
Scenario: argument_mapping only valid with wraps
|
||
When I try to create a validation with argument_mapping but no wraps
|
||
Then a tool model validation error should be raised
|
||
And the tool model error should mention "argument_mapping is only valid"
|
||
|
||
|
||
Scenario: Valid wrapped validation
|
||
When I create a valid wrapped validation
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool validation wraps should be "devops/run-linter"
|
||
|
||
# ---- from_config loading ----
|
||
|
||
|
||
Scenario: Load tool from config dict
|
||
When I load a tool from config with name "local/my-tool" source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/my-tool"
|
||
|
||
|
||
Scenario: Load tool from config missing name raises error
|
||
When I try to load a tool from config missing "name"
|
||
Then a tool model config error should be raised with "name"
|
||
|
||
|
||
Scenario: Load tool from config missing description raises error
|
||
When I try to load a tool from config missing "description"
|
||
Then a tool model config error should be raised with "description"
|
||
|
||
|
||
Scenario: Load tool from config missing source raises error
|
||
When I try to load a tool from config missing "source"
|
||
Then a tool model config error should be raised with "source"
|
||
|
||
|
||
Scenario: Load validation from config with wraps
|
||
When I load a validation from config with wraps "devops/linter"
|
||
Then the tool model should be created
|
||
And the tool model source should be "wrapped"
|
||
And the tool validation wraps should be "devops/linter"
|
||
|
||
|
||
Scenario: Load validation from config missing name raises error
|
||
When I try to load a validation config missing "name"
|
||
Then a tool model config error should be raised with "name"
|
||
|
||
# ---- as_cli_dict output stability ----
|
||
|
||
|
||
Scenario: Tool as_cli_dict has required keys
|
||
When I create a tool and call as_cli_dict
|
||
Then the tool cli dict should have key "name"
|
||
And the tool cli dict should have key "description"
|
||
And the tool cli dict should have key "source"
|
||
And the tool cli dict should have key "tool_type"
|
||
And the tool cli dict should have key "namespace"
|
||
And the tool cli dict should have key "short_name"
|
||
And the tool cli dict should have key "capability"
|
||
And the tool cli dict should have key "timeout"
|
||
|
||
|
||
Scenario: Validation as_cli_dict includes mode
|
||
When I create a validation and call as_cli_dict
|
||
Then the tool cli dict should have key "mode"
|
||
|
||
# ---- Tool timeout validation ----
|
||
|
||
|
||
Scenario: Tool timeout must be >= 1
|
||
When I try to create a tool with timeout 0
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Tool timeout default is 300
|
||
When I create a tool with name "local/default-timeout" and source "builtin"
|
||
Then the tool model timeout should be 300
|
||
|
||
# ---- ToolLifecycle ----
|
||
|
||
|
||
Scenario: ToolLifecycle accepts optional hooks
|
||
When I create a tool lifecycle with discover "my.discover"
|
||
Then the tool lifecycle discover should be "my.discover"
|
||
|
||
# ---- Enum values ----
|
||
|
||
|
||
Scenario: ToolSource enum has all expected values
|
||
Then the ToolSource enum should have values "mcp,agent_skill,builtin,custom,wrapped"
|
||
|
||
|
||
Scenario: ToolType enum has all expected values
|
||
Then the ToolType enum should have values "tool,validation"
|
||
|
||
|
||
Scenario: ValidationMode enum has all expected values
|
||
Then the ValidationMode enum should have values "required,informational"
|
||
|
||
|
||
Scenario: CheckpointScope enum has all expected values
|
||
Then the CheckpointScope enum should have values "file,transaction,snapshot,composite"
|
||
|
||
|
||
Scenario: BindingMode enum has all expected values
|
||
Then the BindingMode enum should have values "contextual,static,parameter"
|
||
|
||
|
||
Scenario: ResourceAccessMode enum has all expected values
|
||
Then the ResourceAccessMode enum should have values "read_only,read_write"
|
||
|
||
# ---- Tool with resource slots via from_config ----
|
||
|
||
|
||
Scenario: Load tool from config with resource slots
|
||
When I load a tool from config with resource slots
|
||
Then the tool model should be created
|
||
And the tool model should have 1 resource slot
|
||
|
||
# ---- Validation from_config missing description ----
|
||
|
||
|
||
Scenario: Load validation from config missing description raises error
|
||
When I try to load a validation config missing "description"
|
||
Then a tool model config error should be raised with "description"
|
||
|
||
# ---- Tool description validation ----
|
||
|
||
|
||
Scenario: Tool description must not be empty
|
||
When I try to create a tool with empty description
|
||
Then a tool model validation error should be raised
|
||
|
||
# ---- Invalid scenarios ----
|
||
|
||
|
||
Scenario: Tool with conflicting capability flags
|
||
When I try to create a tool capability with read_only true and writes true
|
||
Then a tool model validation error should be raised
|
||
|
||
|
||
Scenario: Validation wraps with custom source is rejected
|
||
When I try to create a validation with wraps and wrong source
|
||
Then a tool model validation error should be raised
|
||
|
||
# ---- as_cli_dict coverage for optional fields ----
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes code when set
|
||
When I create a custom tool and call as_cli_dict
|
||
Then the tool cli dict should have key "code"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes mcp fields when set
|
||
When I create an MCP tool and call as_cli_dict
|
||
Then the tool cli dict should have key "mcp_server"
|
||
And the tool cli dict should have key "mcp_tool_name"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes agent_skill_path when set
|
||
When I create an agent skill tool and call as_cli_dict
|
||
Then the tool cli dict should have key "agent_skill_path"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes schemas when set
|
||
When I create a tool with schemas and call as_cli_dict
|
||
Then the tool cli dict should have key "input_schema"
|
||
And the tool cli dict should have key "output_schema"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes side_effects and cost_profile
|
||
When I create a tool with side effects and cost profile and call as_cli_dict
|
||
Then the tool cli dict capability should have key "side_effects"
|
||
And the tool cli dict capability should have key "cost_profile"
|
||
|
||
|
||
Scenario: Tool as_cli_dict includes resource slots when set
|
||
When I create a tool with resource slots and call as_cli_dict
|
||
Then the tool cli dict should have key "resource_slots"
|
||
|
||
# ---- Validation as_cli_dict wraps fields ----
|
||
|
||
|
||
Scenario: Validation as_cli_dict includes wraps and transform
|
||
When I create a wrapped validation and call as_cli_dict
|
||
Then the tool cli dict should have key "wraps"
|
||
And the tool cli dict should have key "transform"
|
||
And the tool cli dict should have key "argument_mapping"
|
||
|
||
# ---- Validation from_config without wraps ----
|
||
|
||
|
||
Scenario: Load validation from config without wraps needs source
|
||
When I try to load a validation config without source or wraps
|
||
Then a tool model config error should be raised with "source"
|
||
|
||
|
||
Scenario: Load validation from config with explicit source
|
||
When I load a validation from config with source "builtin"
|
||
Then the tool model should be created
|
||
And the tool model source should be "builtin"
|
||
|
||
# ---- Validation forces read_only on writable capability ----
|
||
|
||
|
||
Scenario: Validation forces read_only on writable capability
|
||
When I create a validation with writes capability
|
||
Then the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
And the tool validation capability should not be checkpointable
|
||
|
||
# ---- YAML loader output alignment (mirrors Robot smoke suite) ----
|
||
|
||
|
||
Scenario: Load custom tool from example YAML config
|
||
When I load a tool from YAML file "examples/tools/custom-tool.yaml"
|
||
Then the tool model should be created
|
||
And the tool model name should be "local/line-counter"
|
||
And the tool model source should be "custom"
|
||
And the tool model should have 1 resource slot
|
||
And the tool model timeout should be 60
|
||
|
||
|
||
Scenario: Load MCP tool from example YAML config
|
||
When I load a tool from YAML file "examples/tools/mcp-tool.yaml"
|
||
Then the tool model should be created
|
||
And the tool model name should be "devops/docker-build"
|
||
And the tool model source should be "mcp"
|
||
And the tool model timeout should be 600
|
||
|
||
|
||
Scenario: Load wrapped validation from example YAML config
|
||
When I load a validation from YAML file "examples/validations/wrapped-validation.yaml"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool model source should be "wrapped"
|
||
And the tool validation wraps should be "devops/run-linter"
|
||
And the tool validation mode should be "required"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
|
||
|
||
Scenario: Load required validation from example YAML config
|
||
When I load a validation from YAML file "examples/validations/required-validation.yaml"
|
||
Then the tool model should be created
|
||
And the tool model type should be "validation"
|
||
And the tool model source should be "custom"
|
||
And the tool validation mode should be "required"
|
||
And the tool validation capability should be read only
|
||
And the tool validation capability should not have writes
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_coverage.feature
|
||
# Feature: Tool Registry Service – uncovered error paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool propagates DuplicateToolError from repo.create
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo create method raises DuplicateToolError
|
||
When I attempt to register the tool via the coverage service
|
||
Then a coverage DuplicateToolError should be raised
|
||
|
||
# --- register_tool: DatabaseError propagation ------------------------------
|
||
|
||
|
||
Scenario: register_tool propagates DatabaseError from repo.create
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo create method raises DatabaseError
|
||
When I attempt to register the tool via the coverage service
|
||
Then a coverage DatabaseError should be raised with message containing "Failed to create"
|
||
|
||
# --- update_tool: ToolNotFoundError propagation ----------------------------
|
||
|
||
|
||
Scenario: update_tool propagates ToolNotFoundError from repo.update
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo update method raises ToolNotFoundError
|
||
When I attempt to update a tool via the coverage service
|
||
Then a coverage ToolNotFoundError should be raised
|
||
|
||
# --- update_tool: DatabaseError propagation --------------------------------
|
||
|
||
|
||
Scenario: update_tool propagates DatabaseError from repo.update
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo update method raises DatabaseError
|
||
When I attempt to update a tool via the coverage service
|
||
Then a coverage DatabaseError should be raised with message containing "Failed to update"
|
||
|
||
# --- remove_tool: returns False for missing tool ---------------------------
|
||
|
||
|
||
Scenario: remove_tool returns False when tool not found
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo delete method returns False
|
||
When I attempt to remove tool "local/ghost" via the coverage service
|
||
Then the removal result should be False
|
||
|
||
# --- remove_tool: ToolInUseError propagation -------------------------------
|
||
|
||
|
||
Scenario: remove_tool propagates ToolInUseError from repo.delete
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo delete method raises ToolInUseError
|
||
When I attempt to remove tool "local/in-use" via the coverage service
|
||
Then a coverage ToolInUseError should be raised
|
||
|
||
# --- attach_validation: validation not found -------------------------------
|
||
|
||
|
||
Scenario: attach_validation raises NotFoundError when validation does not exist
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo returns None for get_by_name
|
||
When I attempt to attach validation via the coverage service
|
||
Then a coverage NotFoundError should be raised with message containing "not found"
|
||
|
||
# --- attach_validation: successful delegation ------------------------------
|
||
|
||
|
||
Scenario: attach_validation delegates to attachment repo when validation exists
|
||
Given a mock-based tool registry service
|
||
Given the mock tool repo returns a sentinel for get_by_name
|
||
When I attempt to attach validation via the coverage service
|
||
Then no error should be raised by the coverage service
|
||
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_fallback_coverage.feature
|
||
# Feature: Tool Registry Service - fallback method paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool falls back to add when create does not exist
|
||
Given a tool registry service with a repo that only has add method
|
||
When I register a tool via the fallback service
|
||
Then the tool should be registered via the add method
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: register_tool falls back to create attribute when neither create nor add are callable
|
||
Given a tool registry service with a repo that has no create or add methods
|
||
When I register a tool via the fallback service and expect error
|
||
Then a fallback AttributeError should be raised
|
||
|
||
# --- update_tool with tool_config parameter (dict) ----------------------
|
||
|
||
|
||
Scenario: update_tool with string tool name and dict config merges them
|
||
Given a tool registry service with a standard mock repo
|
||
When I update tool "local/my-tool" with dict config via the fallback service
|
||
Then the updated tool should have name "local/my-tool"
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: update_tool with string tool name and Tool model config uses model
|
||
Given a tool registry service with a standard mock repo
|
||
When I update tool "local/my-tool" with Tool model config via the fallback service
|
||
Then the update should use the Tool model directly
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: update_tool with dict tool and dict config uses the dict tool
|
||
Given a tool registry service with a standard mock repo
|
||
When I update with dict tool and dict config via the fallback service
|
||
Then the updated tool should be the original dict
|
||
And no error should be raised by the fallback service
|
||
|
||
# --- remove_tool fallback: no delete, uses remove -----------------------
|
||
|
||
|
||
Scenario: remove_tool falls back to remove when delete does not exist
|
||
Given a tool registry service with a repo that only has remove method
|
||
When I remove tool "local/gone" via the fallback service
|
||
Then the tool should be removed via the remove method
|
||
And no error should be raised by the fallback service
|
||
|
||
|
||
Scenario: remove_tool falls back to delete attribute when neither delete nor remove are callable
|
||
Given a tool registry service with a repo that has no delete or remove methods
|
||
When I remove tool "local/gone" via the fallback service and expect error
|
||
Then a fallback AttributeError should be raised
|
||
|
||
# --- list_tools delegates to list_all -----------------------------------
|
||
|
||
|
||
Scenario: list_tools delegates to repo list_all with filters
|
||
Given a tool registry service with a standard mock repo
|
||
When I list tools with namespace "local" and type "validation" via the fallback service
|
||
Then the list result should be returned from list_all
|
||
And no error should be raised by the fallback service
|
||
|
||
# ============================================================
|
||
# Originally from: tool_registry_service_uncovered_branches.feature
|
||
# Feature: Tool Registry Service – uncovered branch paths
|
||
# ============================================================
|
||
|
||
Scenario: register_tool falls back to add when create is not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where create is not callable but add is callable
|
||
When I trs branch register a tool
|
||
Then the trs branch result should equal the registered tool
|
||
And the trs branch add method should have been called
|
||
|
||
# --- register_tool: create AND add both not callable (L54-55,57) -----------
|
||
|
||
|
||
Scenario: register_tool falls through to direct create call when both create and add are not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where neither create nor add is callable
|
||
When I trs branch register a tool expecting an error
|
||
Then a trs branch TypeError should be raised
|
||
|
||
# --- remove_tool: delete not callable, remove IS callable (L101-103) -------
|
||
|
||
|
||
Scenario: remove_tool falls back to remove when delete is not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where delete is not callable but remove is callable
|
||
When I trs branch remove tool "local/some-tool"
|
||
Then the trs branch removal result should be True
|
||
And the trs branch remove method should have been called
|
||
|
||
# --- remove_tool: delete AND remove both not callable (L101-102,105) -------
|
||
|
||
|
||
Scenario: remove_tool falls through to direct delete call when both delete and remove are not callable
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch tool repo where neither delete nor remove is callable
|
||
When I trs branch remove tool "local/some-tool" expecting an error
|
||
Then a trs branch TypeError should be raised
|
||
|
||
# --- update_tool: tool_config is a Tool instance (L77-78,82) ---------------
|
||
|
||
|
||
Scenario: update_tool uses Tool instance directly when tool_config is a Tool
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a Tool instance as tool_config
|
||
Then the trs branch updated result should be the Tool instance
|
||
|
||
# --- update_tool: tool_config is a dict and tool is a string (L80,82) ------
|
||
|
||
|
||
Scenario: update_tool merges name from string tool into dict tool_config
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a string name and dict tool_config
|
||
Then the trs branch updated result should contain the merged name and config
|
||
|
||
# --- update_tool: tool_config is a dict and tool is not a string (L80,82) --
|
||
|
||
|
||
Scenario: update_tool uses tool directly when tool_config is a dict but tool is not a string
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch update tool with a dict tool and dict tool_config
|
||
Then the trs branch updated result should be the original tool dict
|
||
|
||
# --- list_tools: with filters (L123) ---------------------------------------
|
||
|
||
|
||
Scenario: list_tools passes filters through to repo list_all
|
||
Given a trs branch mock-based tool registry service
|
||
Given a trs branch standard tool repo
|
||
When I trs branch list tools with namespace "local" type "tool" source "builtin"
|
||
Then the trs branch list result should come from the repo
|
||
|
||
# ============================================================
|
||
# Originally from: tool_runtime.feature
|
||
# Feature: Tool Runtime Core
|
||
# ============================================================
|
||
|
||
Scenario: Register a tool in the registry
|
||
Given a tool registry
|
||
And a tool spec named "test/echo" with a handler
|
||
When I register the tool spec
|
||
Then the tool should be in the registry
|
||
|
||
|
||
Scenario: Get a registered tool by name
|
||
Given a tool registry
|
||
And a registered tool spec named "test/echo"
|
||
When I get the tool by name "test/echo"
|
||
Then the retrieved tool spec name should be "test/echo"
|
||
|
||
|
||
Scenario: List all tools in the registry
|
||
Given a tool registry
|
||
And a registered tool spec named "test/alpha"
|
||
And a registered tool spec named "test/beta"
|
||
When I list all tools
|
||
Then the tool list should contain 2 tools
|
||
|
||
|
||
Scenario: List tools filtered by namespace
|
||
Given a tool registry
|
||
And a registered tool spec named "ns1/tool-a"
|
||
And a registered tool spec named "ns2/tool-b"
|
||
And a registered tool spec named "ns1/tool-c"
|
||
When I list tools with namespace "ns1"
|
||
Then the tool list should contain 2 tools
|
||
|
||
|
||
Scenario: Remove a tool from the registry
|
||
Given a tool registry
|
||
And a registered tool spec named "test/remove-me"
|
||
When I remove the tool "test/remove-me"
|
||
Then the removal should succeed
|
||
And getting "test/remove-me" should return None
|
||
|
||
# ---- Missing Tool Errors ----
|
||
|
||
|
||
Scenario: Get a missing tool returns None
|
||
Given a tool registry
|
||
When I get the tool by name "test/nonexistent"
|
||
Then the retrieved tool spec should be None
|
||
|
||
|
||
Scenario: Remove a missing tool returns False
|
||
Given a tool registry
|
||
When I remove the tool "test/nonexistent"
|
||
Then the removal should fail
|
||
|
||
# ---- Name Collision Detection ----
|
||
|
||
|
||
Scenario: Duplicate registration raises ToolError
|
||
Given a tool registry
|
||
And a registered tool spec named "test/duplicate"
|
||
When I try to register a duplicate tool "test/duplicate"
|
||
Then a ToolError should be raised with type "RegistrationError"
|
||
|
||
# ---- Capability Flag Validation ----
|
||
|
||
|
||
Scenario: Tool spec with read-only capability
|
||
Given a tool spec named "test/reader" with read_only capability
|
||
Then the tool spec capabilities read_only should be True
|
||
And the tool spec capabilities writes should be False
|
||
|
||
|
||
Scenario: Tool spec with write capability
|
||
Given a tool spec named "test/writer" with writes capability
|
||
Then the tool spec capabilities writes should be True
|
||
|
||
# ---- Tool Lifecycle Hook Ordering ----
|
||
|
||
|
||
Scenario: Full lifecycle discover activate execute deactivate
|
||
Given a tool registry
|
||
And a registered tool spec named "test/lifecycle"
|
||
And a tool runner for the registry
|
||
When I discover tools from the runner
|
||
And I activate tool "test/lifecycle"
|
||
And I execute tool "test/lifecycle" with inputs {"key": "value"}
|
||
And I deactivate tool "test/lifecycle"
|
||
Then the lifecycle should complete in order
|
||
|
||
# ---- Execute With Valid Inputs ----
|
||
|
||
|
||
Scenario: Execute with valid inputs returns success
|
||
Given a tool registry
|
||
And a registered tool spec named "test/add" with an adder handler
|
||
And a tool runner for the registry
|
||
When I activate tool "test/add"
|
||
And I execute tool "test/add" with inputs {"a": 1, "b": 2}
|
||
Then the tool result should be successful
|
||
And the tool result output should contain key "sum"
|
||
And the tool result duration_ms should be non-negative
|
||
|
||
# ---- Execute With Handler Failure ----
|
||
|
||
|
||
Scenario: Execute with handler failure returns failure result
|
||
Given a tool registry
|
||
And a registered tool spec named "test/fail" with a failing handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/fail" with inputs {}
|
||
Then the tool result should not be successful
|
||
And the tool result error should not be empty
|
||
|
||
# ---- Thread-Safe Concurrent Registration ----
|
||
|
||
|
||
Scenario: Concurrent registrations are thread-safe
|
||
Given a tool registry
|
||
When I concurrently register 50 tools
|
||
Then all 50 tools should be in the registry
|
||
|
||
# ---- JSON-Serialisable IO Validation ----
|
||
|
||
|
||
Scenario: Execute rejects non-JSON-serialisable inputs
|
||
Given a tool registry
|
||
And a registered tool spec named "test/json-check"
|
||
And a tool runner for the registry
|
||
When I execute tool "test/json-check" with non-serialisable inputs
|
||
Then the tool result should not be successful
|
||
And the tool result error should mention "JSON"
|
||
|
||
|
||
Scenario: Execute rejects non-JSON-serialisable outputs
|
||
Given a tool registry
|
||
And a registered tool spec named "test/bad-output" with a non-serialisable handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/bad-output" with inputs {}
|
||
Then the tool result should not be successful
|
||
And the tool result error should mention "JSON"
|
||
|
||
# ---- ToolError Exception ----
|
||
|
||
|
||
Scenario: ToolError carries structured context
|
||
When I create a ToolError with name "test/err" type "TestError" and details "something broke"
|
||
Then the ToolError tool_name should be "test/err"
|
||
And the ToolError error_type should be "TestError"
|
||
And the ToolError details should be "something broke"
|
||
And the ToolError message should contain "TestError"
|
||
|
||
# ---- ToolResult Model ----
|
||
|
||
|
||
Scenario: ToolResult with success and metadata
|
||
When I create a ToolResult with success True and metadata
|
||
Then the ToolResult success should be True
|
||
And the ToolResult metadata should not be empty
|
||
|
||
|
||
Scenario: ToolResult with failure
|
||
When I create a ToolResult with success False and error "handler crashed"
|
||
Then the ToolResult success should be False
|
||
And the ToolResult error should be "handler crashed"
|
||
|
||
# ---- Runner Activate Errors ----
|
||
|
||
|
||
Scenario: Activate missing tool raises ToolError
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I try to activate a missing tool "test/missing"
|
||
Then a ToolError should be raised with type "ActivationError"
|
||
|
||
# ---- Runner Execute Missing Tool ----
|
||
|
||
|
||
Scenario: Execute missing tool raises ToolError
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I try to execute a missing tool "test/missing" with inputs {}
|
||
Then a ToolError should be raised with type "ExecutionError"
|
||
|
||
# ---- Runner Deactivate ----
|
||
|
||
|
||
Scenario: Deactivate a non-active tool returns False
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
When I deactivate tool "test/nonexistent"
|
||
Then the deactivation should return False
|
||
|
||
# ---- Discover With Alternate Registry ----
|
||
|
||
|
||
Scenario: Discover from an alternate registry
|
||
Given a tool registry
|
||
And a tool runner for the registry
|
||
And an alternate registry with 3 tools
|
||
When I discover tools from the alternate registry
|
||
Then the discovered list should contain 3 tools
|
||
|
||
# ---- Execute Without Prior Activate ----
|
||
|
||
|
||
Scenario: Execute without prior activate still works via registry lookup
|
||
Given a tool registry
|
||
And a registered tool spec named "test/no-activate" with an adder handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/no-activate" with inputs {"a": 5, "b": 3}
|
||
Then the tool result should be successful
|
||
|
||
# ---- Handler Returns Non-Dict ----
|
||
|
||
|
||
Scenario: Handler returning non-dict is wrapped in result dict
|
||
Given a tool registry
|
||
And a registered tool spec named "test/scalar" with a scalar handler
|
||
And a tool runner for the registry
|
||
When I execute tool "test/scalar" with inputs {}
|
||
Then the tool result should be successful
|
||
And the tool result output should contain key "result"
|
||
|
||
# ---- List Tools With tool_type Filter ----
|
||
|
||
|
||
Scenario: List tools with tool_type filter returns matching tools
|
||
Given a tool registry
|
||
And a registered tool spec named "test/filtered" typed as "tool"
|
||
When I list tools with tool_type "tool"
|
||
Then the tool list should contain 1 tools
|
||
|
||
Scenario: List tools with tool_type validation returns only validations
|
||
Given a tool registry
|
||
And a registered tool spec named "test/my-tool" typed as "tool"
|
||
And a registered tool spec named "test/my-validation" typed as "validation"
|
||
When I list tools with tool_type "validation"
|
||
Then the tool list should contain 1 tools
|
||
|
||
Scenario: List tools with tool_type tool excludes validations
|
||
Given a tool registry
|
||
And a registered tool spec named "test/plain-tool" typed as "tool"
|
||
And a registered tool spec named "test/plain-validation" typed as "validation"
|
||
When I list tools with tool_type "tool"
|
||
Then the tool list should contain 1 tools
|
||
|
||
Scenario: List tools without tool_type filter returns all tools
|
||
Given a tool registry
|
||
And a registered tool spec named "test/all-tool" typed as "tool"
|
||
And a registered tool spec named "test/all-validation" typed as "validation"
|
||
When I list all tools
|
||
Then the tool list should contain 2 tools
|
||
|
||
Scenario: List tools with tool_type None returns all tools
|
||
Given a tool registry
|
||
And a registered tool spec named "test/none-tool" typed as "tool"
|
||
And a registered tool spec named "test/none-validation" typed as "validation"
|
||
When I list all tools
|
||
Then the tool list should contain 2 tools
|
||
|
||
Scenario: ToolSpec default tool_type is tool
|
||
Given a tool registry
|
||
And a registered tool spec named "test/default-type"
|
||
When I list tools with tool_type "tool"
|
||
Then the tool list should contain 1 tools
|
||
|
||
Scenario: ToolSpec with validation tool_type is excluded from tool filter
|
||
Given a tool registry
|
||
And a registered tool spec named "test/excl-validation" typed as "validation"
|
||
When I list tools with tool_type "tool"
|
||
Then the tool list should contain 0 tools
|
||
|
||
|