202 lines
9.7 KiB
Gherkin
202 lines
9.7 KiB
Gherkin
Feature: Skill Protocol Types
|
|
As a developer
|
|
I want skill protocol types for metadata, definition, result, and error
|
|
So that skills can be discovered, composed, executed, and errors reported uniformly
|
|
|
|
# ---- SkillErrorType Enum ----
|
|
|
|
Scenario: SkillErrorType has all expected members
|
|
When I list all skill_protocol error types
|
|
Then the skill_protocol error types should include "skill_not_found"
|
|
And the skill_protocol error types should include "resolution_failure"
|
|
And the skill_protocol error types should include "cycle_detected"
|
|
And the skill_protocol error types should include "tool_activation_failure"
|
|
And the skill_protocol error types should include "tool_execution_failure"
|
|
And the skill_protocol error types should include "validation_error"
|
|
And the skill_protocol error types should include "permission_denied"
|
|
|
|
Scenario: SkillErrorType has exactly 7 members
|
|
When I list all skill_protocol error types
|
|
Then there should be 7 skill_protocol error types
|
|
|
|
# ---- SkillError Creation ----
|
|
|
|
Scenario: Create a SkillError with required fields
|
|
When I create a skill_protocol error with type "tool_execution_failure" and message "Tool timed out" for skill "local/my-skill"
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error type should be "tool_execution_failure"
|
|
And the skill_protocol error message should be "Tool timed out"
|
|
And the skill_protocol error skill_name should be "local/my-skill"
|
|
And the skill_protocol error tool_name should be none
|
|
|
|
Scenario: Create a SkillError with optional tool_name
|
|
When I create a skill_protocol error with tool_name "local/my-tool"
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error tool_name should be "local/my-tool"
|
|
|
|
Scenario: Create a SkillError with details
|
|
When I create a skill_protocol error with details
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error details should have key "trace_id"
|
|
|
|
Scenario: SkillError is frozen (immutable)
|
|
When I create a skill_protocol error with type "validation_error" and message "Bad input" for skill "local/test"
|
|
Then the skill_protocol error should be immutable
|
|
|
|
# ---- SkillMetadata Creation ----
|
|
|
|
Scenario: Create SkillMetadata from Skill domain model
|
|
When I create a skill_protocol metadata from a basic skill
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata name should be "local/test-skill"
|
|
And the skill_protocol metadata description should be "A test skill"
|
|
And the skill_protocol metadata version should be "0.0.0"
|
|
And the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
|
|
Scenario: Create SkillMetadata from Skill with tool refs
|
|
When I create a skill_protocol metadata from a skill with tool refs
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata tool_count should be 2
|
|
And the skill_protocol metadata source_types should contain "tool_ref"
|
|
|
|
Scenario: Create SkillMetadata from Skill with inline write tools
|
|
When I create a skill_protocol metadata from a skill with write tools
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata writes should be true
|
|
And the skill_protocol metadata read_only should be false
|
|
|
|
Scenario: Create SkillMetadata from Skill with MCP sources
|
|
When I create a skill_protocol metadata from a skill with mcp sources
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata source_types should contain "mcp"
|
|
|
|
Scenario: Create SkillMetadata from Skill with agent skill sources
|
|
When I create a skill_protocol metadata from a skill with agent skill sources
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata source_types should contain "agent_skill"
|
|
|
|
Scenario: Create SkillMetadata with custom version
|
|
When I create a skill_protocol metadata with version "1.2.3"
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata version should be "1.2.3"
|
|
|
|
Scenario: SkillMetadata is frozen (immutable)
|
|
When I create a skill_protocol metadata from a basic skill
|
|
Then the skill_protocol metadata should be immutable
|
|
|
|
Scenario: SkillMetadata from_skill rejects None
|
|
When I try to create skill_protocol metadata from None
|
|
Then a skill_protocol value error should be raised
|
|
|
|
# ---- SkillDefinition ----
|
|
|
|
Scenario: Create a SkillDefinition with resolved tools
|
|
When I create a skill_protocol definition with resolved tools
|
|
Then the skill_protocol definition should be created
|
|
And the skill_protocol definition skill name should be "local/def-skill"
|
|
And the skill_protocol definition should have 1 resolved tools
|
|
|
|
Scenario: Create a SkillDefinition with input and output schemas
|
|
When I create a skill_protocol definition with input and output schemas
|
|
Then the skill_protocol definition should be created
|
|
And the skill_protocol definition input_schema should have key "type"
|
|
And the skill_protocol definition output_schema should have key "type"
|
|
|
|
Scenario: SkillDefinition rejects input_schema without type key
|
|
When I try to create a skill_protocol definition with invalid input_schema
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
Scenario: SkillDefinition rejects output_schema without type key
|
|
When I try to create a skill_protocol definition with invalid output_schema
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
Scenario: SkillDefinition detects writes inconsistency
|
|
When I try to create a skill_protocol definition with writes inconsistency
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
Scenario: SkillDefinition is frozen (immutable)
|
|
When I create a skill_protocol definition with resolved tools
|
|
Then the skill_protocol definition should be immutable
|
|
|
|
# ---- SkillResult ----
|
|
|
|
Scenario: Create a successful SkillResult
|
|
When I create a skill_protocol result with success
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result success should be true
|
|
And the skill_protocol result skill_name should be "local/my-skill"
|
|
And the skill_protocol result tool_name should be "local/my-tool"
|
|
And the skill_protocol result error should be none
|
|
And the skill_protocol result output_data should be "hello"
|
|
|
|
Scenario: Create a failed SkillResult with error
|
|
When I create a skill_protocol result with failure
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result success should be false
|
|
And the skill_protocol result error should not be none
|
|
And the skill_protocol result error type should be "tool_execution_failure"
|
|
|
|
Scenario: Create a SkillResult with duration and resource changes
|
|
When I create a skill_protocol result with duration and resource changes
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result duration_ms should be 150.5
|
|
And the skill_protocol result should have 1 resource changes
|
|
|
|
Scenario: SkillResult is frozen (immutable)
|
|
When I create a skill_protocol result with success
|
|
Then the skill_protocol result should be immutable
|
|
|
|
# ---- Error Mapping ----
|
|
|
|
Scenario: Map ValueError with cycle message to CYCLE_DETECTED
|
|
When I map a skill_protocol ValueError with message "Cycle detected in skill includes"
|
|
Then the skill_protocol mapped error type should be "cycle_detected"
|
|
|
|
Scenario: Map ValueError with not found message to SKILL_NOT_FOUND
|
|
When I map a skill_protocol ValueError with message "Skill not found"
|
|
Then the skill_protocol mapped error type should be "skill_not_found"
|
|
|
|
Scenario: Map generic ValueError to RESOLUTION_FAILURE
|
|
When I map a skill_protocol ValueError with message "Something went wrong"
|
|
Then the skill_protocol mapped error type should be "resolution_failure"
|
|
|
|
Scenario: Map PermissionError to PERMISSION_DENIED
|
|
When I map a skill_protocol PermissionError
|
|
Then the skill_protocol mapped error type should be "permission_denied"
|
|
|
|
Scenario: Map RuntimeError to TOOL_EXECUTION_FAILURE
|
|
When I map a skill_protocol RuntimeError with message "Unexpected failure"
|
|
Then the skill_protocol mapped error type should be "tool_execution_failure"
|
|
|
|
Scenario: Map RuntimeError with activation in message
|
|
When I map a skill_protocol RuntimeError with message "activation failed"
|
|
Then the skill_protocol mapped error type should be "tool_activation_failure"
|
|
|
|
Scenario: Error mapping with tool_name populated
|
|
When I map a skill_protocol error with tool_name "local/my-tool"
|
|
Then the skill_protocol mapped error tool_name should be "local/my-tool"
|
|
And the skill_protocol mapped error details should have key "tool_name"
|
|
|
|
Scenario: Error mapping rejects empty skill_name
|
|
When I try to map a skill_protocol error with empty skill_name
|
|
Then a skill_protocol value error should be raised
|
|
|
|
# ---- Writes/Read-Only Metadata Propagation ----
|
|
|
|
Scenario: Skill with only read-only tools has read_only=True
|
|
When I create a skill_protocol metadata from a skill with read_only tools
|
|
Then the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
|
|
Scenario: Skill with write tools has writes=True
|
|
When I create a skill_protocol metadata from a skill with write tools
|
|
Then the skill_protocol metadata writes should be true
|
|
And the skill_protocol metadata read_only should be false
|
|
|
|
Scenario: Empty skill has read_only=True
|
|
When I create a skill_protocol metadata from an empty skill
|
|
Then the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
And the skill_protocol metadata tool_count should be 0
|