Jeff
20321f21ee
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / security (push) Waiting to run
CI / quality (push) Waiting to run
CI / unit_tests (push) Waiting to run
CI / integration_tests (push) Waiting to run
CI / coverage (push) Blocked by required conditions
CI / build (push) Waiting to run
CI / docker (push) Blocked by required conditions
278 lines
12 KiB
Gherkin
278 lines
12 KiB
Gherkin
Feature: Skill Domain Model and Resolution
|
|
As a developer
|
|
I want skill domain models and a resolver for the skill registry
|
|
So that skills can be registered, validated, resolved, and managed via CLI
|
|
|
|
# ---- Skill Creation ----
|
|
|
|
Scenario: Create a skill with valid fields
|
|
When I create a skill_model with name "local/code-tools" and description "Code editing tools"
|
|
Then the skill_model should be created
|
|
And the skill_model name should be "local/code-tools"
|
|
And the skill_model description should be "Code editing tools"
|
|
And the skill_model namespace should be "local"
|
|
And the skill_model short_name should be "code-tools"
|
|
|
|
Scenario: Create a skill with tool refs
|
|
When I create a skill_model with tool refs "local/edit-file,local/read-file"
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
|
|
Scenario: Create a skill with all fields populated
|
|
When I create a skill_model with all fields
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
And the skill_model should have 1 include
|
|
And the skill_model should have 1 anonymous tool
|
|
And the skill_model should have 1 mcp server
|
|
And the skill_model should have 1 agent skill
|
|
|
|
# ---- Skill Name Validation ----
|
|
|
|
Scenario: Skill name must match namespace/name pattern
|
|
When I try to create a skill_model with invalid name "no-namespace"
|
|
Then a skill_model validation error should be raised
|
|
And the skill_model error should mention "namespace/name"
|
|
|
|
Scenario: Skill name rejects spaces
|
|
When I try to create a skill_model with invalid name "local/bad name"
|
|
Then a skill_model validation error should be raised
|
|
|
|
Scenario: Skill name rejects double slashes
|
|
When I try to create a skill_model with invalid name "local//bad"
|
|
Then a skill_model validation error should be raised
|
|
|
|
Scenario: Skill name allows hyphens and underscores
|
|
When I create a skill_model with name "my-ns/my_skill" and description "Test"
|
|
Then the skill_model should be created
|
|
And the skill_model namespace should be "my-ns"
|
|
And the skill_model short_name should be "my_skill"
|
|
|
|
Scenario: Skill description must not be empty
|
|
When I try to create a skill_model with empty description
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- Include Resolution (flat) ----
|
|
|
|
Scenario: Resolve skill with flat includes
|
|
Given a skill_resolver registry with "local/basic-tools" containing refs "local/edit-file,local/read-file"
|
|
And a skill_resolver root skill "local/code-tools" including "local/basic-tools" with own refs "local/format"
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver result should contain tool "local/edit-file"
|
|
And the skill_resolver result should contain tool "local/read-file"
|
|
And the skill_resolver result should contain tool "local/format"
|
|
|
|
# ---- Include Resolution (nested) ----
|
|
|
|
Scenario: Resolve skill with nested includes
|
|
Given a skill_resolver registry with "local/base" containing refs "local/core-tool"
|
|
And a skill_resolver registry with "local/mid" including "local/base" and containing refs "local/mid-tool"
|
|
And a skill_resolver root skill "local/top" including "local/mid" with own refs "local/top-tool"
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver result should contain tool "local/core-tool"
|
|
And the skill_resolver result should contain tool "local/mid-tool"
|
|
And the skill_resolver result should contain tool "local/top-tool"
|
|
|
|
# ---- Cycle Detection ----
|
|
|
|
Scenario: Cycle detection with error message
|
|
Given a skill_resolver registry with "local/a" including "local/b"
|
|
And a skill_resolver registry with "local/b" including "local/a"
|
|
When I try to resolve skill_resolver "local/a"
|
|
Then a skill_resolver cycle error should be raised
|
|
And the skill_resolver error should mention "local/a"
|
|
And the skill_resolver error should mention "local/b"
|
|
|
|
Scenario: Self-referencing cycle detection
|
|
Given a skill_resolver registry with "local/self" including "local/self"
|
|
When I try to resolve skill_resolver "local/self"
|
|
Then a skill_resolver cycle error should be raised
|
|
|
|
# ---- De-duplication ----
|
|
|
|
Scenario: De-duplication behavior last wins for overrides
|
|
Given a skill_resolver registry with "local/a" containing refs "local/shared-tool"
|
|
And a skill_resolver registry with "local/b" containing refs "local/shared-tool"
|
|
And a skill_resolver root skill "local/top" including "local/a,local/b" with no own refs
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 1 tools
|
|
And the skill_resolver tool "local/shared-tool" should come from "local/b"
|
|
|
|
# ---- Override Application ----
|
|
|
|
Scenario: Override application on tool refs
|
|
When I create a skill_model with overrides for "local/edit-file"
|
|
And I resolve the overridden skill_model
|
|
Then the skill_resolver tool "local/edit-file" should have overrides
|
|
|
|
# ---- Empty Skill ----
|
|
|
|
Scenario: Empty skill with no tools or includes
|
|
When I create a skill_model with name "local/empty" and description "Empty skill"
|
|
And I resolve the empty skill_model
|
|
Then the skill_resolver result should have 0 tools
|
|
|
|
# ---- from_config Loading ----
|
|
|
|
Scenario: Load skill from config dict
|
|
When I load a skill_model from config with name "local/my-skill"
|
|
Then the skill_model should be created
|
|
And the skill_model name should be "local/my-skill"
|
|
|
|
Scenario: Load skill from config missing name raises error
|
|
When I try to load a skill_model from config missing "name"
|
|
Then a skill_model config error should be raised with "name"
|
|
|
|
Scenario: Load skill from config missing description raises error
|
|
When I try to load a skill_model from config missing "description"
|
|
Then a skill_model config error should be raised with "description"
|
|
|
|
Scenario: Load skill from config with full YAML structure
|
|
When I load a skill_model from full config
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
And the skill_model should have 1 include
|
|
And the skill_model should have 1 anonymous tool
|
|
And the skill_model should have 1 mcp server
|
|
And the skill_model should have 1 agent skill
|
|
|
|
Scenario: Load skill from config with string includes
|
|
When I load a skill_model from config with string includes
|
|
Then the skill_model should be created
|
|
And the skill_model should have 1 include
|
|
|
|
Scenario: Load skill from config with string agent skills
|
|
When I load a skill_model from config with string agent skills
|
|
Then the skill_model should be created
|
|
And the skill_model should have 1 agent skill
|
|
|
|
# ---- as_cli_dict Output ----
|
|
|
|
Scenario: Skill as_cli_dict has required keys
|
|
When I create a skill_model and call as_cli_dict
|
|
Then the skill_model cli dict should have key "name"
|
|
And the skill_model cli dict should have key "description"
|
|
And the skill_model cli dict should have key "namespace"
|
|
And the skill_model cli dict should have key "short_name"
|
|
|
|
Scenario: Skill as_cli_dict includes optional sections when set
|
|
When I create a skill_model with all fields and call as_cli_dict
|
|
Then the skill_model cli dict should have key "tool_refs"
|
|
And the skill_model cli dict should have key "includes"
|
|
And the skill_model cli dict should have key "anonymous_tools"
|
|
And the skill_model cli dict should have key "mcp_servers"
|
|
And the skill_model cli dict should have key "agent_skills"
|
|
And the skill_model cli dict should have key "overrides"
|
|
|
|
Scenario: Skill as_cli_dict omits empty sections
|
|
When I create a skill_model with name "local/minimal" and description "Minimal"
|
|
And I get the skill_model cli dict
|
|
Then the skill_model cli dict should not have key "tool_refs"
|
|
And the skill_model cli dict should not have key "includes"
|
|
And the skill_model cli dict should not have key "anonymous_tools"
|
|
And the skill_model cli dict should not have key "mcp_servers"
|
|
And the skill_model cli dict should not have key "agent_skills"
|
|
And the skill_model cli dict should not have key "overrides"
|
|
|
|
# ---- Inline Tool Validation ----
|
|
|
|
Scenario: Inline tool with valid custom source
|
|
When I create a skill_model inline tool with source "custom" and code "print('hi')"
|
|
Then the skill_model inline tool should be created
|
|
|
|
Scenario: Inline tool has default timeout of 300
|
|
When I create a skill_model inline tool with source "custom" and code "print('hi')"
|
|
Then the skill_model inline tool timeout should be 300
|
|
|
|
Scenario: Inline tool timeout must be >= 1
|
|
When I try to create a skill_model inline tool with timeout 0
|
|
Then a skill_model validation error should be raised
|
|
|
|
Scenario: Inline tool description must not be empty
|
|
When I try to create a skill_model inline tool with empty description
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- MCP Source Validation ----
|
|
|
|
Scenario: MCP source with specific tools
|
|
When I create a skill_model mcp source with server "npx @mcp/server" and tools "read,write"
|
|
Then the skill_model mcp source should be created
|
|
And the skill_model mcp source should have 2 tools
|
|
|
|
Scenario: MCP source with all tools (none specified)
|
|
When I create a skill_model mcp source with server "npx @mcp/server" and no tools
|
|
Then the skill_model mcp source should be created
|
|
And the skill_model mcp source tools should be none
|
|
|
|
Scenario: MCP source with env vars
|
|
When I create a skill_model mcp source with env vars
|
|
Then the skill_model mcp source should be created
|
|
|
|
Scenario: MCP source server must not be empty
|
|
When I try to create a skill_model mcp source with empty server
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- Agent Skill Source Validation ----
|
|
|
|
Scenario: Agent skill source with valid path
|
|
When I create a skill_model agent source with path "./skills/review"
|
|
Then the skill_model agent source should be created
|
|
|
|
Scenario: Agent skill source path must not be empty
|
|
When I try to create a skill_model agent source with empty path
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- CapabilitySummary Computation ----
|
|
|
|
Scenario: CapabilitySummary computation for skill with inline tools
|
|
When I compute skill_model capability summary for a skill with inline tools
|
|
Then the skill_model summary total tools should be 4
|
|
And the skill_model summary read_only_tools should be 1
|
|
And the skill_model summary write_tools should be 1
|
|
And the skill_model summary has_side_effects should be true
|
|
And the skill_model summary mcp_sources should be 1
|
|
And the skill_model summary agent_skill_sources should be 1
|
|
|
|
Scenario: CapabilitySummary for empty skill
|
|
When I compute skill_model capability summary for an empty skill
|
|
Then the skill_model summary total tools should be 0
|
|
|
|
# ---- SkillToolRef ----
|
|
|
|
Scenario: SkillToolRef creation
|
|
When I create a skill_model tool ref with name "local/my-tool"
|
|
Then the skill_model tool ref should be created
|
|
|
|
# ---- SkillInclude ----
|
|
|
|
Scenario: SkillInclude creation with overrides
|
|
When I create a skill_model include with name "local/base" and overrides
|
|
Then the skill_model include should be created
|
|
And the skill_model include should have overrides
|
|
|
|
Scenario: SkillInclude creation without overrides
|
|
When I create a skill_model include with name "local/base" without overrides
|
|
Then the skill_model include should be created
|
|
|
|
# ---- Missing included skill ----
|
|
|
|
Scenario: Resolve with missing included skill raises error
|
|
Given a skill_resolver root skill "local/broken" including "local/nonexistent" with no own refs
|
|
When I try to resolve skill_resolver "local/broken"
|
|
Then a skill_resolver missing skill error should be raised
|
|
And the skill_resolver error should mention "local/nonexistent"
|
|
|
|
# ---- MCP and agent skill resolution ----
|
|
|
|
Scenario: Resolve skill with MCP and agent skill sources
|
|
When I create and resolve a skill_model with mcp and agent sources
|
|
Then the skill_resolver result should have 3 tools
|
|
|
|
# ---- ResolvedToolEntry ----
|
|
|
|
Scenario: ResolvedToolEntry creation
|
|
When I create a skill_resolver resolved tool entry
|
|
Then the skill_resolver entry should be created
|