hamza.khyari
bdd1ea4f3a
CI / push-validation (pull_request) Successful in 10s
CI / build (pull_request) Successful in 16s
CI / helm (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 30s
CI / lint (pull_request) Successful in 35s
CI / quality (pull_request) Successful in 35s
CI / security (pull_request) Successful in 50s
CI / integration_tests (pull_request) Successful in 4m3s
CI / e2e_tests (pull_request) Successful in 4m14s
CI / unit_tests (pull_request) Successful in 5m13s
CI / docker (pull_request) Successful in 8s
CI / coverage (pull_request) Successful in 11m2s
CI / status-check (pull_request) Successful in 1s
CheckpointManager was never wired into PlanExecutor — the CLI factory constructed PlanExecutor without a checkpoint_manager (defaulted to None), silently skipping all checkpoint hooks. Fix: - Register CheckpointManager as Singleton in DI container - Resolve container singleton in _get_plan_executor() and pass to PlanExecutor constructor - Bridge infra→domain: _try_create_checkpoint() now persists last_checkpoint_id on the plan via _commit_plan(), raises PlanError if persistence fails - Default checkpointable=True for writable+sandboxable resources and write-capable tools (model_validators on ResourceCapabilities and ToolCapability) - Validate that non-writable/non-sandboxable resources cannot be checkpointable (ValueError guard) - Add post-execute A2A facade notification using plan.status to avoid duplicate execute→execute transition errors Tests: - 10 Behave scenarios covering DI wiring, singleton identity, checkpoint creation, plan metadata update, rollback, graceful fallback, no-arg constructor, capability defaults (positive + 2 negative) - Updated consolidated_resource, consolidated_skill, and Robot helper_skill_flatten for new checkpointable defaults ISSUES CLOSED: #1253
1238 lines
51 KiB
Gherkin
1238 lines
51 KiB
Gherkin
Feature: Consolidated Skill
|
|
Combined scenarios from: skill_context, skill_discovery, skill_flatten, skill_inline_coverage, skill_protocol, skill_resolution
|
|
|
|
# ============================================================
|
|
# Originally from: skill_context.feature
|
|
# Feature: Skill Context and Registry
|
|
# ============================================================
|
|
|
|
Scenario: Create SkillContext with plan/project/sandbox info
|
|
When I create a skill_context with plan "plan-001" project "proj-alpha" and sandbox "/tmp/sandbox"
|
|
Then the skill_context should be created
|
|
And the skill_context plan_id should be "plan-001"
|
|
And the skill_context project_id should be "proj-alpha"
|
|
And the skill_context sandbox_path should be "/tmp/sandbox"
|
|
And the skill_context should not be read_only
|
|
|
|
|
|
Scenario: Create read-only SkillContext
|
|
When I create a read_only skill_context with plan "plan-002" project "proj-beta"
|
|
Then the skill_context should be created
|
|
And the skill_context should be read_only
|
|
|
|
|
|
Scenario: SkillContext with custom metadata
|
|
When I create a skill_context with metadata key "env" value "staging"
|
|
Then the skill_context plan metadata should contain key "env"
|
|
And the skill_context plan metadata key "env" should be "staging"
|
|
|
|
# ---- Resource Resolution ----
|
|
|
|
|
|
Scenario: Resolve bound resources from context
|
|
When I create a skill_context with resource "git-checkout" bound to "/repo/checkout"
|
|
And I resolve resource "git-checkout" from the skill_context
|
|
Then the resolved resource should be "/repo/checkout"
|
|
|
|
|
|
Scenario: Resolve missing resource raises SkillError
|
|
When I create a skill_context with no resources
|
|
And I try to resolve resource "missing-resource" from the skill_context
|
|
Then a skill_context resolution error should be raised
|
|
|
|
# ---- Change Tracking ----
|
|
|
|
|
|
Scenario: Register tool invocation in change tracker
|
|
When I create a skill_context with plan "plan-003" project "proj-gamma" and sandbox "/tmp/sb"
|
|
And I register a tool invocation for "local/edit-file" with duration 42.5
|
|
Then the skill_context change tracker should have 1 records
|
|
And the last change tracker record tool_name should be "local/edit-file"
|
|
And the last change tracker record duration_ms should be 42.5
|
|
|
|
|
|
Scenario: Multiple tool invocations are tracked in order
|
|
When I create a skill_context with plan "plan-004" project "proj-delta" and sandbox "/tmp/sb2"
|
|
And I register a tool invocation for "local/read-file" with duration 10.0
|
|
And I register a tool invocation for "local/write-file" with duration 20.0
|
|
Then the skill_context change tracker should have 2 records
|
|
|
|
# ---- Write Guard ----
|
|
|
|
|
|
Scenario: Enforce read-only write guard raises SkillError
|
|
When I create a read_only skill_context with plan "plan-005" project "proj-epsilon"
|
|
And I try to enforce write guard for tool "local/write-file"
|
|
Then a skill_context permission denied error should be raised
|
|
And the skill_context permission error tool_name should be "local/write-file"
|
|
|
|
|
|
Scenario: Write guard passes in writable context
|
|
When I create a skill_context with plan "plan-006" project "proj-zeta" and sandbox "/tmp/sb3"
|
|
And I enforce write guard for tool "local/write-file" in writable context
|
|
Then no skill_context error should be raised
|
|
|
|
# ---- Plan Metadata ----
|
|
|
|
|
|
Scenario: Get plan metadata includes plan_id and project_id
|
|
When I create a skill_context with plan "plan-007" project "proj-eta" and sandbox "/tmp/sb4"
|
|
Then the skill_context plan metadata should contain key "plan_id"
|
|
And the skill_context plan metadata key "plan_id" should be "plan-007"
|
|
And the skill_context plan metadata should contain key "project_id"
|
|
And the skill_context plan metadata key "project_id" should be "proj-eta"
|
|
|
|
# ---- SkillRegistry Register/Get/List/Unregister ----
|
|
|
|
|
|
Scenario: SkillRegistry register and get skill
|
|
When I create a skill_registry
|
|
And I register a skill "local/test-reg" in the registry
|
|
And I get skill "local/test-reg" from the registry
|
|
Then the registry skill name should be "local/test-reg"
|
|
|
|
|
|
Scenario: SkillRegistry register duplicate raises error
|
|
When I create a skill_registry
|
|
And I register a skill "local/dup-skill" in the registry
|
|
And I try to register a duplicate skill "local/dup-skill"
|
|
Then a skill_context validation error should be raised
|
|
|
|
|
|
Scenario: SkillRegistry get missing skill raises error
|
|
When I create a skill_registry
|
|
And I try to get skill "local/missing" from the registry
|
|
Then a skill_context not found error should be raised
|
|
|
|
|
|
Scenario: SkillRegistry list_all returns metadata
|
|
When I create a skill_registry
|
|
And I register a skill "local/skill-a" in the registry
|
|
And I register a skill "local/skill-b" in the registry
|
|
And I list all skills from the registry
|
|
Then the registry should have 2 skills
|
|
And the registry skill list should contain "local/skill-a"
|
|
And the registry skill list should contain "local/skill-b"
|
|
|
|
|
|
Scenario: SkillRegistry unregister removes skill
|
|
When I create a skill_registry
|
|
And I register a skill "local/to-remove" in the registry
|
|
And I unregister skill "local/to-remove" from the registry
|
|
And I try to get skill "local/to-remove" from the registry
|
|
Then a skill_context not found error should be raised
|
|
|
|
|
|
Scenario: SkillRegistry unregister missing skill raises error
|
|
When I create a skill_registry
|
|
And I try to unregister skill "local/nonexistent" from the registry
|
|
Then a skill_context not found error should be raised
|
|
|
|
# ---- SkillRegistry Tool Resolution ----
|
|
|
|
|
|
Scenario: SkillRegistry resolve_tools through tool references
|
|
When I create a skill_registry
|
|
And I register a skill "local/resolve-test" with tool refs in the registry
|
|
And I resolve tools for "local/resolve-test" from the registry
|
|
Then the resolved tools should have 2 entries
|
|
And the resolved tools should contain "local/tool-a"
|
|
And the resolved tools should contain "local/tool-b"
|
|
|
|
# ---- SkillRegistry Validation ----
|
|
|
|
|
|
@tdd_issue @tdd_issue_4241 @tdd_expected_fail
|
|
Scenario: SkillRegistry validate_skill detects missing tools
|
|
When I create a skill_registry with a mock tool registry
|
|
And I validate a skill with missing tool refs
|
|
Then the validation errors should contain "not found in tool registry"
|
|
|
|
|
|
Scenario: SkillRegistry validate_skill detects missing includes
|
|
When I create a skill_registry
|
|
And I validate a skill with missing includes
|
|
Then the validation errors should contain "is not registered"
|
|
|
|
|
|
Scenario: SkillRegistry validate_skill returns empty for valid skill
|
|
When I create a skill_registry
|
|
And I validate a skill with no issues
|
|
Then the validation errors should be empty
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: skill_discovery.feature
|
|
# Feature: Agent Skills Discovery
|
|
# ============================================================
|
|
|
|
Scenario: Config key skills.agent_skills_paths is registered
|
|
Given the agent skills discovery module is available
|
|
When I check the config registry for "skills.agent_skills_paths"
|
|
Then the config entry should exist
|
|
And the config entry type should be "str"
|
|
And the config entry env var should be "CLEVERAGENTS_SKILLS_AGENT_SKILLS_PATHS"
|
|
|
|
# --- Path parsing ---
|
|
|
|
|
|
Scenario: Parse single agent skills path
|
|
Given the agent skills discovery module is available
|
|
When I parse agent skills paths "~/.cleveragents/agent_skills"
|
|
Then the result should contain 1 path
|
|
|
|
|
|
Scenario: Parse multiple comma-separated paths
|
|
Given the agent skills discovery module is available
|
|
When I parse agent skills paths "/opt/skills,/home/user/skills"
|
|
Then the result should contain 2 paths
|
|
|
|
|
|
Scenario: Parse empty agent skills paths
|
|
Given the agent skills discovery module is available
|
|
When I parse an empty agent skills path string
|
|
Then the result should contain 0 paths
|
|
|
|
|
|
Scenario: Parse paths with whitespace around commas
|
|
Given the agent skills discovery module is available
|
|
When I parse agent skills paths "/opt/skills , /home/user/skills"
|
|
Then the result should contain 2 paths
|
|
|
|
# --- Discovery ---
|
|
|
|
|
|
Scenario: Discover agent skills from directory with SKILL.md
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with agent skill folders
|
|
| folder_name | skill_name | description |
|
|
| my-tool | my-tool | A test agent skill |
|
|
When I scan the directory for agent skills
|
|
Then I should discover 1 agent skill
|
|
And the discovered skill "my-tool" should have description "A test agent skill"
|
|
|
|
|
|
Scenario: Discover multiple agent skills
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with agent skill folders
|
|
| folder_name | skill_name | description |
|
|
| tool-a | tool-a | First tool |
|
|
| tool-b | tool-b | Second tool |
|
|
| tool-c | tool-c | Third tool |
|
|
When I scan the directory for agent skills
|
|
Then I should discover 3 agent skills
|
|
|
|
|
|
Scenario: Skip directories without SKILL.md
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with some folders missing SKILL.md
|
|
When I scan the directory for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: Handle non-existent directory gracefully
|
|
Given the agent skills discovery module is available
|
|
When I run discovery on a non-existent directory
|
|
Then the discovery result should have 0 discovered skills
|
|
And the discovery result should have errors mentioning "does not exist"
|
|
|
|
|
|
Scenario: Handle SKILL.md without front-matter
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has no front-matter
|
|
When I scan the directory for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: Fall back to folder name when SKILL.md has no name field
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with agent skill folders
|
|
| folder_name | skill_name | description |
|
|
| fallback-tool | _none_ | Some description |
|
|
When I scan the directory for agent skills
|
|
Then I should discover 1 agent skill
|
|
And the discovered skill should use folder name as name
|
|
|
|
# --- ToolSpec building ---
|
|
|
|
|
|
Scenario: Build ToolSpec from discovered agent skill
|
|
Given the agent skills discovery module is available
|
|
Given a discovered agent skill named "my-tool" at "/opt/skills/my-tool"
|
|
When I build a ToolSpec from the discovered skill
|
|
Then the ToolSpec name should be "agent_skills/my-tool"
|
|
And the ToolSpec source should be "agent_skills"
|
|
And the ToolSpec source_metadata should contain path "/opt/skills/my-tool"
|
|
|
|
# --- Registration ---
|
|
|
|
|
|
Scenario: Register discovered agent skills in ToolRegistry
|
|
Given the agent skills discovery module is available
|
|
Given a ToolRegistry with no existing tools
|
|
And a list of 2 discovered agent skills
|
|
When I register discovered skills with "skip" conflict strategy
|
|
Then 2 tools should be registered in the ToolRegistry
|
|
And 0 conflicts should be reported
|
|
|
|
|
|
Scenario: Skip registration on name collision with skip strategy
|
|
Given the agent skills discovery module is available
|
|
Given a ToolRegistry with an existing tool "agent_skills/collider"
|
|
And a discovered agent skill named "collider"
|
|
When I register discovered skills with "skip" conflict strategy
|
|
Then 0 tools should be registered in the ToolRegistry
|
|
And 1 conflict should be reported with tool name "agent_skills/collider"
|
|
|
|
|
|
Scenario: Error on name collision with error strategy
|
|
Given the agent skills discovery module is available
|
|
Given a ToolRegistry with an existing tool "agent_skills/collider"
|
|
And a discovered agent skill named "collider"
|
|
When I register discovered skills with "error" conflict strategy
|
|
Then a discovery ValueError should be raised mentioning "name collision"
|
|
|
|
|
|
Scenario: Replace existing tool on collision with replace strategy
|
|
Given the agent skills discovery module is available
|
|
Given a ToolRegistry with an existing tool "agent_skills/collider"
|
|
And a discovered agent skill named "collider"
|
|
When I register discovered skills with "replace" conflict strategy
|
|
Then 1 tool should be registered in the ToolRegistry
|
|
And the tool "agent_skills/collider" should have source "agent_skills"
|
|
|
|
# --- Refresh ---
|
|
|
|
|
|
Scenario: Refresh removes old agent skills and re-discovers
|
|
Given the agent skills discovery module is available
|
|
Given a SkillRegistryService with a ToolRegistry
|
|
And agent skills paths pointing to a directory with 2 skills
|
|
When I call discover_and_register
|
|
Then 2 agent skills should be registered
|
|
When the directory now has 3 skills and I call refresh_agent_skills
|
|
Then 3 agent skills should be registered
|
|
|
|
# --- Edge cases ---
|
|
|
|
|
|
Scenario: Register empty discovered list returns empty
|
|
Given the agent skills discovery module is available
|
|
Given a ToolRegistry with no existing tools
|
|
When I register an empty discovered skills list
|
|
Then the registration should return 0 specs and 0 conflicts
|
|
|
|
|
|
Scenario: Scan non-directory path returns empty
|
|
Given the agent skills discovery module is available
|
|
When I scan a non-directory path for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: Discover with path that is a file not a directory
|
|
Given the agent skills discovery module is available
|
|
When I run discovery on a path that is a file
|
|
Then the discovery result should have 0 discovered skills
|
|
And the discovery result should have errors mentioning "not a directory"
|
|
|
|
|
|
Scenario: SKILL.md with invalid YAML front-matter
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has invalid YAML
|
|
When I scan the directory for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: SKILL.md with empty YAML block
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has empty front-matter
|
|
When I scan the directory for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: SKILL.md with non-dict YAML
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has non-dict YAML
|
|
When I scan the directory for agent skills
|
|
Then I should discover 0 agent skills
|
|
|
|
|
|
Scenario: Skill with non-string description in front-matter
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has numeric description
|
|
When I scan the directory for agent skills
|
|
Then I should discover 1 agent skill
|
|
|
|
|
|
Scenario: Skill with empty description falls back to default
|
|
Given the agent skills discovery module is available
|
|
Given a temporary directory with a SKILL.md that has empty description
|
|
When I scan the directory for agent skills
|
|
Then I should discover 1 agent skill
|
|
And the first discovered skill description should contain "Agent skill from"
|
|
|
|
|
|
Scenario: Noop handler returns expected placeholder
|
|
Given the agent skills discovery module is available
|
|
When I call the noop handler
|
|
Then the noop handler should return status "agent_skill_placeholder"
|
|
|
|
# --- Source metadata ---
|
|
|
|
|
|
Scenario: Source metadata is included when resolving skill tools
|
|
Given the agent skills discovery module is available
|
|
Given a registered skill "local/test-skill" with an agent_skill source
|
|
When I resolve tools for "local/test-skill"
|
|
Then the resolved tools should include agent_skill source entries
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: skill_flatten.feature
|
|
# Feature: Skill Registry Flattening and Capability Summaries
|
|
# ============================================================
|
|
|
|
@flatten_tool_refs
|
|
Scenario: Flattening a skill with named tool refs produces deterministic order
|
|
Given a flatten registry skill "local/basic" with tool refs "local/alpha,local/beta,local/gamma"
|
|
When I flatten the skill "local/basic"
|
|
Then the flatten result should have 3 entries
|
|
And the flatten entry at index 0 should be "local/alpha"
|
|
And the flatten entry at index 1 should be "local/beta"
|
|
And the flatten entry at index 2 should be "local/gamma"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Flattening with includes (depth-first ordering)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_includes
|
|
Scenario: Flattening a skill with includes uses depth-first ordering
|
|
Given a flatten registry skill "local/base" with tool refs "local/base-tool"
|
|
And a flatten registry skill "local/mid" including "local/base" with refs "local/mid-tool"
|
|
And a flatten registry skill "local/top" including "local/mid" with refs "local/top-tool"
|
|
When I flatten the skill "local/top"
|
|
Then the flatten result should have 3 entries
|
|
And the flatten entry at index 0 should be "local/base-tool"
|
|
And the flatten entry at index 1 should be "local/mid-tool"
|
|
And the flatten entry at index 2 should be "local/top-tool"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Flattening with inline tools
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_inline
|
|
Scenario: Flattening a skill with inline tools includes anonymous entries
|
|
Given a flatten registry skill "local/inline-skill" with 2 inline tools
|
|
When I flatten the skill "local/inline-skill"
|
|
Then the flatten result should have 2 entries
|
|
And the flatten entry at index 0 should be "local/inline-skill/_anon_0"
|
|
And the flatten entry at index 1 should be "local/inline-skill/_anon_1"
|
|
And the flatten entry at index 0 should be marked inline
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Cycle detection with clear error path
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_cycle
|
|
Scenario: Flattening detects cycles and reports a clear path
|
|
Given a flatten registry skill "local/cycle-a" including "local/cycle-b" with no refs
|
|
And a flatten registry skill "local/cycle-b" including "local/cycle-a" with no refs
|
|
When I try to flatten the skill "local/cycle-a"
|
|
Then a flatten cycle error should be raised
|
|
And the flatten error should mention "local/cycle-a"
|
|
And the flatten error should mention "local/cycle-b"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Per-include override application
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_include_overrides
|
|
Scenario: Per-include overrides are applied to included tool entries
|
|
Given a flatten registry skill "local/child" with tool refs "local/tool-x"
|
|
And a flatten registry skill "local/parent" including "local/child" with overrides timeout 600
|
|
When I flatten the skill "local/parent"
|
|
Then the flatten result should have 1 entries
|
|
And the flatten entry "local/tool-x" should have override "timeout" equal to 600
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Non-overridable field rejection
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_non_overridable
|
|
Scenario: Non-overridable fields are rejected with clear error
|
|
Given a flatten registry skill "local/child-nr" with tool refs "local/tool-y"
|
|
And a flatten registry skill "local/parent-nr" including "local/child-nr" with overrides on non-overridable field "name"
|
|
When I try to flatten the skill "local/parent-nr"
|
|
Then a flatten override error should be raised
|
|
And the flatten error should mention "name"
|
|
And the flatten error should mention "Non-overridable"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Capability summary computation
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_capability_summary
|
|
Scenario: Capability summary aggregates read/write/checkpoint/side-effect flags
|
|
Given a flatten registry skill "local/cap-skill" with mixed capability inline tools
|
|
When I flatten the skill "local/cap-skill" and compute summary
|
|
Then the flatten summary total_tools should be 3
|
|
And the flatten summary read_only_tools should be 1
|
|
And the flatten summary write_tools should be 1
|
|
And the flatten summary has_side_effects should be true
|
|
And the flatten summary checkpointable_tools should be 2
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# tools() method returns both entries and summary
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_tools_method
|
|
Scenario: The tools() method returns entries and capability summary
|
|
Given a flatten skill registry with skill "local/tools-test" having 3 refs
|
|
When I call the flatten tools method for "local/tools-test"
|
|
Then the flatten tools result should contain 3 entries
|
|
And the flatten tools result should include a capability summary
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# validate_plan() with valid plan
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_validate_plan
|
|
Scenario: validate_plan with a valid plan returns no errors
|
|
Given a flatten skill registry with skill "local/plan-skill" having 2 refs
|
|
When I call flatten validate_plan with skills "local/plan-skill"
|
|
Then the flatten validation result should have 0 errors
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# validate_plan() with missing skill
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_validate_plan_missing
|
|
Scenario: validate_plan with a missing skill reports error
|
|
Given an empty flatten skill registry
|
|
When I call flatten validate_plan with skills "local/nonexistent"
|
|
Then the flatten validation result should have 1 errors
|
|
And the flatten validation error should mention "local/nonexistent"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# validate_plan() with cycle in includes
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_validate_plan_cycle
|
|
Scenario: validate_plan detects cycle in includes
|
|
Given a flatten registry skill "local/loop-a" including "local/loop-b" with no refs
|
|
And a flatten registry skill "local/loop-b" including "local/loop-a" with no refs
|
|
And the flatten registry skills are registered in the skill registry
|
|
When I call flatten validate_plan with skills "local/loop-a"
|
|
Then the flatten validation result should have 1 errors
|
|
And the flatten validation error should mention "ycle"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# De-duplication semantics (last-wins)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_dedup
|
|
Scenario: De-duplication uses last-wins for entry metadata
|
|
Given a flatten registry skill "local/dup-a" with tool refs "local/shared"
|
|
And a flatten registry skill "local/dup-b" with tool refs "local/shared"
|
|
And a flatten registry skill "local/dup-top" including "local/dup-a,local/dup-b" with no refs
|
|
When I flatten the skill "local/dup-top"
|
|
Then the flatten result should have 1 entries
|
|
And the flatten entry "local/shared" source_skill should be "local/dup-b"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Override merging (shallow merge)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@flatten_override_merge
|
|
Scenario: Override merging uses shallow merge semantics
|
|
Given a flatten registry skill "local/merge-child" with tool refs "local/merge-tool" and overrides priority 1
|
|
And a flatten registry skill "local/merge-parent" including "local/merge-child" with overrides priority 2 and extra key
|
|
When I flatten the skill "local/merge-parent"
|
|
Then the flatten entry "local/merge-tool" should have override "priority" equal to 2
|
|
And the flatten entry "local/merge-tool" should have override "extra" equal to "yes"
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: skill_inline_coverage.feature
|
|
# Feature: Inline Tool Executor Coverage Boost
|
|
# ============================================================
|
|
|
|
Scenario: Executor rejects non-positive max_runtime_seconds
|
|
Given I create an inline executor with invalid max runtime
|
|
Then creating the executor should raise a ValueError about runtime
|
|
|
|
|
|
Scenario: Executor rejects non-positive max_output_bytes
|
|
Given I create an inline executor with invalid max output
|
|
Then creating the executor should raise a ValueError about output
|
|
|
|
# ---- Property Accessors ----
|
|
|
|
|
|
Scenario: Executor exposes max_runtime_seconds property
|
|
Given an inline executor with max runtime 5.0 seconds
|
|
Then the executor max_runtime_seconds should be 5.0
|
|
|
|
|
|
Scenario: Executor exposes max_output_bytes property
|
|
Given an inline executor with max output 512 bytes
|
|
Then the executor max_output_bytes should be 512
|
|
|
|
# ---- Null Argument Guards ----
|
|
|
|
|
|
Scenario: Execute rejects None tool argument
|
|
Given a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
And an inline executor with max runtime 5.0 seconds
|
|
When I execute with None tool
|
|
Then executing should raise a ValueError about tool
|
|
|
|
|
|
Scenario: Execute rejects None context argument
|
|
Given an inline tool with code that prints "test"
|
|
When I execute with None context
|
|
Then executing should raise a ValueError about context
|
|
|
|
|
|
Scenario: Execute rejects None input_data argument
|
|
Given an inline tool with code that prints "test"
|
|
And a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
When I execute with None input_data
|
|
Then executing should raise a ValueError about input_data
|
|
|
|
|
|
Scenario: Validate_tool rejects None tool argument
|
|
Given an inline executor with max runtime 5.0 seconds
|
|
When I validate with None tool
|
|
Then validating should raise a ValueError about tool
|
|
|
|
# ---- Validation Failure Return Path ----
|
|
|
|
|
|
Scenario: Execute returns validation failure without running code
|
|
Given an inline tool with no code
|
|
And a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
When I execute the inline tool
|
|
Then the inline result should not be successful
|
|
And the inline result error message should contain "Validation failed"
|
|
And the inline result duration should be zero
|
|
|
|
# ---- Sandbox Path Validation ----
|
|
|
|
|
|
Scenario: Execute rejects input paths outside sandbox
|
|
Given an inline tool with code that prints "file op"
|
|
And a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
When I execute the inline tool with path input escaping sandbox
|
|
Then the inline result should not be successful
|
|
And the inline result error message should contain "escapes sandbox"
|
|
|
|
|
|
Scenario: Execute accepts input paths inside sandbox
|
|
Given an inline tool with code that prints "ok"
|
|
And a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
When I execute the inline tool with path input inside sandbox
|
|
Then the inline result should be successful
|
|
|
|
|
|
Scenario: Execute handles invalid path values gracefully
|
|
Given an inline tool with code that prints "test"
|
|
And a writable skill context with sandbox "/tmp/inline-sandbox"
|
|
When I execute the inline tool with an invalid path value
|
|
Then the inline result should not be successful
|
|
And the inline result error message should contain "Invalid path"
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: skill_protocol.feature
|
|
# Feature: Skill Protocol Types
|
|
# ============================================================
|
|
|
|
Scenario: SkillErrorType has all expected members
|
|
When I list all skill_protocol error types
|
|
Then the skill_protocol error types should include "skill_not_found"
|
|
And the skill_protocol error types should include "resolution_failure"
|
|
And the skill_protocol error types should include "cycle_detected"
|
|
And the skill_protocol error types should include "tool_activation_failure"
|
|
And the skill_protocol error types should include "tool_execution_failure"
|
|
And the skill_protocol error types should include "validation_error"
|
|
And the skill_protocol error types should include "permission_denied"
|
|
|
|
|
|
Scenario: SkillErrorType has exactly 7 members
|
|
When I list all skill_protocol error types
|
|
Then there should be 7 skill_protocol error types
|
|
|
|
# ---- SkillError Creation ----
|
|
|
|
|
|
Scenario: Create a SkillError with required fields
|
|
When I create a skill_protocol error with type "tool_execution_failure" and message "Tool timed out" for skill "local/my-skill"
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error type should be "tool_execution_failure"
|
|
And the skill_protocol error message should be "Tool timed out"
|
|
And the skill_protocol error skill_name should be "local/my-skill"
|
|
And the skill_protocol error tool_name should be none
|
|
|
|
|
|
Scenario: Create a SkillError with optional tool_name
|
|
When I create a skill_protocol error with tool_name "local/my-tool"
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error tool_name should be "local/my-tool"
|
|
|
|
|
|
Scenario: Create a SkillError with details
|
|
When I create a skill_protocol error with details
|
|
Then the skill_protocol error should be created
|
|
And the skill_protocol error details should have key "trace_id"
|
|
|
|
|
|
Scenario: SkillError is frozen (immutable)
|
|
When I create a skill_protocol error with type "validation_error" and message "Bad input" for skill "local/test"
|
|
Then the skill_protocol error should be immutable
|
|
|
|
# ---- SkillMetadata Creation ----
|
|
|
|
|
|
Scenario: Create SkillMetadata from Skill domain model
|
|
When I create a skill_protocol metadata from a basic skill
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata name should be "local/test-skill"
|
|
And the skill_protocol metadata description should be "A test skill"
|
|
And the skill_protocol metadata version should be "0.0.0"
|
|
And the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
|
|
|
|
Scenario: Create SkillMetadata from Skill with tool refs
|
|
When I create a skill_protocol metadata from a skill with tool refs
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata tool_count should be 2
|
|
And the skill_protocol metadata source_types should contain "tool_ref"
|
|
|
|
|
|
Scenario: Create SkillMetadata from Skill with inline write tools
|
|
When I create a skill_protocol metadata from a skill with write tools
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata writes should be true
|
|
And the skill_protocol metadata read_only should be false
|
|
|
|
|
|
Scenario: Create SkillMetadata from Skill with MCP sources
|
|
When I create a skill_protocol metadata from a skill with mcp sources
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata source_types should contain "mcp"
|
|
|
|
|
|
Scenario: Create SkillMetadata from Skill with agent skill sources
|
|
When I create a skill_protocol metadata from a skill with agent skill sources
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata source_types should contain "agent_skill"
|
|
|
|
|
|
Scenario: Create SkillMetadata with custom version
|
|
When I create a skill_protocol metadata with version "1.2.3"
|
|
Then the skill_protocol metadata should be created
|
|
And the skill_protocol metadata version should be "1.2.3"
|
|
|
|
|
|
Scenario: SkillMetadata is frozen (immutable)
|
|
When I create a skill_protocol metadata from a basic skill
|
|
Then the skill_protocol metadata should be immutable
|
|
|
|
|
|
Scenario: SkillMetadata from_skill rejects None
|
|
When I try to create skill_protocol metadata from None
|
|
Then a skill_protocol value error should be raised
|
|
|
|
# ---- SkillDefinition ----
|
|
|
|
|
|
Scenario: Create a SkillDefinition with resolved tools
|
|
When I create a skill_protocol definition with resolved tools
|
|
Then the skill_protocol definition should be created
|
|
And the skill_protocol definition skill name should be "local/def-skill"
|
|
And the skill_protocol definition should have 1 resolved tools
|
|
|
|
|
|
Scenario: Create a SkillDefinition with input and output schemas
|
|
When I create a skill_protocol definition with input and output schemas
|
|
Then the skill_protocol definition should be created
|
|
And the skill_protocol definition input_schema should have key "type"
|
|
And the skill_protocol definition output_schema should have key "type"
|
|
|
|
|
|
Scenario: SkillDefinition rejects input_schema without type key
|
|
When I try to create a skill_protocol definition with invalid input_schema
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
|
|
Scenario: SkillDefinition rejects output_schema without type key
|
|
When I try to create a skill_protocol definition with invalid output_schema
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
|
|
Scenario: SkillDefinition detects writes inconsistency
|
|
When I try to create a skill_protocol definition with writes inconsistency
|
|
Then a skill_protocol validation error should be raised
|
|
|
|
|
|
Scenario: SkillDefinition is frozen (immutable)
|
|
When I create a skill_protocol definition with resolved tools
|
|
Then the skill_protocol definition should be immutable
|
|
|
|
# ---- SkillResult ----
|
|
|
|
|
|
Scenario: Create a successful SkillResult
|
|
When I create a skill_protocol result with success
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result success should be true
|
|
And the skill_protocol result skill_name should be "local/my-skill"
|
|
And the skill_protocol result tool_name should be "local/my-tool"
|
|
And the skill_protocol result error should be none
|
|
And the skill_protocol result output_data should be "hello"
|
|
|
|
|
|
Scenario: Create a failed SkillResult with error
|
|
When I create a skill_protocol result with failure
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result success should be false
|
|
And the skill_protocol result error should not be none
|
|
And the skill_protocol result error type should be "tool_execution_failure"
|
|
|
|
|
|
Scenario: Create a SkillResult with duration and resource changes
|
|
When I create a skill_protocol result with duration and resource changes
|
|
Then the skill_protocol result should be created
|
|
And the skill_protocol result duration_ms should be 150.5
|
|
And the skill_protocol result should have 1 resource changes
|
|
|
|
|
|
Scenario: SkillResult is frozen (immutable)
|
|
When I create a skill_protocol result with success
|
|
Then the skill_protocol result should be immutable
|
|
|
|
# ---- Error Mapping ----
|
|
|
|
|
|
Scenario: Map ValueError with cycle message to CYCLE_DETECTED
|
|
When I map a skill_protocol ValueError with message "Cycle detected in skill includes"
|
|
Then the skill_protocol mapped error type should be "cycle_detected"
|
|
|
|
|
|
Scenario: Map ValueError with not found message to SKILL_NOT_FOUND
|
|
When I map a skill_protocol ValueError with message "Skill not found"
|
|
Then the skill_protocol mapped error type should be "skill_not_found"
|
|
|
|
|
|
Scenario: Map generic ValueError to RESOLUTION_FAILURE
|
|
When I map a skill_protocol ValueError with message "Something went wrong"
|
|
Then the skill_protocol mapped error type should be "resolution_failure"
|
|
|
|
|
|
Scenario: Map PermissionError to PERMISSION_DENIED
|
|
When I map a skill_protocol PermissionError
|
|
Then the skill_protocol mapped error type should be "permission_denied"
|
|
|
|
|
|
Scenario: Map RuntimeError to TOOL_EXECUTION_FAILURE
|
|
When I map a skill_protocol RuntimeError with message "Unexpected failure"
|
|
Then the skill_protocol mapped error type should be "tool_execution_failure"
|
|
|
|
|
|
Scenario: Map RuntimeError with activation in message
|
|
When I map a skill_protocol RuntimeError with message "activation failed"
|
|
Then the skill_protocol mapped error type should be "tool_activation_failure"
|
|
|
|
|
|
Scenario: Error mapping with tool_name populated
|
|
When I map a skill_protocol error with tool_name "local/my-tool"
|
|
Then the skill_protocol mapped error tool_name should be "local/my-tool"
|
|
And the skill_protocol mapped error details should have key "tool_name"
|
|
|
|
|
|
Scenario: Error mapping rejects empty skill_name
|
|
When I try to map a skill_protocol error with empty skill_name
|
|
Then a skill_protocol value error should be raised
|
|
|
|
# ---- Writes/Read-Only Metadata Propagation ----
|
|
|
|
|
|
Scenario: Skill with only read-only tools has read_only=True
|
|
When I create a skill_protocol metadata from a skill with read_only tools
|
|
Then the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
|
|
|
|
Scenario: Skill with write tools has writes=True
|
|
When I create a skill_protocol metadata from a skill with write tools
|
|
Then the skill_protocol metadata writes should be true
|
|
And the skill_protocol metadata read_only should be false
|
|
|
|
|
|
Scenario: Empty skill has read_only=True
|
|
When I create a skill_protocol metadata from an empty skill
|
|
Then the skill_protocol metadata read_only should be true
|
|
And the skill_protocol metadata writes should be false
|
|
And the skill_protocol metadata tool_count should be 0
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: skill_resolution.feature
|
|
# Feature: Skill Domain Model and Resolution
|
|
# ============================================================
|
|
|
|
Scenario: Create a skill with valid fields
|
|
When I create a skill_model with name "local/code-tools" and description "Code editing tools"
|
|
Then the skill_model should be created
|
|
And the skill_model name should be "local/code-tools"
|
|
And the skill_model description should be "Code editing tools"
|
|
And the skill_model namespace should be "local"
|
|
And the skill_model short_name should be "code-tools"
|
|
|
|
|
|
Scenario: Create a skill with tool refs
|
|
When I create a skill_model with tool refs "local/edit-file,local/read-file"
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
|
|
|
|
Scenario: Create a skill with all fields populated
|
|
When I create a skill_model with all fields
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
And the skill_model should have 1 include
|
|
And the skill_model should have 1 anonymous tool
|
|
And the skill_model should have 1 mcp server
|
|
And the skill_model should have 1 agent skill
|
|
|
|
# ---- Skill Name Validation ----
|
|
|
|
|
|
Scenario: Skill name must match namespace/name pattern
|
|
When I try to create a skill_model with invalid name "no-namespace"
|
|
Then a skill_model validation error should be raised
|
|
And the skill_model error should mention "namespace/name"
|
|
|
|
|
|
Scenario: Skill name rejects spaces
|
|
When I try to create a skill_model with invalid name "local/bad name"
|
|
Then a skill_model validation error should be raised
|
|
|
|
|
|
Scenario: Skill name rejects double slashes
|
|
When I try to create a skill_model with invalid name "local//bad"
|
|
Then a skill_model validation error should be raised
|
|
|
|
|
|
Scenario: Skill name allows hyphens and underscores
|
|
When I create a skill_model with name "my-ns/my_skill" and description "Test"
|
|
Then the skill_model should be created
|
|
And the skill_model namespace should be "my-ns"
|
|
And the skill_model short_name should be "my_skill"
|
|
|
|
|
|
Scenario: Skill description must not be empty
|
|
When I try to create a skill_model with empty description
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- Include Resolution (flat) ----
|
|
|
|
|
|
Scenario: Resolve skill with flat includes
|
|
Given a skill_resolver registry with "local/basic-tools" containing refs "local/edit-file,local/read-file"
|
|
And a skill_resolver root skill "local/code-tools" including "local/basic-tools" with own refs "local/format"
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver result should contain tool "local/edit-file"
|
|
And the skill_resolver result should contain tool "local/read-file"
|
|
And the skill_resolver result should contain tool "local/format"
|
|
|
|
# ---- Include Resolution (nested) ----
|
|
|
|
|
|
Scenario: Resolve skill with nested includes
|
|
Given a skill_resolver registry with "local/base" containing refs "local/core-tool"
|
|
And a skill_resolver registry with "local/mid" including "local/base" and containing refs "local/mid-tool"
|
|
And a skill_resolver root skill "local/top" including "local/mid" with own refs "local/top-tool"
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver result should contain tool "local/core-tool"
|
|
And the skill_resolver result should contain tool "local/mid-tool"
|
|
And the skill_resolver result should contain tool "local/top-tool"
|
|
|
|
# ---- Cycle Detection ----
|
|
|
|
|
|
Scenario: Cycle detection with error message
|
|
Given a skill_resolver registry with "local/a" including "local/b"
|
|
And a skill_resolver registry with "local/b" including "local/a"
|
|
When I try to resolve skill_resolver "local/a"
|
|
Then a skill_resolver cycle error should be raised
|
|
And the skill_resolver error should mention "local/a"
|
|
And the skill_resolver error should mention "local/b"
|
|
|
|
|
|
Scenario: Self-referencing cycle detection
|
|
Given a skill_resolver registry with "local/self" including "local/self"
|
|
When I try to resolve skill_resolver "local/self"
|
|
Then a skill_resolver cycle error should be raised
|
|
|
|
# ---- De-duplication ----
|
|
|
|
|
|
Scenario: De-duplication behavior last wins for overrides
|
|
Given a skill_resolver registry with "local/a" containing refs "local/shared-tool"
|
|
And a skill_resolver registry with "local/b" containing refs "local/shared-tool"
|
|
And a skill_resolver root skill "local/top" including "local/a,local/b" with no own refs
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 1 tools
|
|
And the skill_resolver tool "local/shared-tool" should come from "local/b"
|
|
|
|
# ---- Override Application ----
|
|
|
|
|
|
Scenario: Override application on tool refs
|
|
When I create a skill_model with overrides for "local/edit-file"
|
|
And I resolve the overridden skill_model
|
|
Then the skill_resolver tool "local/edit-file" should have overrides
|
|
|
|
# ---- Empty Skill ----
|
|
|
|
|
|
Scenario: Empty skill with no tools or includes
|
|
When I create a skill_model with name "local/empty" and description "Empty skill"
|
|
And I resolve the empty skill_model
|
|
Then the skill_resolver result should have 0 tools
|
|
|
|
# ---- from_config Loading ----
|
|
|
|
|
|
Scenario: Load skill from config dict
|
|
When I load a skill_model from config with name "local/my-skill"
|
|
Then the skill_model should be created
|
|
And the skill_model name should be "local/my-skill"
|
|
|
|
|
|
Scenario: Load skill from config missing name raises error
|
|
When I try to load a skill_model from config missing "name"
|
|
Then a skill_model config error should be raised with "name"
|
|
|
|
|
|
Scenario: Load skill from config missing description raises error
|
|
When I try to load a skill_model from config missing "description"
|
|
Then a skill_model config error should be raised with "description"
|
|
|
|
|
|
Scenario: Load skill from config with full YAML structure
|
|
When I load a skill_model from full config
|
|
Then the skill_model should be created
|
|
And the skill_model should have 2 tool refs
|
|
And the skill_model should have 1 include
|
|
And the skill_model should have 1 anonymous tool
|
|
And the skill_model should have 1 mcp server
|
|
And the skill_model should have 1 agent skill
|
|
|
|
|
|
Scenario: Load skill from config with string includes
|
|
When I load a skill_model from config with string includes
|
|
Then the skill_model should be created
|
|
And the skill_model should have 1 include
|
|
|
|
|
|
Scenario: Load skill from config with string agent skills
|
|
When I load a skill_model from config with string agent skills
|
|
Then the skill_model should be created
|
|
And the skill_model should have 1 agent skill
|
|
|
|
# ---- as_cli_dict Output ----
|
|
|
|
|
|
Scenario: Skill as_cli_dict has required keys
|
|
When I create a skill_model and call as_cli_dict
|
|
Then the skill_model cli dict should have key "name"
|
|
And the skill_model cli dict should have key "description"
|
|
And the skill_model cli dict should have key "namespace"
|
|
And the skill_model cli dict should have key "short_name"
|
|
|
|
|
|
Scenario: Skill as_cli_dict includes optional sections when set
|
|
When I create a skill_model with all fields and call as_cli_dict
|
|
Then the skill_model cli dict should have key "tool_refs"
|
|
And the skill_model cli dict should have key "includes"
|
|
And the skill_model cli dict should have key "anonymous_tools"
|
|
And the skill_model cli dict should have key "mcp_servers"
|
|
And the skill_model cli dict should have key "agent_skills"
|
|
And the skill_model cli dict should have key "overrides"
|
|
|
|
|
|
Scenario: Skill as_cli_dict omits empty sections
|
|
When I create a skill_model with name "local/minimal" and description "Minimal"
|
|
And I get the skill_model cli dict
|
|
Then the skill_model cli dict should not have key "tool_refs"
|
|
And the skill_model cli dict should not have key "includes"
|
|
And the skill_model cli dict should not have key "anonymous_tools"
|
|
And the skill_model cli dict should not have key "mcp_servers"
|
|
And the skill_model cli dict should not have key "agent_skills"
|
|
And the skill_model cli dict should not have key "overrides"
|
|
|
|
# ---- Inline Tool Validation ----
|
|
|
|
|
|
Scenario: Inline tool with valid custom source
|
|
When I create a skill_model inline tool with source "custom" and code "print('hi')"
|
|
Then the skill_model inline tool should be created
|
|
|
|
|
|
Scenario: Inline tool has default timeout of 300
|
|
When I create a skill_model inline tool with source "custom" and code "print('hi')"
|
|
Then the skill_model inline tool timeout should be 300
|
|
|
|
|
|
Scenario: Inline tool timeout must be >= 1
|
|
When I try to create a skill_model inline tool with timeout 0
|
|
Then a skill_model validation error should be raised
|
|
|
|
|
|
Scenario: Inline tool description must not be empty
|
|
When I try to create a skill_model inline tool with empty description
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- MCP Source Validation ----
|
|
|
|
|
|
Scenario: MCP source with specific tools
|
|
When I create a skill_model mcp source with server "npx @mcp/server" and tools "read,write"
|
|
Then the skill_model mcp source should be created
|
|
And the skill_model mcp source should have 2 tools
|
|
|
|
|
|
Scenario: MCP source with all tools (none specified)
|
|
When I create a skill_model mcp source with server "npx @mcp/server" and no tools
|
|
Then the skill_model mcp source should be created
|
|
And the skill_model mcp source tools should be none
|
|
|
|
|
|
Scenario: MCP source with env vars
|
|
When I create a skill_model mcp source with env vars
|
|
Then the skill_model mcp source should be created
|
|
|
|
|
|
Scenario: MCP source server must not be empty
|
|
When I try to create a skill_model mcp source with empty server
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- Agent Skill Source Validation ----
|
|
|
|
|
|
Scenario: Agent skill source with valid path
|
|
When I create a skill_model agent source with path "./skills/review"
|
|
Then the skill_model agent source should be created
|
|
|
|
|
|
Scenario: Agent skill source path must not be empty
|
|
When I try to create a skill_model agent source with empty path
|
|
Then a skill_model validation error should be raised
|
|
|
|
# ---- CapabilitySummary Computation ----
|
|
|
|
|
|
Scenario: CapabilitySummary computation for skill with inline tools
|
|
When I compute skill_model capability summary for a skill with inline tools
|
|
Then the skill_model summary total tools should be 4
|
|
And the skill_model summary read_only_tools should be 1
|
|
And the skill_model summary write_tools should be 1
|
|
And the skill_model summary has_side_effects should be true
|
|
And the skill_model summary mcp_sources should be 1
|
|
And the skill_model summary agent_skill_sources should be 1
|
|
|
|
|
|
Scenario: CapabilitySummary for empty skill
|
|
When I compute skill_model capability summary for an empty skill
|
|
Then the skill_model summary total tools should be 0
|
|
|
|
# ---- SkillToolRef ----
|
|
|
|
|
|
Scenario: SkillToolRef creation
|
|
When I create a skill_model tool ref with name "local/my-tool"
|
|
Then the skill_model tool ref should be created
|
|
|
|
# ---- SkillInclude ----
|
|
|
|
|
|
Scenario: SkillInclude creation with overrides
|
|
When I create a skill_model include with name "local/base" and overrides
|
|
Then the skill_model include should be created
|
|
And the skill_model include should have overrides
|
|
|
|
|
|
Scenario: SkillInclude creation without overrides
|
|
When I create a skill_model include with name "local/base" without overrides
|
|
Then the skill_model include should be created
|
|
|
|
# ---- Missing included skill ----
|
|
|
|
|
|
Scenario: Resolve with missing included skill raises error
|
|
Given a skill_resolver root skill "local/broken" including "local/nonexistent" with no own refs
|
|
When I try to resolve skill_resolver "local/broken"
|
|
Then a skill_resolver missing skill error should be raised
|
|
And the skill_resolver error should mention "local/nonexistent"
|
|
|
|
# ---- MCP and agent skill resolution ----
|
|
|
|
|
|
Scenario: Resolve skill with MCP and agent skill sources
|
|
When I create and resolve a skill_model with mcp and agent sources
|
|
Then the skill_resolver result should have 3 tools
|
|
|
|
# ---- ResolvedToolEntry ----
|
|
|
|
|
|
Scenario: ResolvedToolEntry creation
|
|
When I create a skill_resolver resolved tool entry
|
|
Then the skill_resolver entry should be created
|
|
|
|
# ---- Robot Smoke Mirrors (Behave equivalents of robot/skill_resolution.robot) ----
|
|
|
|
|
|
Scenario: Resolver output ordering matches depth-first include order
|
|
Given a skill_resolver registry with "local/base-tools" containing refs "local/edit-file,local/read-file"
|
|
And a skill_resolver root skill "local/code-tools" including "local/base-tools" with own refs "local/format"
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver tool at index 0 should be "local/edit-file"
|
|
And the skill_resolver tool at index 1 should be "local/read-file"
|
|
And the skill_resolver tool at index 2 should be "local/format"
|
|
|
|
|
|
Scenario: Resolver de-duplicates overlapping include tools with last-wins
|
|
Given a skill_resolver registry with "local/a" containing refs "local/shared-tool,local/a-only"
|
|
And a skill_resolver registry with "local/b" containing refs "local/shared-tool,local/b-only"
|
|
And a skill_resolver root skill "local/top" including "local/a,local/b" with no own refs
|
|
When I resolve the skill_resolver root skill
|
|
Then the skill_resolver result should have 3 tools
|
|
And the skill_resolver tool "local/shared-tool" should come from "local/b"
|
|
|
|
|
|
Scenario: Resolver includes inline tools with correct anon keys
|
|
When I create and resolve a skill_model with inline tools
|
|
Then the skill_resolver result should contain tool "local/inline-test/_anon_0"
|
|
And the skill_resolver result should contain tool "local/inline-test/_anon_1"
|
|
And the skill_resolver inline tool at index 1 should be marked inline
|
|
|