forked from cleveragents/cleveragents-core
freemo
1f94b7e8d5
- Implemented a pre-registration validation in SkillService.add_skill() to ensure all skills listed in the includes field are already registered in self._skills before accepting the new skill registration. This prevents registering a skill that depends on non-existent skills. - The validation applies to both the initial add and when update=True (re-registration), ensuring dependencies are consistently enforced across both creation and update flows. - If any included skills are missing, a clear ValueError is raised listing all missing skill names with guidance to register them first, providing complete visibility rather than failing on the first missing item. - Added new Behave feature file features/skill_add_include_validation.feature with 8 scenarios covering: single missing include, multiple missing includes, partial includes, all includes registered (success), update with missing include, and service-level assertions. Also added step definitions in features/steps/skill_add_include_validation_steps.py. - Updated features/skill_cli.feature to pre-register local/file-reader in scenarios that use the full skill YAML (which includes local/file-reader) to reflect the new validation behavior. - Updated features/steps/skill_cli_coverage_r3_steps.py to pre-register local/new-include in the update-with-different-includes scenario to align with the updated validation flow. - Rationale and design decisions: - Validation is placed in the service layer (SkillService.add_skill()) rather than the CLI layer to ensure consistent enforcement regardless of how the service is invoked. - The error message reports all missing includes at once for better user guidance and faster remediation. - Tests that relied on the full skill YAML were updated to pre-register dependencies to accurately reflect the validation changes and maintain test integrity. ISSUES CLOSED: #2555
415 lines
22 KiB
Gherkin
415 lines
22 KiB
Gherkin
Feature: Skill CLI commands
|
|
As a developer
|
|
I want to manage skills via CLI commands
|
|
So that I can register, inspect, and manage reusable tool collections
|
|
|
|
Background:
|
|
Given a skill CLI test runner
|
|
And the skill service is reset
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill add — registration
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Add skill from valid config file
|
|
Given a valid skill config YAML file at a temp path
|
|
When I run skill CLI add with --config pointing to the YAML file
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should contain "Skill Registered"
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should contain "✓ OK"
|
|
|
|
Scenario: Add skill with all sections populated
|
|
Given a full skill config YAML file at a temp path
|
|
And the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI add with --config pointing to the YAML file
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should contain "Skill Registered"
|
|
And the skill CLI output should contain "Tool Sources"
|
|
|
|
Scenario: Add skill with duplicate name fails without --update
|
|
Given a valid skill config YAML file at a temp path
|
|
And the skill "local/file-reader" is already registered
|
|
When I run skill CLI add with --config pointing to the YAML file
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "already registered"
|
|
And the skill CLI output should contain "--update"
|
|
|
|
Scenario: Add skill with --update overwrites existing
|
|
Given a valid skill config YAML file at a temp path
|
|
And the skill "local/file-reader" is already registered
|
|
When I run skill CLI add with --config and --update pointing to the YAML file
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should contain "Skill Updated"
|
|
And the skill CLI output should contain "Changes"
|
|
|
|
Scenario: Add skill with missing config file fails
|
|
When I run skill CLI add with --config pointing to a missing file
|
|
Then the skill CLI command should abort
|
|
|
|
Scenario: Add skill with invalid YAML fails
|
|
Given an invalid skill YAML file at a temp path
|
|
When I run skill CLI add with --config pointing to the YAML file
|
|
Then the skill CLI command should abort
|
|
|
|
Scenario: Add skill with --format json produces valid JSON
|
|
Given a valid skill config YAML file at a temp path
|
|
When I run skill CLI add with --config and --format json
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should be valid JSON
|
|
|
|
Scenario: Add skill with --format yaml produces valid YAML
|
|
Given a valid skill config YAML file at a temp path
|
|
When I run skill CLI add with --config and --format yaml
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should be valid YAML
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill show — details
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Show skill displays all panels
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI show "local/file-reader"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "Skill Details"
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should contain "Capability Summary"
|
|
And the skill CLI output should contain "✓ OK"
|
|
|
|
Scenario: Show skill not found
|
|
When I run skill CLI show "local/nonexistent"
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "not found"
|
|
|
|
Scenario: Show skill with --format json
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI show "local/file-reader" with --format json
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should be valid JSON
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill tools — flattened tool list
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Tools command shows resolved tool list
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI tools "local/file-reader"
|
|
Then the skill CLI tools should succeed
|
|
And the skill CLI output should contain "Tools for local/file-reader"
|
|
And the skill CLI output should contain "Summary"
|
|
And the skill CLI output should contain "✓ OK"
|
|
|
|
Scenario: Tools command with includes shows source skill
|
|
Given a composed skill "local/composed" including "local/file-reader" is registered
|
|
When I run skill CLI tools "local/composed"
|
|
Then the skill CLI tools should succeed
|
|
And the skill CLI output should contain "local/file-re"
|
|
|
|
Scenario: Tools command for nonexistent skill fails
|
|
When I run skill CLI tools "local/nonexistent"
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "not found"
|
|
|
|
Scenario: Tools with --format json
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI tools "local/file-reader" with --format json
|
|
Then the skill CLI tools should succeed
|
|
And the skill CLI output should be valid JSON
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill list — listing and filtering
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: List shows all registered skills in table
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "local/git-ops" is registered with tools
|
|
When I run skill CLI list
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should contain "local/git-ops"
|
|
And the skill CLI output should contain "Summary"
|
|
|
|
Scenario: List with namespace filter
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "devops/deploy-tools" is registered with tools
|
|
When I run skill CLI list with --namespace "local"
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should not contain "devops/deploy-tools"
|
|
|
|
Scenario: List with --source mcp filter
|
|
Given a skill "local/mcp-backed" with MCP servers is registered
|
|
And the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI list with --source "mcp"
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should contain "local/mcp-backed"
|
|
And the skill CLI output should not contain "local/file-reader"
|
|
|
|
Scenario: List with no skills shows helpful message
|
|
When I run skill CLI list
|
|
Then the skill CLI output should contain "No skills found"
|
|
|
|
Scenario: List with --format json
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI list with --format json
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should be valid JSON
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill remove — removal
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Remove skill with --yes skips confirmation
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI remove "local/file-reader" with --yes
|
|
Then the skill CLI remove should succeed
|
|
And the skill CLI output should contain "Skill Removed"
|
|
And the skill CLI output should contain "✓ OK"
|
|
|
|
Scenario: Remove nonexistent skill fails
|
|
When I run skill CLI remove "local/nonexistent" with --yes
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "not found"
|
|
|
|
Scenario: Remove skill shows dependency check
|
|
Given a composed skill "local/composed" including "local/file-reader" is registered
|
|
When I run skill CLI remove "local/file-reader" with --yes
|
|
Then the skill CLI remove should succeed
|
|
And the skill CLI output should contain "Dependency Check"
|
|
And the skill CLI output should contain "local/composed"
|
|
|
|
Scenario: Remove skill with --format json
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI remove "local/file-reader" with --yes and --format json
|
|
Then the skill CLI remove should succeed
|
|
And the skill CLI output should be valid JSON
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Include cycle detection
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Tools command detects circular includes
|
|
Given a skill "local/cycle-a" including "local/cycle-b" is registered
|
|
And a skill "local/cycle-b" including "local/cycle-a" is registered
|
|
When I run skill CLI tools "local/cycle-a"
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "Cycle detected"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — show with rich panels
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Show skill with includes shows Includes panel
|
|
Given a composed skill "local/composed" including "local/file-reader" is registered
|
|
When I run skill CLI show "local/composed"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "Includes"
|
|
And the skill CLI output should contain "local/file-reader"
|
|
|
|
Scenario: Show skill with MCP servers shows MCP panel and tools table
|
|
Given a skill "local/mcp-detail" with MCP servers and tools is registered
|
|
When I run skill CLI show "local/mcp-detail"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "MCP Servers"
|
|
And the skill CLI output should contain "Direct Tools"
|
|
|
|
Scenario: Show skill with agent_skills and inline tools
|
|
Given a skill "local/mixed-sources" with agent_skills and inline tools is registered
|
|
When I run skill CLI show "local/mixed-sources"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "Direct Tools"
|
|
|
|
Scenario: Show skill with dependents shows Referenced By
|
|
Given a composed skill "local/parent" including "local/base-tools" is registered
|
|
When I run skill CLI show "local/base-tools"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "Referenced By"
|
|
And the skill CLI output should contain "local/parent"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — tools with varied source types
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Tools command with MCP and inline entries
|
|
Given a skill "local/mixed-tools" with MCP servers and inline tools is registered
|
|
When I run skill CLI tools "local/mixed-tools"
|
|
Then the skill CLI tools should succeed
|
|
And the skill CLI output should contain "mcp:"
|
|
And the skill CLI output should contain "custom"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — list with non-local namespace
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: List includes non-local namespace in summary
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "devops/deploy-tools" is registered with tools
|
|
When I run skill CLI list
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should contain "Server"
|
|
And the skill CLI output should contain "devops/deploy-tools"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — remove without --yes
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Remove skill without --yes prompts and aborts on deny
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI remove "local/file-reader" without --yes and deny
|
|
Then the skill CLI command should abort
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — update with dependent skills
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Update skill with dependents shows Affected Actors panel
|
|
Given a full skill config YAML file at a temp path
|
|
And the skill "local/file-reader" is registered with tools
|
|
And a composed skill "local/uses-full" including "local/full-skill" is registered
|
|
When I run skill CLI add with --config and --update pointing to the YAML file
|
|
Then the skill CLI add should succeed
|
|
And the skill CLI output should contain "Affected Actors"
|
|
And the skill CLI output should contain "local/uses-full"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Coverage — skill_service edge cases
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Service skill_count returns correct count
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "local/git-ops" is registered with tools
|
|
Then the skill service should report skill count 2
|
|
|
|
Scenario: Service add_skill without config_path
|
|
Given a skill is registered without a config path
|
|
Then the config path should be empty for the skill
|
|
|
|
Scenario: Service list_skills by source builtin
|
|
Given a skill "local/builtin-src" with only tool_refs is registered
|
|
When I list skills with source "builtin"
|
|
Then the skill list should contain "local/builtin-src"
|
|
|
|
Scenario: Service list_skills by source agent_skill
|
|
Given a skill "local/agent-src" with agent_skills is registered
|
|
When I list skills with source "agent_skill"
|
|
Then the skill list should contain "local/agent-src"
|
|
|
|
Scenario: Service remove_skill with empty name raises error
|
|
Then removing a skill with empty name should raise ValueError
|
|
|
|
Scenario: Service remove_skill with nonexistent name raises error
|
|
Then removing skill "local/phantom" should raise KeyError
|
|
|
|
Scenario: Service get_skill with empty name raises error
|
|
Then getting a skill with empty name should raise ValueError
|
|
|
|
Scenario: Add skill with no-description config fails validation
|
|
Given a skill config YAML with no description at a temp path
|
|
When I run skill CLI add with --config pointing to the YAML file
|
|
Then the skill CLI command should abort
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# skill refresh — recompute flattening and sync
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: Refresh single skill recomputes tool flattening
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI refresh "local/file-reader"
|
|
Then the skill CLI refresh should succeed
|
|
And the skill CLI output should contain "Skill Refreshed"
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should contain "Total Tools"
|
|
And the skill CLI output should contain "Read-Only"
|
|
And the skill CLI output should contain "✓ OK"
|
|
|
|
Scenario: Refresh single skill with --format json
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI refresh "local/file-reader" with --format json
|
|
Then the skill CLI refresh should succeed
|
|
And the skill CLI output should be valid JSON
|
|
And the skill CLI JSON output should have field "refreshed"
|
|
And the skill CLI JSON output should have field "skills"
|
|
|
|
Scenario: Refresh all skills with --all
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "local/git-ops" is registered with tools
|
|
When I run skill CLI refresh with --all
|
|
Then the skill CLI refresh should succeed
|
|
And the skill CLI output should contain "Skills Refreshed"
|
|
And the skill CLI output should contain "local/file-reader"
|
|
And the skill CLI output should contain "local/git-ops"
|
|
|
|
Scenario: Refresh with both name and --all fails
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI refresh "local/file-reader" with --all
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "Cannot specify both"
|
|
|
|
Scenario: Refresh without name or --all fails
|
|
When I run skill CLI refresh without arguments
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "Must specify either"
|
|
|
|
Scenario: Refresh nonexistent skill fails
|
|
When I run skill CLI refresh "local/nonexistent"
|
|
Then the skill CLI command should abort
|
|
And the skill CLI output should contain "not found"
|
|
|
|
Scenario: Refresh skill with MCP servers shows sync status
|
|
Given a skill "local/mcp-backed" with MCP servers is registered
|
|
When I run skill CLI refresh "local/mcp-backed"
|
|
Then the skill CLI refresh should succeed
|
|
And the skill CLI output should contain "MCP Servers"
|
|
|
|
Scenario: Refresh all with empty registry shows helpful message
|
|
When I run skill CLI refresh with --all
|
|
Then the skill CLI output should contain "No skills registered"
|
|
|
|
Scenario: Refresh all shows table with multiple skills
|
|
Given the skill "local/file-reader" is registered with tools
|
|
And the skill "local/git-ops" is registered with tools
|
|
When I run skill CLI refresh with --all
|
|
Then the skill CLI refresh should succeed
|
|
And the skill CLI output should contain "Skills Refreshed"
|
|
|
|
# ───────────────────────────────────────────────────────
|
|
# Enhanced outputs — capability summary in JSON/YAML
|
|
# ───────────────────────────────────────────────────────
|
|
|
|
Scenario: List with --format json includes capability summary
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI list with --format json
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should be valid JSON
|
|
And the skill CLI JSON output should have field "capability_summary" in first skill
|
|
|
|
Scenario: Show with --format json includes capability summary
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI show "local/file-reader" with --format json
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should be valid JSON
|
|
And the skill CLI JSON output should have field "capability_summary"
|
|
And the skill CLI JSON output should have field "tool_count"
|
|
|
|
Scenario: Tools with --format json includes capability summary
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI tools "local/file-reader" with --format json
|
|
Then the skill CLI tools should succeed
|
|
And the skill CLI output should be valid JSON
|
|
And the skill CLI JSON output should have field "capability_summary"
|
|
And the skill CLI JSON output should have field "skill_name"
|
|
And the skill CLI JSON output should have field "tools"
|
|
|
|
Scenario: List rich output shows Description column
|
|
Given the skill "local/file-reader" is registered with tools
|
|
When I run skill CLI list
|
|
Then the skill CLI list should succeed
|
|
And the skill CLI output should contain "Description"
|
|
|
|
Scenario: Show with includes displays tool count from includes
|
|
Given a composed skill "local/composed" including "local/file-reader" is registered
|
|
When I run skill CLI show "local/composed"
|
|
Then the skill CLI show should succeed
|
|
And the skill CLI output should contain "tools"
|