Add agent skills discovery system that scans configurable filesystem paths for Agent Skills Standard folders (containing SKILL.md with YAML front-matter) and registers them as tools in ToolRegistry. Core implementation: - New skills/discovery.py module with path parsing, directory scanning, SKILL.md front-matter parsing, ToolSpec building, and registration with configurable conflict handling (skip/error/replace strategies) - Config key skills.agent_skills_paths for comma-separated discovery paths - ToolSpec extended with source and source_metadata fields for provenance - ToolRegistry.list_tools() gains source filter parameter - SkillRegistryService rewritten with discover_and_register() and refresh_agent_skills() hooks for on-demand re-scanning - CLI "agents skill tools" command updated with --refresh flag and source metadata in JSON output Testing: - 27 Behave scenarios covering config key, path parsing, discovery, ToolSpec building, registration, conflict handling, refresh, edge cases (invalid YAML, empty front-matter, non-dict YAML, empty descriptions, noop handler), and source metadata - 10 Robot Framework integration tests for end-to-end discovery flow - ASV benchmarks for discovery scan performance overhead Documentation: - docs/reference/skill_registry.md updated with Agent Skills Discovery section covering config, algorithm, SKILL.md format, conflict handling, refresh hook, CLI output, and service API ISSUES CLOSED: #161
10 KiB
Skill Registry
The Skill Registry provides persistent storage for skill definitions in
CleverAgents. Skills are namespaced, reusable collections of tools
registered via YAML config and persisted to the skills and
skill_items database tables.
Registration Behavior
Skills are registered with a namespaced name (namespace/short_name)
as the unique identifier. The registry enforces:
- Name uniqueness: Duplicate names are rejected with
DuplicateSkillError. - Name validation: Names must match
^[a-zA-Z0-9_-]+/[a-zA-Z0-9_-]+$. - Cascading deletes: Removing a skill cascades to all child
skill_itemsrows.
Skill Items
Each skill stores its components as ordered items in the skill_items
table. The item_type discriminator distinguishes:
| item_type | Description |
|---|---|
tool_ref |
Reference to a named tool in the registry |
include |
Recursive inclusion of another skill |
inline_tool |
Anonymous tool defined inline |
mcp_source |
MCP server tool source |
agent_source |
Agent Skills Standard folder source |
Items are stored with a stable item_order to preserve definition
ordering across round-trips.
Filters
The SkillRegistryService.list_skills() method supports:
- Namespace filter: Pass
namespace="local"to list only skills in thelocalnamespace.
Service API
from cleveragents.application.services import SkillRegistryService
from cleveragents.infrastructure.database.repositories import SkillRepository
svc = SkillRegistryService(skill_repo=SkillRepository(session_factory))
# Register
svc.add_skill(skill)
# Retrieve
skill = svc.get_skill("local/code-tools")
# List with filter
skills = svc.list_skills(namespace="local")
# Update
svc.update_skill(updated_skill)
# Remove
svc.remove_skill("local/code-tools")
Agent Skills Discovery
The Skill Registry integrates with the Agent Skills Standard —
filesystem-based tool bundles identified by a SKILL.md file with
YAML front-matter.
Configuration
The skills.agent_skills_paths config key controls which directories
are scanned. Multiple paths are separated by commas:
# ~/.cleveragents/config.toml
[skills]
agent_skills_paths = "/home/user/.cleveragents/agent_skills,/opt/skills"
The default path is ~/.cleveragents/agent_skills.
Environment variable override:
CLEVERAGENTS_SKILLS_AGENT_SKILLS_PATHS.
Discovery Algorithm
- Parse the comma-separated
skills.agent_skills_pathsvalue. - For each directory, find immediate subdirectories containing a
SKILL.mdfile. - Parse the YAML front-matter (between
---fences) to extract the toolname,description, and optionalinput_schema/output_schema. - Construct
ToolSpecinstances withsource="agent_skills"andsource_metadatacontaining the filesystem path. - Register each tool in the
ToolRegistryunder theagent_skills/<name>namespace.
SKILL.md Format
---
name: my-tool
description: A custom agent skill
input_schema:
type: object
properties:
query:
type: string
---
# My Tool
Additional documentation here.
Conflict Handling
When an Agent Skill name collides with an existing tool in the
ToolRegistry, the discovery process records a DiscoveryConflict.
The caller decides the strategy:
| Strategy | Behavior |
|---|---|
skip |
Keep existing tool, log warning (default) |
error |
Raise ValueError with collision details |
replace |
Remove existing tool and register the new one |
Refresh Hook
The SkillRegistryService.refresh_agent_skills() method re-scans all
configured paths. It first removes previously registered agent skill
tools, then re-discovers and re-registers. The CLI agents skill tools
command accepts a --refresh flag to trigger this.
CLI Output
The agents skill tools command includes source metadata when tools
originate from Agent Skills:
agents skill tools local/devops-toolkit --format json
Each entry in the output includes a "source" field indicating the
tool's origin (builtin, agent_skills, mcp, or custom).
Service API (Discovery)
from cleveragents.application.services import SkillRegistryService
from cleveragents.infrastructure.database.repositories import SkillRepository
from cleveragents.tool.registry import ToolRegistry
tool_registry = ToolRegistry()
svc = SkillRegistryService(
skill_repo=SkillRepository(session_factory),
tool_registry=tool_registry,
)
# Initial discovery
result = svc.discover_and_register(
"~/.cleveragents/agent_skills,/opt/skills"
)
print(f"Discovered: {len(result.discovered)} skills")
print(f"Conflicts: {len(result.conflicts)}")
# Refresh (re-scan)
result = svc.refresh_agent_skills(
"~/.cleveragents/agent_skills,/opt/skills"
)
Database Schema
skills
| Column | Type | Description |
|---|---|---|
| name | String(255) | PK, namespaced name |
| namespace | String(100) | Extracted namespace |
| short_name | String(150) | Extracted short name |
| description | Text | Skill description |
| version | String(50) | Optional version string |
| metadata_json | Text | JSON metadata (overrides, etc.) |
| created_at | String(30) | ISO-8601 creation timestamp |
| updated_at | String(30) | ISO-8601 update timestamp |
skill_items
| Column | Type | Description |
|---|---|---|
| id | Integer | Auto-increment PK |
| skill_name | String(255) | FK to skills.name (CASCADE) |
| item_type | String(30) | Discriminator (tool_ref, etc.) |
| item_name | String(500) | Item identifier or name |
| item_config | Text | Optional JSON configuration |
| item_order | Integer | Stable ordering index |
| created_at | String(30) | ISO-8601 creation timestamp |
Flattening Rules
The SkillResolver flattens a skill's tool set using the following
deterministic algorithm:
Include Order
Tools are collected via depth-first traversal of includes:
- For each include (left-to-right), recurse into the included skill
first, processing its includes before its own
tool_refs. - After all includes are resolved, add the current skill's
tool_refsin definition order. - Add inline tools as
{skill_name}/_anon_{index}. - Add MCP tools as
mcp:{server}/{tool_name}. - Add agent skills as
agent_skill:{path}.
De-Duplication
When the same tool name appears multiple times (from overlapping includes), the first occurrence determines the position in the output list, but the last occurrence's entry metadata wins (last-wins semantics).
Override Merging
Overrides are applied at two levels:
- Skill-level overrides (
skill.overridesdict): Applied to matchingtool_refsduring resolution. - Per-include overrides (
SkillInclude.overrides): Applied to all tool entries contributed by the included skill. Uses shallow merge -- include overrides are merged on top of any existing entry overrides.
Non-Overridable Fields
The following ResolvedToolEntry fields cannot be overridden:
namesource_skillis_inline
Attempting to override these raises ValueError with the skill name
and include path for traceability.
Capability Summary
The SkillCapabilitySummary aggregates capability metadata across all
resolved tools:
| Field | Description |
|---|---|
total_tools |
Total number of resolved tool entries |
read_only_tools |
Count of inline tools with read_only=True |
write_tools |
Count of inline tools with writes=True |
checkpointable_tools |
Count of inline tools with checkpointable=True |
has_side_effects |
True if any inline tool has side effects |
mcp_sources |
Number of MCP server sources on the root skill |
agent_skill_sources |
Number of agent skill sources on the root skill |
validate_plan() Usage
The SkillRegistry.validate_plan() method checks that a plan's skill
references are satisfiable:
from cleveragents.skills.registry import SkillRegistry
registry = SkillRegistry()
# ... register skills ...
errors = registry.validate_plan({"skills": ["local/code-tools", "local/deploy"]})
if errors:
for err in errors:
print(f"Plan validation error: {err}")
Checks Performed
- All referenced skills exist in the registry.
- All includes are resolvable (no missing skills in include chains).
- No cycles in include chains.
- Tool refs are valid (if a tool registry is configured).
Error Messages
| Condition | Error message pattern |
|---|---|
| Missing skill | Skill '{name}' referenced in plan is not registered |
| Missing include | Skill '{name}': Included skill '{inc}' not found ... |
| Cycle detected | Skill '{name}': Cycle detected in skill includes: A -> B |
| Non-overridable field | Non-overridable field(s) ... in overrides from skill ... |
tools() Method
The SkillRegistry.tools() method combines resolve_tools() and
compute_capability_summary() into a single call:
entries, summary = registry.tools("local/code-tools")
print(f"Total tools: {summary.total_tools}")
print(f"Has writes: {summary.write_tools > 0}")
Returns a 2-tuple of (list[ResolvedToolEntry], SkillCapabilitySummary).