cleveragents-core/docs/reference/skill_registry.md

# Skill Registry

The Skill Registry provides persistent storage for skill definitions in
CleverAgents. Skills are namespaced, reusable collections of tools
registered via YAML config and persisted to the `skills` and
`skill_items` database tables.

## Registration Behavior

Skills are registered with a namespaced name (`namespace/short_name`)
as the unique identifier. The registry enforces:

- **Name uniqueness**: Duplicate names are rejected with
  `DuplicateSkillError`.
- **Name validation**: Names must match `^[a-zA-Z0-9_-]+/[a-zA-Z0-9_-]+$`.
- **Cascading deletes**: Removing a skill cascades to all child
  `skill_items` rows.

## Skill Items

Each skill stores its components as ordered items in the `skill_items`
table. The `item_type` discriminator distinguishes:

| item_type      | Description                              |
|----------------|------------------------------------------|
| `tool_ref`     | Reference to a named tool in the registry |
| `include`      | Recursive inclusion of another skill      |
| `inline_tool`  | Anonymous tool defined inline             |
| `mcp_source`   | MCP server tool source                    |
| `agent_source` | Agent Skills Standard folder source       |

Items are stored with a stable `item_order` to preserve definition
ordering across round-trips.

## Filters

The `SkillRegistryService.list_skills()` method supports:

- **Namespace filter**: Pass `namespace="local"` to list only skills
  in the `local` namespace.

## Service API

```python
from cleveragents.application.services import SkillRegistryService
from cleveragents.infrastructure.database.repositories import SkillRepository

svc = SkillRegistryService(skill_repo=SkillRepository(session_factory))

# Register
svc.add_skill(skill)

# Retrieve
skill = svc.get_skill("local/code-tools")

# List with filter
skills = svc.list_skills(namespace="local")

# Update
svc.update_skill(updated_skill)

# Remove
svc.remove_skill("local/code-tools")
```

## Agent Skills Discovery

The Skill Registry integrates with the **Agent Skills Standard** —
filesystem-based tool bundles identified by a `SKILL.md` file with
YAML front-matter.

### Configuration

The `skills.agent_skills_paths` config key controls which directories
are scanned. Multiple paths are separated by commas:

```toml
# ~/.cleveragents/config.toml
[skills]
agent_skills_paths = "/home/user/.cleveragents/agent_skills,/opt/skills"
```

The default path is `~/.cleveragents/agent_skills`.

Environment variable override:
`CLEVERAGENTS_SKILLS_AGENT_SKILLS_PATHS`.

### Discovery Algorithm

1. Parse the comma-separated `skills.agent_skills_paths` value.
2. For each directory, find immediate subdirectories containing a
   `SKILL.md` file.
3. Parse the YAML front-matter (between `---` fences) to extract the
   tool `name`, `description`, and optional `input_schema` / `output_schema`.
4. Construct `ToolSpec` instances with `source="agent_skills"` and
   `source_metadata` containing the filesystem path.
5. Register each tool in the `ToolRegistry` under the
   `agent_skills/<name>` namespace.

### SKILL.md Format

```markdown
---
name: my-tool
description: A custom agent skill
input_schema:
  type: object
  properties:
    query:
      type: string
---

# My Tool

Additional documentation here.
```

### Conflict Handling

When an Agent Skill name collides with an existing tool in the
`ToolRegistry`, the discovery process records a `DiscoveryConflict`.
The caller decides the strategy:

| Strategy   | Behavior                                          |
|------------|---------------------------------------------------|
| `skip`     | Keep existing tool, log warning (default)         |
| `error`    | Raise `ValueError` with collision details         |
| `replace`  | Remove existing tool and register the new one     |

### Refresh Hook

The `SkillRegistryService.refresh_agent_skills()` method re-scans all
configured paths. It first removes previously registered agent skill
tools, then re-discovers and re-registers. The CLI `agents skill tools`
command accepts a `--refresh` flag to trigger this.

### CLI Output

The `agents skill tools` command includes source metadata when tools
originate from Agent Skills:

```bash
agents skill tools local/devops-toolkit --format json
```

Each entry in the output includes a `"source"` field indicating the
tool's origin (`builtin`, `agent_skills`, `mcp`, or `custom`).

### Service API (Discovery)

```python
from cleveragents.application.services import SkillRegistryService
from cleveragents.infrastructure.database.repositories import SkillRepository
from cleveragents.tool.registry import ToolRegistry

tool_registry = ToolRegistry()
svc = SkillRegistryService(
    skill_repo=SkillRepository(session_factory),
    tool_registry=tool_registry,
)

# Initial discovery
result = svc.discover_and_register(
    "~/.cleveragents/agent_skills,/opt/skills"
)
print(f"Discovered: {len(result.discovered)} skills")
print(f"Conflicts: {len(result.conflicts)}")

# Refresh (re-scan)
result = svc.refresh_agent_skills(
    "~/.cleveragents/agent_skills,/opt/skills"
)
```

## Database Schema

### skills

| Column         | Type         | Description                     |
|----------------|--------------|---------------------------------|
| name           | String(255)  | PK, namespaced name             |
| namespace      | String(100)  | Extracted namespace              |
| short_name     | String(150)  | Extracted short name             |
| description    | Text         | Skill description                |
| version        | String(50)   | Optional version string          |
| metadata_json  | Text         | JSON metadata (overrides, etc.)  |
| created_at     | String(30)   | ISO-8601 creation timestamp      |
| updated_at     | String(30)   | ISO-8601 update timestamp        |

### skill_items

| Column      | Type         | Description                          |
|-------------|--------------|--------------------------------------|
| id          | Integer      | Auto-increment PK                    |
| skill_name  | String(255)  | FK to skills.name (CASCADE)          |
| item_type   | String(30)   | Discriminator (tool_ref, etc.)       |
| item_name   | String(500)  | Item identifier or name              |
| item_config | Text         | Optional JSON configuration          |
| item_order  | Integer      | Stable ordering index                |
| created_at  | String(30)   | ISO-8601 creation timestamp          |

## Flattening Rules

The `SkillResolver` flattens a skill's tool set using the following
deterministic algorithm:

### Include Order

Tools are collected via **depth-first traversal** of includes:

1. For each include (left-to-right), recurse into the included skill
   first, processing *its* includes before its own `tool_refs`.
2. After all includes are resolved, add the current skill's `tool_refs`
   in definition order.
3. Add inline tools as `{skill_name}/_anon_{index}`.
4. Add MCP tools as `mcp:{server}/{tool_name}`.
5. Add agent skills as `agent_skill:{path}`.

### De-Duplication

When the same tool name appears multiple times (from overlapping
includes), the **first occurrence** determines the position in the
output list, but the **last occurrence's entry metadata wins**
(last-wins semantics).

### Override Merging

Overrides are applied at two levels:

1. **Skill-level overrides** (`skill.overrides` dict): Applied to
   matching `tool_refs` during resolution.
2. **Per-include overrides** (`SkillInclude.overrides`): Applied to
   *all* tool entries contributed by the included skill. Uses
   **shallow merge** -- include overrides are merged on top of any
   existing entry overrides.

### Non-Overridable Fields

The following `ResolvedToolEntry` fields cannot be overridden:

- `name`
- `source_skill`
- `is_inline`

Attempting to override these raises `ValueError` with the skill name
and include path for traceability.

## Capability Summary

The `SkillCapabilitySummary` aggregates capability metadata across all
resolved tools:

| Field                 | Description                                     |
|-----------------------|-------------------------------------------------|
| `total_tools`         | Total number of resolved tool entries            |
| `read_only_tools`     | Count of inline tools with `read_only=True`      |
| `write_tools`         | Count of inline tools with `writes=True`         |
| `checkpointable_tools`| Count of inline tools with `checkpointable=True` |
| `has_side_effects`    | `True` if any inline tool has side effects       |
| `mcp_sources`         | Number of MCP server sources on the root skill   |
| `agent_skill_sources` | Number of agent skill sources on the root skill  |

## `validate_plan()` Usage

The `SkillRegistry.validate_plan()` method checks that a plan's skill
references are satisfiable:

```python
from cleveragents.skills.registry import SkillRegistry

registry = SkillRegistry()
# ... register skills ...

errors = registry.validate_plan({"skills": ["local/code-tools", "local/deploy"]})
if errors:
    for err in errors:
        print(f"Plan validation error: {err}")
```

### Checks Performed

1. All referenced skills exist in the registry.
2. All includes are resolvable (no missing skills in include chains).
3. No cycles in include chains.
4. Tool refs are valid (if a tool registry is configured).

### Error Messages

| Condition              | Error message pattern                                        |
|------------------------|--------------------------------------------------------------|
| Missing skill          | `Skill '{name}' referenced in plan is not registered`        |
| Missing include        | `Skill '{name}': Included skill '{inc}' not found ...`       |
| Cycle detected         | `Skill '{name}': Cycle detected in skill includes: A -> B`   |
| Non-overridable field  | `Non-overridable field(s) ... in overrides from skill ...`   |

## `tools()` Method

The `SkillRegistry.tools()` method combines `resolve_tools()` and
`compute_capability_summary()` into a single call:

```python
entries, summary = registry.tools("local/code-tools")
print(f"Total tools: {summary.total_tools}")
print(f"Has writes: {summary.write_tools > 0}")
```

Returns a 2-tuple of `(list[ResolvedToolEntry], SkillCapabilitySummary)`.

## Testing

### Behave BDD Tests

The Behave feature file `features/skill_registry.feature` contains 23
scenarios covering the full `SkillRegistryService` and `SkillRepository`
persistence layer:

| Area | Scenarios | Description |
|------|-----------|-------------|
| Registration | 6 | Each skill item type (`tool_ref`, `include`, `inline_tool`, `mcp_source`, `agent_source`) plus basic round-trip |
| Duplicate rejection | 1 | `DuplicateSkillError` on name collision |
| Retrieval | 2 | Get by name, non-existent returns `None` |
| Listing | 2 | List all, list with namespace filter |
| Update | 2 | Description change, non-existent raises `SkillNotFoundError` |
| Deletion | 2 | Remove existing (cascades items), remove non-existent returns `False` |
| Overrides | 1 | Override metadata survives persistence round-trip |
| Item ordering | 1 | Mixed item types maintain stable order across round-trip |
| Name validation | 3 | Empty name, no-slash name, special characters rejected at persistence layer |
| Large payload | 1 | 50 tool refs persist correctly |
| Domain objects | 1 | Listed skills are `Skill` domain instances |

Step definitions: `features/steps/skill_registry_steps.py`

**Session management note:** Each scenario creates a fresh in-memory
SQLite database with a single shared `Session` that is reused across all
repository calls within the scenario.  This mirrors the production
`UnitOfWork` pattern (ADR-019) and avoids transaction-scoping issues
inherent to multiple ephemeral sessions on a single `StaticPool`
connection.

Run the Behave suite via:

```bash
nox -s unit_tests -- features/skill_registry.feature
```

### Robot Smoke Tests

A Robot Framework smoke suite validates core registry operations:

```
robot/skill_registry.robot
robot/helper_skill_registry.py
```

| Test Case | What it validates |
|---|---|
| Register And Retrieve A Skill | Round-trip: register then get by name |
| List Skills With Namespace Filter | Register multiple skills, filter by namespace |
| Update A Skill | Change description after registration |
| Reject Duplicate Skill Name | Duplicate name produces error |
| Remove A Skill | Remove and verify absence |

```bash
nox -s integration_tests -- robot/skill_registry.robot
```

### ASV Benchmarks

Performance benchmarks live in `benchmarks/skill_registry_bench.py`:

- **`SkillModelConstruction`** -- `time_from_domain` (domain-to-ORM mapping)
- **`SkillModelReconstruction`** -- `time_to_domain` (ORM-to-domain mapping)
- **`SkillRepositoryCRUD`** -- `time_create_skill`, `time_get_skill`,
  `time_list_all`, `time_list_namespace`
- **`SkillRegistryListPerformance`** -- `time_list_100_skills`,
  `time_list_100_skills_filtered`

```bash
nox -s benchmark
```