feat(skill): persist flattened tool sets #166

Closed
opened 2026-02-22 23:39:46 +00:00 by freemo · 2 comments
Owner

Metadata

  • Commit Message: feat(skill): persist flattened tool sets
  • Branch: feature/m4-skill-registry-db

Background

Migration adds fields/tables for flattened_tools_json, includes_json, and capability_summary_json per skill. Original YAML text and a computed flattening hash support refresh invalidation. SkillRepository reads/writes flattened tool sets and invalidates cached summaries on update.

Acceptance Criteria

  • Add migration fields/tables to store flattened_tools_json, includes_json, and capability_summary_json for each skill.
  • Persist original skill YAML text and computed flattening hash for refresh invalidation.
  • Extend SkillRepository to read/write flattened tool sets and invalidate cached summaries on update.
  • Add uniqueness constraint for skill names and index on namespace for list filters.
  • Add repository method to recompute flattening hash on update and store in DB.

Definition of Done

This issue is complete when:

  • All subtasks below are completed and checked off.
  • A Git commit is created where the first line of the commit message matches
    the Commit Message in Metadata exactly, followed by a blank line, then
    additional lines providing relevant details about the implementation. The
    commit body should be appropriate in size for a commit message and relatively
    complete in describing what was done.
  • The commit is pushed to the remote on the branch matching the Branch in
    Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and
    merged before this issue is marked done.

Subtasks

  • Add migration fields/tables to store flattened_tools_json, includes_json, and capability_summary_json for each skill.
  • Persist original skill YAML text and computed flattening hash for refresh invalidation.
  • Extend SkillRepository to read/write flattened tool sets and invalidate cached summaries on update.
  • Add uniqueness constraint for skill names and index on namespace for list filters.
  • Add repository method to recompute flattening hash on update and store in DB.
  • Update docs/reference/database_schema.md with skill registry persistence fields.
  • Add mapping table of persistence fields to domain model properties.
  • Tests (Behave): Add features/skill_registry_persistence.feature scenarios for add/update/refresh persistence.
  • Tests (Robot): Add repository persistence smoke tests for skills.
  • Tests (ASV): Add benchmarks/skill_registry_persist_bench.py for persistence overhead.
  • Verify coverage >=97% via nox -s coverage_report. If coverage is <97% then review the current unit test coverage report at build/coverage.xml and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun nox -s coverage_report to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%.
  • Run nox (all default sessions, including benchmark), fix any errors if needed ensuring nox passes across entire code base, do not ignore any failure even if it seems unrelated to this commit, fix it.

Section: #### M2: Actor Graphs + Tool Sources (Day 14)
Status: Open

## Metadata - **Commit Message**: `feat(skill): persist flattened tool sets` - **Branch**: `feature/m4-skill-registry-db` ## Background Migration adds fields/tables for `flattened_tools_json`, `includes_json`, and `capability_summary_json` per skill. Original YAML text and a computed flattening hash support refresh invalidation. `SkillRepository` reads/writes flattened tool sets and invalidates cached summaries on update. ## Acceptance Criteria - [ ] Add migration fields/tables to store `flattened_tools_json`, `includes_json`, and `capability_summary_json` for each skill. - [ ] Persist original skill YAML text and computed flattening hash for refresh invalidation. - [ ] Extend SkillRepository to read/write flattened tool sets and invalidate cached summaries on update. - [ ] Add uniqueness constraint for skill names and index on namespace for list filters. - [ ] Add repository method to recompute flattening hash on update and store in DB. ## Definition of Done This issue is complete when: - All subtasks below are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. The commit body should be appropriate in size for a commit message and relatively complete in describing what was done. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. ## Subtasks - [ ] Add migration fields/tables to store `flattened_tools_json`, `includes_json`, and `capability_summary_json` for each skill. - [ ] Persist original skill YAML text and computed flattening hash for refresh invalidation. - [ ] Extend SkillRepository to read/write flattened tool sets and invalidate cached summaries on update. - [ ] Add uniqueness constraint for skill names and index on namespace for list filters. - [ ] Add repository method to recompute flattening hash on update and store in DB. - [ ] Update `docs/reference/database_schema.md` with skill registry persistence fields. - [ ] Add mapping table of persistence fields to domain model properties. - [ ] Tests (Behave): Add `features/skill_registry_persistence.feature` scenarios for add/update/refresh persistence. - [ ] Tests (Robot): Add repository persistence smoke tests for skills. - [ ] Tests (ASV): Add `benchmarks/skill_registry_persist_bench.py` for persistence overhead. - [ ] Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun `nox -s coverage_report` to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%. - [ ] Run `nox` (all default sessions, including benchmark), fix any errors if needed ensuring nox passes across **entire** code base, do not ignore any failure even if it seems unrelated to this commit, fix it. **Section**: #### M2: Actor Graphs + Tool Sources (Day 14) **Status**: Open
freemo added this to the v3.1.0 milestone 2026-02-22 23:39:46 +00:00
Author
Owner

Expected completion (Day 15 rebaseline): Day 26 / 2026-03-06

**Expected completion (Day 15 rebaseline):** Day 26 / 2026-03-06
freemo added the due date 2026-02-25 2026-02-23 18:41:25 +00:00
freemo self-assigned this 2026-02-24 21:53:07 +00:00
Author
Owner

Implementation Notes (Day 19 — 2026-02-27)

PR #460: feat(skill): persist flattened tool sets

Branch: feature/m4-skill-registry-db
Commit: 515e54991e0acf25b19862267a367702e34ba4ab

Changes (9 files, +995 lines)

  1. Alembic Migration (m4_002_skill_flattened_tools):

    • 5 new columns: flattened_tools_json, includes_json, capability_summary_json, yaml_text, flattening_hash
    • Uniqueness constraint on skill name, SQLite-compatible batch_alter_table
  2. SkillModel: 5 new Column definitions + UniqueConstraint

  3. SkillRepository Extensions (5 new methods):

    • update_flattened_tools() — bulk cache persistence
    • get_flattened_tools() — cached field retrieval
    • needs_refresh() — hash comparison for staleness detection
    • recompute_flattening_hash() — SHA-256 computation + storage
    • invalidate_cached_summaries() — cache nullification
    • update() now auto-invalidates cache on mutation
  4. Documentation: Updated docs/reference/database_schema.md with field mapping table

Tests

  • Behave: 6 scenarios — creation persistence, update invalidation, hash staleness, refresh, uniqueness, namespace index
  • Robot: 2 smoke tests — round-trip persistence, update invalidation
  • ASV: 3 benchmarks — persist, refresh check, namespace listing

Quality Gates: All PASS (lint, typecheck, unit_tests, integration_tests)

## Implementation Notes (Day 19 — 2026-02-27) ### PR #460: `feat(skill): persist flattened tool sets` **Branch:** `feature/m4-skill-registry-db` **Commit:** `515e54991e0acf25b19862267a367702e34ba4ab` ### Changes (9 files, +995 lines) 1. **Alembic Migration** (`m4_002_skill_flattened_tools`): - 5 new columns: `flattened_tools_json`, `includes_json`, `capability_summary_json`, `yaml_text`, `flattening_hash` - Uniqueness constraint on skill name, SQLite-compatible `batch_alter_table` 2. **SkillModel**: 5 new Column definitions + UniqueConstraint 3. **SkillRepository Extensions** (5 new methods): - `update_flattened_tools()` — bulk cache persistence - `get_flattened_tools()` — cached field retrieval - `needs_refresh()` — hash comparison for staleness detection - `recompute_flattening_hash()` — SHA-256 computation + storage - `invalidate_cached_summaries()` — cache nullification - `update()` now auto-invalidates cache on mutation 4. **Documentation**: Updated `docs/reference/database_schema.md` with field mapping table ### Tests - **Behave**: 6 scenarios — creation persistence, update invalidation, hash staleness, refresh, uniqueness, namespace index - **Robot**: 2 smoke tests — round-trip persistence, update invalidation - **ASV**: 3 benchmarks — persist, refresh check, namespace listing ### Quality Gates: All PASS (lint, typecheck, unit_tests, integration_tests)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

2026-02-25

Blocks
#393 Epic: Skill & Tool Lifecycle
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#166
No description provided.