freemo
b2e923173f
CI / lint (pull_request) Successful in 18s
CI / typecheck (pull_request) Successful in 32s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 33s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / unit_tests (pull_request) Successful in 2m5s
CI / integration_tests (pull_request) Successful in 2m50s
CI / docker (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 4m19s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 18s
CI / security (push) Successful in 30s
CI / build (push) Successful in 22s
CI / typecheck (push) Successful in 58s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m6s
CI / docker (push) Successful in 50s
CI / integration_tests (push) Successful in 3m21s
CI / coverage (push) Successful in 4m12s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (pull_request) Successful in 24m46s
Consolidate 141 BDD feature files that each complete in under 0.1 seconds into 25 domain-grouped feature files, reducing subprocess count from 339 to ~223. Each consolidated file groups scenarios from the same domain/module that share step definitions and fixtures. All scenarios are preserved with clear comment headers indicating their original source file. This reduces subprocess overhead by ~116 invocations (141 original files replaced by 25 consolidated files), targeting the 42% of subprocess count that contributed only 0.2% of actual test runtime. ISSUES CLOSED: #485
169 lines
9.0 KiB
Gherkin
169 lines
9.0 KiB
Gherkin
Feature: Skill Registry Refresh and MCP Notification Hooks
|
|
As a CleverAgents runtime operator
|
|
I want the SkillRegistry to refresh tool sets on demand and in response to
|
|
MCP notifications/tools/list_changed events
|
|
So that skills always reflect the current state of connected MCP servers
|
|
|
|
# -------------------------------------------------------------------
|
|
# SkillRefreshResult
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Refresh result summary contains counts for refreshed skills
|
|
Given a skill refresh result with 3 refreshed, 1 failed, 2 skipped
|
|
Then the refresh result total_refreshed should be 3
|
|
And the refresh result total_failed should be 1
|
|
And the refresh result total_skipped should be 2
|
|
And the refresh result summary should be "refreshed: 3, failed: 1, skipped: 2"
|
|
|
|
Scenario: Refresh result for all-success is correct
|
|
Given a skill refresh result with 5 refreshed, 0 failed, 0 skipped
|
|
Then the refresh result total_refreshed should be 5
|
|
And the refresh result total_failed should be 0
|
|
And the refresh result total_skipped should be 0
|
|
And the refresh result summary should be "refreshed: 5, failed: 0, skipped: 0"
|
|
|
|
Scenario: Refresh result for all-skipped is correct
|
|
Given a skill refresh result with 0 refreshed, 0 failed, 3 skipped
|
|
Then the refresh result total_skipped should be 3
|
|
And the refresh result summary should be "refreshed: 0, failed: 0, skipped: 3"
|
|
|
|
# -------------------------------------------------------------------
|
|
# SkillRegistry.refresh(name) — happy path
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Refresh a single skill recomputes its tool set
|
|
Given a skill registry without a tool registry
|
|
And a registered skill "local/my-skill" with tool refs "local/tool-a,local/tool-b"
|
|
When I refresh skill "local/my-skill"
|
|
Then the refresh result total_refreshed should be 1
|
|
And the refresh result total_failed should be 0
|
|
And the refresh result total_skipped should be 0
|
|
|
|
Scenario: Refresh a skill with a configured tool registry validates tool refs
|
|
Given a skill registry with a tool registry containing "local/tool-a"
|
|
And a registered skill "local/my-skill" with tool refs "local/tool-a"
|
|
When I refresh skill "local/my-skill"
|
|
Then the refresh result total_refreshed should be 1
|
|
And the refresh result total_failed should be 0
|
|
|
|
Scenario: Refresh a skill whose tool ref is missing from tool registry returns failure
|
|
Given a skill registry with a tool registry containing "local/tool-a"
|
|
And a registered skill "local/bad-skill" with tool refs "local/missing-tool"
|
|
When I refresh skill "local/bad-skill"
|
|
Then the refresh result total_failed should be 1
|
|
And the refresh result failed skill "local/bad-skill" error mentions "local/missing-tool"
|
|
|
|
# -------------------------------------------------------------------
|
|
# SkillRegistry.refresh(name) — safeguard: no tool registry
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Refresh skips validation when tool registry is unavailable
|
|
Given a skill registry without a tool registry
|
|
And a registered skill "local/unvalidated-skill" with tool refs "local/any-tool"
|
|
When I refresh skill "local/unvalidated-skill"
|
|
Then the refresh result total_skipped should be 0
|
|
And the refresh result total_refreshed should be 1
|
|
|
|
Scenario: Refresh emits a warning when tool registry is unavailable
|
|
Given a skill registry without a tool registry
|
|
And a registered skill "local/warn-skill" with tool refs "local/some-tool"
|
|
When I refresh skill "local/warn-skill" and capture log warnings
|
|
Then no "tool registry" warning is logged during refresh without tool registry
|
|
|
|
# -------------------------------------------------------------------
|
|
# SkillRegistry.refresh(name) — error cases
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Refresh a non-existent skill raises SkillExecutionError
|
|
Given an empty skill registry without a tool registry
|
|
When I try to refresh non-existent skill "local/ghost-skill"
|
|
Then a SkillExecutionError is raised mentioning "local/ghost-skill"
|
|
|
|
# -------------------------------------------------------------------
|
|
# SkillRegistry.refresh_all()
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: refresh_all refreshes all registered skills
|
|
Given a skill registry without a tool registry
|
|
And a registered skill "local/skill-a" with tool refs "local/tool-x"
|
|
And a registered skill "local/skill-b" with tool refs "local/tool-y"
|
|
When I call refresh_all on the skill registry
|
|
Then the refresh result total_refreshed should be 2
|
|
And the refresh result total_failed should be 0
|
|
|
|
Scenario: refresh_all with tool registry validates all skills
|
|
Given a skill registry with a tool registry containing "local/tool-x"
|
|
And a registered skill "local/good-skill" with tool refs "local/tool-x"
|
|
And a registered skill "local/bad-skill" with tool refs "local/missing-tool"
|
|
When I call refresh_all on the skill registry
|
|
Then the refresh result total_refreshed should be 1
|
|
And the refresh result total_failed should be 1
|
|
|
|
Scenario: refresh_all on empty registry returns zero counts
|
|
Given an empty skill registry without a tool registry
|
|
When I call refresh_all on the skill registry
|
|
Then the refresh result total_refreshed should be 0
|
|
And the refresh result total_failed should be 0
|
|
And the refresh result total_skipped should be 0
|
|
|
|
# -------------------------------------------------------------------
|
|
# MCPRefreshHook — notification wiring
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: MCPToolAdapter dispatches registered notification listeners
|
|
Given a fresh MCP adapter for notification testing
|
|
And a notification listener registered on the adapter
|
|
When the adapter dispatches "notifications/tools/list_changed" notification
|
|
Then the notification listener should have been called with method "notifications/tools/list_changed"
|
|
|
|
Scenario: MCPToolAdapter ignores unknown notification methods
|
|
Given a fresh MCP adapter for notification testing
|
|
And a notification listener registered on the adapter
|
|
When the adapter dispatches "notifications/unknown" notification
|
|
Then the notification listener should have been called with method "notifications/unknown"
|
|
|
|
Scenario: MCPRefreshHook wires adapter notifications to skill registry refresh_all
|
|
Given a fresh MCP adapter for notification testing
|
|
And a skill registry without a tool registry
|
|
And a registered skill "local/wired-skill" with tool refs "local/tool-z"
|
|
And an MCPRefreshHook connecting the adapter to the skill registry with debounce 0.0
|
|
When the adapter dispatches "notifications/tools/list_changed" notification
|
|
And I wait for 0.1 seconds for debounce to fire
|
|
Then the skill registry refresh_all should have been triggered
|
|
|
|
Scenario: Multiple notifications within debounce window collapse to one refresh
|
|
Given a fresh MCP adapter for notification testing
|
|
And a skill registry without a tool registry
|
|
And a registered skill "local/debounce-skill" with tool refs "local/tool-z"
|
|
And an MCPRefreshHook connecting the adapter to the skill registry with debounce 2.0
|
|
When the adapter dispatches "notifications/tools/list_changed" notification
|
|
And the adapter dispatches "notifications/tools/list_changed" notification
|
|
And the adapter dispatches "notifications/tools/list_changed" notification
|
|
And I wait for 3.0 seconds for debounce to fire
|
|
Then the skill registry refresh count should be 1
|
|
|
|
Scenario: Non-tools-list-changed notifications are ignored by MCPRefreshHook
|
|
Given a fresh MCP adapter for notification testing
|
|
And a skill registry without a tool registry
|
|
And an MCPRefreshHook connecting the adapter to the skill registry with debounce 0.0
|
|
When the adapter dispatches "notifications/resources/list_changed" notification
|
|
And I wait for 0.1 seconds for debounce to fire
|
|
Then the skill registry refresh count should be 0
|
|
|
|
Scenario: MCPRefreshHook cancel stops pending debounced refresh
|
|
Given a fresh MCP adapter for notification testing
|
|
And a skill registry without a tool registry
|
|
And an MCPRefreshHook connecting the adapter to the skill registry with debounce 0.3
|
|
When the adapter dispatches "notifications/tools/list_changed" notification
|
|
And I cancel the MCPRefreshHook immediately
|
|
And I wait for 0.5 seconds for debounce to fire
|
|
Then the skill registry refresh count should be 0
|
|
|
|
# -------------------------------------------------------------------
|
|
# Refresh result summary for CLI output
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: to_summary renders human-readable output for logging
|
|
Given a skill refresh result with 2 refreshed, 1 failed, 0 skipped
|
|
Then the refresh result summary should be "refreshed: 2, failed: 1, skipped: 0"
|