aditya
b25a886df8
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 20s
CI / lint (pull_request) Successful in 24s
CI / build (pull_request) Successful in 27s
CI / typecheck (pull_request) Successful in 34s
CI / security (pull_request) Successful in 1m11s
CI / integration_tests (pull_request) Successful in 4m40s
CI / unit_tests (pull_request) Successful in 9m55s
CI / docker (pull_request) Successful in 1m0s
CI / benchmark-regression (pull_request) Successful in 25m30s
CI / coverage (pull_request) Successful in 1h11m59s
Implement SkillRegistry.refresh(name) and refresh_all() to recompute flattened tool sets on demand. Wire MCPToolAdapter.dispatch_notification() to MCPRefreshHook, which debounces rapid notifications/tools/list_changed bursts into a single refresh_all() call (default 0.5 s window). Add safeguard that skips tool-ref validation when no ToolRegistry is configured, emitting a single WARNING with recovery steps. Introduce immutable SkillRefreshResult (refreshed/failed/skipped counts) with to_summary() and merge() for CLI and log output. New files: - src/cleveragents/skills/refresh.py (SkillRefreshResult) - src/cleveragents/mcp/refresh_hook.py (MCPRefreshHook) - features/skill_refresh.feature (19 Behave scenarios) - features/steps/skill_refresh_steps.py - robot/skill_refresh.robot (10 Robot tests) - robot/helper_skill_refresh.py - benchmarks/skill_refresh_bench.py (ASV benchmarks) - docs/reference/skill_refresh.md Modified: - src/cleveragents/skills/registry.py (refresh, refresh_all) - src/cleveragents/mcp/adapter.py (notification listener API) - src/cleveragents/skills/__init__.py (export SkillRefreshResult) - src/cleveragents/mcp/__init__.py (export MCPRefreshHook) - CHANGELOG.md ISSUES CLOSED #168
120 lines
6.1 KiB
Plaintext
120 lines
6.1 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for Skill Refresh hooks (feat #168)
|
|
... Covers SkillRefreshResult, SkillRegistry.refresh()/refresh_all(),
|
|
... and MCPRefreshHook notification-to-debounce-to-refresh wiring.
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_skill_refresh.py
|
|
|
|
*** Test Cases ***
|
|
Refresh Single Skill With All Tool Refs Present
|
|
[Documentation] refresh() returns skill in refreshed list when all tool refs resolve
|
|
${result}= Run Process ${PYTHON} ${HELPER} refresh-single-ok cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} refreshed-count: 1
|
|
Should Contain ${result.stdout} failed-count: 0
|
|
Should Contain ${result.stdout} name-in-refreshed: True
|
|
Should Contain ${result.stdout} refresh-single-ok
|
|
|
|
Refresh Single Skill With Missing Tool Ref
|
|
[Documentation] refresh() returns skill in failed dict when a tool ref is absent from ToolRegistry
|
|
${result}= Run Process ${PYTHON} ${HELPER} refresh-single-missing-tool cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} failed-count: 1
|
|
Should Contain ${result.stdout} has-error: True
|
|
Should Contain ${result.stdout} error-mentions-missing: True
|
|
Should Contain ${result.stdout} refresh-missing-tool-ok
|
|
|
|
Refresh Single Skill Without Tool Registry
|
|
[Documentation] refresh() skips validation and counts as refreshed when no ToolRegistry is configured
|
|
${result}= Run Process ${PYTHON} ${HELPER} refresh-single-no-registry cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} refreshed-count: 1
|
|
Should Contain ${result.stdout} failed-count: 0
|
|
Should Contain ${result.stdout} name-in-refreshed: True
|
|
Should Contain ${result.stdout} refresh-no-registry-ok
|
|
|
|
Refresh All Skills With Mixed Results
|
|
[Documentation] refresh_all() aggregates success and failure across all skills
|
|
${result}= Run Process ${PYTHON} ${HELPER} refresh-all-mixed cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} total-refreshed: 1
|
|
Should Contain ${result.stdout} total-failed: 1
|
|
Should Contain ${result.stdout} ok-refreshed: True
|
|
Should Contain ${result.stdout} bad-failed: True
|
|
Should Contain ${result.stdout} refresh-all-mixed-ok
|
|
|
|
Refresh Unknown Skill Raises Error
|
|
[Documentation] refresh() raises SkillExecutionError for an unregistered skill name
|
|
${result}= Run Process ${PYTHON} ${HELPER} refresh-unknown-skill cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} error-type: SkillExecutionError
|
|
Should Contain ${result.stdout} error-contains-name: True
|
|
Should Contain ${result.stdout} refresh-unknown-ok
|
|
|
|
Skill Refresh Result Summary Format
|
|
[Documentation] SkillRefreshResult.to_summary() produces the expected one-liner
|
|
${result}= Run Process ${PYTHON} ${HELPER} result-summary cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} has-refreshed: True
|
|
Should Contain ${result.stdout} has-failed: True
|
|
Should Contain ${result.stdout} has-skipped: True
|
|
Should Contain ${result.stdout} result-summary-ok
|
|
|
|
Skill Refresh Result Merge Combines Two Results
|
|
[Documentation] SkillRefreshResult.merge() joins refreshed, failed, and skipped lists correctly
|
|
${result}= Run Process ${PYTHON} ${HELPER} result-merge cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} total-refreshed: 2
|
|
Should Contain ${result.stdout} total-failed: 1
|
|
Should Contain ${result.stdout} total-skipped: 1
|
|
Should Contain ${result.stdout} a-in-refreshed: True
|
|
Should Contain ${result.stdout} c-in-refreshed: True
|
|
Should Contain ${result.stdout} b-in-skipped: True
|
|
Should Contain ${result.stdout} d-in-failed: True
|
|
Should Contain ${result.stdout} result-merge-ok
|
|
|
|
MCP Refresh Hook Triggers Skill Registry Refresh
|
|
[Documentation] MCPRefreshHook wires notifications/tools/list_changed to refresh_all()
|
|
${result}= Run Process ${PYTHON} ${HELPER} hook-wiring cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} refreshed-at-least-once: True
|
|
Should Contain ${result.stdout} hook-wiring-ok
|
|
|
|
MCP Refresh Hook Debounces Rapid Notifications
|
|
[Documentation] Multiple tool_list_changed notifications within the debounce window collapse to one refresh
|
|
${result}= Run Process ${PYTHON} ${HELPER} hook-debounce cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} debounced-to-one: True
|
|
Should Contain ${result.stdout} hook-debounce-ok
|
|
|
|
MCP Refresh Hook Cancel Prevents Pending Refresh
|
|
[Documentation] hook.cancel() stops the pending debounced refresh before it fires
|
|
${result}= Run Process ${PYTHON} ${HELPER} hook-cancel cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} no-refresh-after-cancel: True
|
|
Should Contain ${result.stdout} hook-cancel-ok
|