UAT: `agents skill refresh` does not actually refresh MCP server tools — MCP re-enumeration is a TODO stub #3597

New issue

Open

opened 2026-04-05 20:16:29 +00:00 by freemo · 0 comments

freemo commented

2026-04-05 20:16:29 +00:00

Owner

Background and Context

The specification defines that agents skill refresh should "recompute tool flattening and sync MCP-backed skills" — specifically re-enumerating available tools from MCP servers. This is a critical feature for skills that use MCP servers, as MCP servers can add/remove tools dynamically.

Current Behavior

In src/cleveragents/cli/commands/skill.py, the refresh command contains a TODO comment that explicitly acknowledges the MCP refresh is not implemented:

# For MCP skills, we would sync here if MCP adapter was available
# Currently, we just mark as refreshed
if has_mcp:
    # TODO: When MCP adapter integration is available, call
    # mcp_adapter.refresh_tools() for each server
    pass

The command:

Calls service.resolve_tools(skill.name) to recompute tool flattening (this works)
Does NOT call any MCP adapter to re-enumerate tools from MCP servers
Reports mcp_status: "synced" in the output even though no MCP sync occurred

This means that if an MCP server adds or removes tools, agents skill refresh will not pick up those changes — the skill's MCP tool list remains stale.

Expected Behavior

Per the specification, agents skill refresh should:

Re-enumerate tools from connected MCP servers via the MCP adapter
Update the skill's tool list with the new MCP tool set
Report accurate MCP sync status (success/failure per server)

The MCPToolAdapter class in src/cleveragents/mcp/adapter.py has a discover_tools() method that can re-enumerate tools. The SkillService needs to be wired to call this when refreshing MCP-backed skills.

Code Locations

src/cleveragents/cli/commands/skill.py — refresh() function (~line 1022–1025): TODO stub for MCP refresh
src/cleveragents/mcp/adapter.py — MCPToolAdapter.discover_tools(): the method that should be called
src/cleveragents/application/services/skill_service.py: needs a refresh_mcp_tools() method

Metadata

Branch: fix/skill-refresh-mcp-reenumeration
Commit Message: fix(skill): wire MCPToolAdapter.discover_tools() into agents skill refresh
Milestone: (none — backlog)
Parent Epic: #392

Subtasks

Write a failing Behave scenario in features/ demonstrating that agents skill refresh does not call MCPToolAdapter.discover_tools() (TDD — failing test first)
Add a Robot Framework integration test in robot/ that verifies MCP tool list is updated after agents skill refresh when an MCP server adds a new tool
Add refresh_mcp_tools(skill_name: str) -> dict[str, SkillRefreshResult] method to SkillService in src/cleveragents/application/services/skill_service.py
Wire SkillService.refresh_mcp_tools() into the refresh() command in src/cleveragents/cli/commands/skill.py, replacing the TODO stub
Ensure per-server error isolation: if one MCP server is unreachable, log the error and continue refreshing remaining servers
Update mcp_status output field to reflect actual per-server sync results (success/failure/unreachable) instead of always reporting "synced"
Add mock MCP adapter to features/mocks/ for unit test use
Run nox -e lint and resolve all issues
Run nox -e typecheck and resolve all Pyright errors (no # type: ignore suppressions)
Run nox -e unit_tests and confirm all Behave scenarios pass
Run nox -e integration_tests and confirm all Robot Framework tests pass
Run nox -e coverage_report and confirm coverage remains >= 97%
Run full nox suite and confirm all stages pass

Definition of Done

All subtasks above are checked off
agents skill refresh calls MCPToolAdapter.discover_tools() for each MCP server configured in the skill
The skill's MCP tool list is updated with newly discovered tools after refresh
mcp_status in the CLI output accurately reflects actual sync result per server (not a hardcoded "synced")
If an MCP server is unreachable, the error is reported per-server and remaining servers continue to be refreshed
A failing Behave unit test was written first (TDD), then made to pass
New Robot Framework integration test covers the end-to-end MCP refresh flow
All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e integration_tests, nox -e coverage_report)
Coverage >= 97%
Commit created with exact message: fix(skill): wire MCPToolAdapter.discover_tools() into agents skill refresh
Branch fix/skill-refresh-mcp-reenumeration pushed and PR merged

Backlog note: This issue was discovered during autonomous operation
on milestone v3.2.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Background and Context The specification defines that `agents skill refresh` should "recompute tool flattening and sync MCP-backed skills" — specifically re-enumerating available tools from MCP servers. This is a critical feature for skills that use MCP servers, as MCP servers can add/remove tools dynamically. ## Current Behavior In `src/cleveragents/cli/commands/skill.py`, the `refresh` command contains a TODO comment that explicitly acknowledges the MCP refresh is not implemented: ```python # For MCP skills, we would sync here if MCP adapter was available # Currently, we just mark as refreshed if has_mcp: # TODO: When MCP adapter integration is available, call # mcp_adapter.refresh_tools() for each server pass ``` The command: 1. Calls `service.resolve_tools(skill.name)` to recompute tool flattening (this works) 2. **Does NOT** call any MCP adapter to re-enumerate tools from MCP servers 3. Reports `mcp_status: "synced"` in the output even though no MCP sync occurred This means that if an MCP server adds or removes tools, `agents skill refresh` will not pick up those changes — the skill's MCP tool list remains stale. ## Expected Behavior Per the specification, `agents skill refresh` should: 1. Re-enumerate tools from connected MCP servers via the MCP adapter 2. Update the skill's tool list with the new MCP tool set 3. Report accurate MCP sync status (success/failure per server) The `MCPToolAdapter` class in `src/cleveragents/mcp/adapter.py` has a `discover_tools()` method that can re-enumerate tools. The `SkillService` needs to be wired to call this when refreshing MCP-backed skills. ## Code Locations - `src/cleveragents/cli/commands/skill.py` — `refresh()` function (~line 1022–1025): TODO stub for MCP refresh - `src/cleveragents/mcp/adapter.py` — `MCPToolAdapter.discover_tools()`: the method that should be called - `src/cleveragents/application/services/skill_service.py`: needs a `refresh_mcp_tools()` method --- ## Metadata - **Branch**: `fix/skill-refresh-mcp-reenumeration` - **Commit Message**: `fix(skill): wire MCPToolAdapter.discover_tools() into agents skill refresh` - **Milestone**: *(none — backlog)* - **Parent Epic**: #392 ## Subtasks - [ ] Write a failing Behave scenario in `features/` demonstrating that `agents skill refresh` does not call `MCPToolAdapter.discover_tools()` (TDD — failing test first) - [ ] Add a Robot Framework integration test in `robot/` that verifies MCP tool list is updated after `agents skill refresh` when an MCP server adds a new tool - [ ] Add `refresh_mcp_tools(skill_name: str) -> dict[str, SkillRefreshResult]` method to `SkillService` in `src/cleveragents/application/services/skill_service.py` - [ ] Wire `SkillService.refresh_mcp_tools()` into the `refresh()` command in `src/cleveragents/cli/commands/skill.py`, replacing the TODO stub - [ ] Ensure per-server error isolation: if one MCP server is unreachable, log the error and continue refreshing remaining servers - [ ] Update `mcp_status` output field to reflect actual per-server sync results (success/failure/unreachable) instead of always reporting `"synced"` - [ ] Add mock MCP adapter to `features/mocks/` for unit test use - [ ] Run `nox -e lint` and resolve all issues - [ ] Run `nox -e typecheck` and resolve all Pyright errors (no `# type: ignore` suppressions) - [ ] Run `nox -e unit_tests` and confirm all Behave scenarios pass - [ ] Run `nox -e integration_tests` and confirm all Robot Framework tests pass - [ ] Run `nox -e coverage_report` and confirm coverage remains >= 97% - [ ] Run full `nox` suite and confirm all stages pass ## Definition of Done - [ ] All subtasks above are checked off - [ ] `agents skill refresh` calls `MCPToolAdapter.discover_tools()` for each MCP server configured in the skill - [ ] The skill's MCP tool list is updated with newly discovered tools after refresh - [ ] `mcp_status` in the CLI output accurately reflects actual sync result per server (not a hardcoded `"synced"`) - [ ] If an MCP server is unreachable, the error is reported per-server and remaining servers continue to be refreshed - [ ] A failing Behave unit test was written first (TDD), then made to pass - [ ] New Robot Framework integration test covers the end-to-end MCP refresh flow - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`, `nox -e coverage_report`) - [ ] Coverage >= 97% - [ ] Commit created with exact message: `fix(skill): wire MCPToolAdapter.discover_tools() into agents skill refresh` - [ ] Branch `fix/skill-refresh-mcp-reenumeration` pushed and PR merged > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.2.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-05 20:16:34 +00:00

#392 Epic: Actor YAML & Compiler

freemo added this to the v3.7.0 milestone

2026-04-05 20:23:31 +00:00

freemo referenced this issue

2026-04-05 20:23:49 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:29:17 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue