UAT: `agents skill refresh` does not actually refresh MCP tools — stub implementation with TODO comment #2108

New issue

Open

opened 2026-04-03 04:08:34 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 04:08:34 +00:00

Owner

Metadata

Branch: fix/skill-refresh-mcp-tools-stub
Commit Message: fix(skill): implement MCP tool re-enumeration in agents skill refresh command
Milestone: v3.7.0
Parent Epic: #936

Summary

The agents skill refresh command claims to "Recompute tool flattening and sync MCP-backed skills" but the MCP refresh portion is a stub with a TODO comment. MCP tools are never actually re-enumerated from their servers. The command reports mcp_status: "synced" even though no actual MCP server communication occurs, misleading users into believing their MCP tools have been refreshed when they have not.

Bug Details

What was tested: Code-level analysis of the agents skill refresh command implementation in src/cleveragents/cli/commands/skill.py.

Expected behavior (from spec): The agents skill refresh command should, for MCP-backed skills, re-enumerate available tools from MCP servers by calling mcp_adapter.refresh_tools() for each configured MCP server. The spec states: "For MCP-backed skills, re-enumerates available tools from MCP servers."

Actual behavior: The implementation has a TODO comment instead of actual MCP refresh logic:

# For MCP skills, we would sync here if MCP adapter was available
# Currently, we just mark as refreshed
if has_mcp:
    # TODO: When MCP adapter integration is available, call
    # mcp_adapter.refresh_tools() for each server
    pass

The command reports mcp_status: "synced" even though no actual MCP server communication occurs.

Code location: src/cleveragents/cli/commands/skill.py, lines 1018–1025

Steps to reproduce:

Register a skill with MCP servers: agents skill add --config mcp-backed.yaml
Run: agents skill refresh local/linear-tracker
Output shows mcp_status: "synced" but no MCP server was contacted

Impact: MCP-backed skills cannot be refreshed via the CLI. If an MCP server's tool list changes, users have no way to pick up those changes through the documented agents skill refresh command.

Severity: High — core MCP integration feature is non-functional

Subtasks

Write a failing Behave scenario (@tdd_expected_fail) that asserts mcp_adapter.refresh_tools() is called for each MCP server when agents skill refresh is invoked on an MCP-backed skill
Implement mcp_adapter.refresh_tools() call in SkillRefreshCommand (or equivalent handler) in src/cleveragents/cli/commands/skill.py for each configured MCP server
Ensure mcp_status in the output envelope is only set to "synced" after actual MCP server communication succeeds; set to "error" or "skipped" on failure or absence of MCP servers
Add type annotations to all new/modified function signatures and verify with nox -e typecheck
Remove the TODO comment and stub pass block once the real implementation is in place
Update or add Robot Framework integration test in robot/ to verify end-to-end MCP refresh behaviour (no mocks)
Run nox (all default sessions) and fix any errors
Verify coverage >= 97% via nox -e coverage_report

Definition of Done

Failing Behave scenario (@tdd_expected_fail) exists and captures the bug before the fix
agents skill refresh calls mcp_adapter.refresh_tools() for each configured MCP server on MCP-backed skills
mcp_status in the output envelope accurately reflects whether MCP server communication occurred and succeeded
The stub pass block and TODO comment are removed from src/cleveragents/cli/commands/skill.py
All new code is fully statically typed and passes nox -e typecheck
No # type: ignore suppressions introduced
Robot Framework integration test covers the end-to-end MCP refresh flow
All nox stages pass
Coverage >= 97%
PR is created, linked to this issue, and has two approving reviews

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/skill-refresh-mcp-tools-stub` - **Commit Message**: `fix(skill): implement MCP tool re-enumeration in agents skill refresh command` - **Milestone**: v3.7.0 - **Parent Epic**: #936 ## Summary The `agents skill refresh` command claims to "Recompute tool flattening and sync MCP-backed skills" but the MCP refresh portion is a stub with a TODO comment. MCP tools are never actually re-enumerated from their servers. The command reports `mcp_status: "synced"` even though no actual MCP server communication occurs, misleading users into believing their MCP tools have been refreshed when they have not. ## Bug Details **What was tested**: Code-level analysis of the `agents skill refresh` command implementation in `src/cleveragents/cli/commands/skill.py`. **Expected behavior (from spec)**: The `agents skill refresh` command should, for MCP-backed skills, re-enumerate available tools from MCP servers by calling `mcp_adapter.refresh_tools()` for each configured MCP server. The spec states: *"For MCP-backed skills, re-enumerates available tools from MCP servers."* **Actual behavior**: The implementation has a TODO comment instead of actual MCP refresh logic: ```python # For MCP skills, we would sync here if MCP adapter was available # Currently, we just mark as refreshed if has_mcp: # TODO: When MCP adapter integration is available, call # mcp_adapter.refresh_tools() for each server pass ``` The command reports `mcp_status: "synced"` even though no actual MCP server communication occurs. **Code location**: `src/cleveragents/cli/commands/skill.py`, lines 1018–1025 **Steps to reproduce**: 1. Register a skill with MCP servers: `agents skill add --config mcp-backed.yaml` 2. Run: `agents skill refresh local/linear-tracker` 3. Output shows `mcp_status: "synced"` but no MCP server was contacted **Impact**: MCP-backed skills cannot be refreshed via the CLI. If an MCP server's tool list changes, users have no way to pick up those changes through the documented `agents skill refresh` command. **Severity**: High — core MCP integration feature is non-functional ## Subtasks - [ ] Write a failing Behave scenario (`@tdd_expected_fail`) that asserts `mcp_adapter.refresh_tools()` is called for each MCP server when `agents skill refresh` is invoked on an MCP-backed skill - [ ] Implement `mcp_adapter.refresh_tools()` call in `SkillRefreshCommand` (or equivalent handler) in `src/cleveragents/cli/commands/skill.py` for each configured MCP server - [ ] Ensure `mcp_status` in the output envelope is only set to `"synced"` after actual MCP server communication succeeds; set to `"error"` or `"skipped"` on failure or absence of MCP servers - [ ] Add type annotations to all new/modified function signatures and verify with `nox -e typecheck` - [ ] Remove the TODO comment and stub `pass` block once the real implementation is in place - [ ] Update or add Robot Framework integration test in `robot/` to verify end-to-end MCP refresh behaviour (no mocks) - [ ] Run `nox` (all default sessions) and fix any errors - [ ] Verify coverage >= 97% via `nox -e coverage_report` ## Definition of Done - [ ] Failing Behave scenario (`@tdd_expected_fail`) exists and captures the bug before the fix - [ ] `agents skill refresh` calls `mcp_adapter.refresh_tools()` for each configured MCP server on MCP-backed skills - [ ] `mcp_status` in the output envelope accurately reflects whether MCP server communication occurred and succeeded - [ ] The stub `pass` block and TODO comment are removed from `src/cleveragents/cli/commands/skill.py` - [ ] All new code is fully statically typed and passes `nox -e typecheck` - [ ] No `# type: ignore` suppressions introduced - [ ] Robot Framework integration test covers the end-to-end MCP refresh flow - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] PR is created, linked to this issue, and has two approving reviews --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-03 04:08:39 +00:00

freemo added a new dependency

2026-04-03 04:10:39 +00:00

#936 Epic: Output Rendering Pipeline Integration — Wire Commands to OutputSession Framework

freemo referenced this issue

2026-04-03 04:11:31 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 711 — v3.7.0 #2115

freemo referenced this issue

2026-04-03 04:20:11 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:37:50 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo self-assigned this

2026-04-03 16:58:06 +00:00

freemo commented

2026-04-03 18:28:15 +00:00

Author

Owner

MoSCoW classification: Should Have

Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

MoSCoW classification: **Should Have** Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner