cleveragents/cleveragents-core

Fork 3

UAT: MockMCPTransport success response uses non-standard content format, masking MCP 1.4.0 protocol violations #2744

New issue

Open

opened 2026-04-04 15:31:39 +00:00 by freemo · 2 comments

freemo commented

2026-04-04 15:31:39 +00:00

Owner

Metadata

Branch: fix/mock-mcp-transport-content-format
Commit Message: fix(mocks): correct MockMCPTransport success response to use MCP 1.4.0 content list format
Milestone: v3.8.0
Parent Epic: #399

Background

The MockMCPTransport in features/mocks/mock_mcp_transport.py returns a non-standard dict format for successful tool invocations, which does not match the real MCP 1.4.0 protocol. This causes unit tests to pass while masking a real protocol compliance bug.

Current (incorrect) mock behaviour:

# In MockMCPTransport.call():
if tool_name in self._invoke_results:
    return {"content": self._invoke_results[tool_name]}
# Where invoke_results = {tool_name: {"id": 42, "status": "created"}}
# → Returns: {"content": {"id": 42, "status": "created"}}  ← dict, NOT list

Real MCP 1.4.0 protocol format for successful tool calls:

{
  "content": [
    {"type": "text", "text": "result text"}
  ]
}

The content field MUST be a list of ContentItem objects, not a bare dict.

Impact:

The test scenario "Invoke a discovered tool successfully" checks "id" in context.mcp_invoke_result.data — this only works because the mock returns a dict. With a real MCP 1.4.0 server, data would be a list and this check would fail.
The MCPToolResult.data field is typed as dict[str, Any], but real MCP 1.4.0 would populate it with a list. The mock masks this type mismatch.
Any downstream code written against the mock's behaviour (treating result.data as a dict with tool-specific keys) will break when connected to a real MCP 1.4.0 server.

Note: The error path in MockMCPTransport already correctly uses MCP 1.4.0 format:

return {
    "isError": True,
    "content": [{"type": "text", "text": self._invoke_errors[tool_name]}],
}

The success path must be updated to match.

Subtasks

Write a failing Behave scenario (TDD capture test) that asserts MockMCPTransport.call() returns content as a list of ContentItem dicts for a successful invocation
Update MockMCPTransport.call() in features/mocks/mock_mcp_transport.py to return the correct MCP 1.4.0 format:
```
return {"content": [{"type": "text", "text": json.dumps(self._invoke_results[tool_name])}]}
```
Review and update MCPToolResult.data type annotation — change from dict[str, Any] to list[dict[str, Any]] (or a proper ContentItem type) to reflect the real MCP 1.4.0 wire format
Update all existing Behave scenarios that assert on context.mcp_invoke_result.data to work with the corrected list-based format
Verify no downstream code treats MCPToolResult.data as a plain dict with tool-specific keys; fix any such code
Run nox -e typecheck and resolve any Pyright errors introduced by the type annotation change
Run nox -e unit_tests and confirm all Behave scenarios pass
Run nox -e coverage_report and confirm coverage remains >= 97%
Run full nox suite and confirm all sessions pass

Definition of Done

All subtasks above are checked off
MockMCPTransport.call() returns {"content": [{"type": "text", "text": ...}]} for success responses, matching MCP 1.4.0 protocol
MCPToolResult.data type annotation correctly reflects the MCP 1.4.0 content list structure
No existing Behave scenario relies on the old incorrect dict format
All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e integration_tests, nox -e coverage_report)
Coverage >= 97%
Commit created with message fix(mocks): correct MockMCPTransport success response to use MCP 1.4.0 content list format on branch fix/mock-mcp-transport-content-format
Pull Request merged and issue closed via ISSUES CLOSED: #<this issue>

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/mock-mcp-transport-content-format` - **Commit Message**: `fix(mocks): correct MockMCPTransport success response to use MCP 1.4.0 content list format` - **Milestone**: v3.8.0 - **Parent Epic**: #399 ## Background The `MockMCPTransport` in `features/mocks/mock_mcp_transport.py` returns a non-standard dict format for successful tool invocations, which does not match the real MCP 1.4.0 protocol. This causes unit tests to pass while masking a real protocol compliance bug. **Current (incorrect) mock behaviour:** ```python # In MockMCPTransport.call(): if tool_name in self._invoke_results: return {"content": self._invoke_results[tool_name]} # Where invoke_results = {tool_name: {"id": 42, "status": "created"}} # → Returns: {"content": {"id": 42, "status": "created"}} ← dict, NOT list ``` **Real MCP 1.4.0 protocol format** for successful tool calls: ```json { "content": [ {"type": "text", "text": "result text"} ] } ``` The `content` field MUST be a list of `ContentItem` objects, not a bare dict. **Impact:** 1. The test scenario "Invoke a discovered tool successfully" checks `"id" in context.mcp_invoke_result.data` — this only works because the mock returns a dict. With a real MCP 1.4.0 server, `data` would be a list and this check would fail. 2. The `MCPToolResult.data` field is typed as `dict[str, Any]`, but real MCP 1.4.0 would populate it with a `list`. The mock masks this type mismatch. 3. Any downstream code written against the mock's behaviour (treating `result.data` as a dict with tool-specific keys) will break when connected to a real MCP 1.4.0 server. **Note:** The error path in `MockMCPTransport` already correctly uses MCP 1.4.0 format: ```python return { "isError": True, "content": [{"type": "text", "text": self._invoke_errors[tool_name]}], } ``` The success path must be updated to match. ## Subtasks - [ ] Write a failing Behave scenario (TDD capture test) that asserts `MockMCPTransport.call()` returns `content` as a list of `ContentItem` dicts for a successful invocation - [ ] Update `MockMCPTransport.call()` in `features/mocks/mock_mcp_transport.py` to return the correct MCP 1.4.0 format: ```python return {"content": [{"type": "text", "text": json.dumps(self._invoke_results[tool_name])}]} ``` - [ ] Review and update `MCPToolResult.data` type annotation — change from `dict[str, Any]` to `list[dict[str, Any]]` (or a proper `ContentItem` type) to reflect the real MCP 1.4.0 wire format - [ ] Update all existing Behave scenarios that assert on `context.mcp_invoke_result.data` to work with the corrected list-based format - [ ] Verify no downstream code treats `MCPToolResult.data` as a plain dict with tool-specific keys; fix any such code - [ ] Run `nox -e typecheck` and resolve any Pyright errors introduced by the type annotation change - [ ] Run `nox -e unit_tests` and confirm all Behave scenarios pass - [ ] Run `nox -e coverage_report` and confirm coverage remains >= 97% - [ ] Run full `nox` suite and confirm all sessions pass ## Definition of Done - [ ] All subtasks above are checked off - [ ] `MockMCPTransport.call()` returns `{"content": [{"type": "text", "text": ...}]}` for success responses, matching MCP 1.4.0 protocol - [ ] `MCPToolResult.data` type annotation correctly reflects the MCP 1.4.0 `content` list structure - [ ] No existing Behave scenario relies on the old incorrect dict format - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`, `nox -e coverage_report`) - [ ] Coverage >= 97% - [ ] Commit created with message `fix(mocks): correct MockMCPTransport success response to use MCP 1.4.0 content list format` on branch `fix/mock-mcp-transport-content-format` - [ ] Pull Request merged and issue closed via `ISSUES CLOSED: #<this issue>` --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.8.0 milestone

2026-04-04 15:31:56 +00:00

freemo added a new dependency

2026-04-04 15:33:20 +00:00

#399 Epic: Post-MVP Server & Clients

freemo referenced this issue

2026-04-04 15:33:48 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker (uat-pool-1) #2739

freemo referenced this issue

2026-04-04 19:15:21 +00:00

UAT: MCPToolResult.data type annotation incompatible with MCP 1.4.0 success content format #2743

freemo commented

2026-04-04 19:15:31 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — Mock fidelity issue. The mock's error path already uses the correct MCP 1.4.0 list format, but the success path uses a non-standard dict format. This masks real protocol violations in tests.
Milestone: v3.8.0 (M9: Server Implementation)
MoSCoW: Should Have — Test fidelity is important for catching real protocol bugs. The mock should accurately reflect the wire format so tests catch issues before they reach production. However, this is a test infrastructure fix, not a production code bug.
Parent Epic: #399 (Post-MVP Server & Clients)

Should be addressed before or alongside #2743 (MCPToolResult.data type fix), as the mock fix will expose the type annotation issue.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — Mock fidelity issue. The mock's error path already uses the correct MCP 1.4.0 list format, but the success path uses a non-standard dict format. This masks real protocol violations in tests. - **Milestone**: v3.8.0 (M9: Server Implementation) - **MoSCoW**: Should Have — Test fidelity is important for catching real protocol bugs. The mock should accurately reflect the wire format so tests catch issues before they reach production. However, this is a test infrastructure fix, not a production code bug. - **Parent Epic**: #399 (Post-MVP Server & Clients) Should be addressed before or alongside #2743 (MCPToolResult.data type fix), as the mock fix will expose the type annotation issue. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

State

In Progress

label

2026-04-05 08:18:42 +00:00

freemo commented

2026-04-05 08:23:07 +00:00

Author

Owner

Transition to State/In Progress completed.

Preconditions checked:

No Blocked precondition encountered (no Blocked label present).
Proceeded with label updates.

What was removed:

State/Verified

What was added:

State/In Progress (repo-scoped label, ID 1343). Note: The repository may also expose org-scoped variants of the same label; this action added the repo-scoped variant to the issue.

Current state: State/In Progress

If you want me to also clean up duplicate org-level labels named State/In Progress or to ensure only a single State/In Progress label remains, I can run a dedup/cleanup pass next.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-state-updater

Transition to State/In Progress completed. Preconditions checked: - No Blocked precondition encountered (no Blocked label present). - Proceeded with label updates. What was removed: - State/Verified What was added: - State/In Progress (repo-scoped label, ID 1343). Note: The repository may also expose org-scoped variants of the same label; this action added the repo-scoped variant to the issue. Current state: State/In Progress If you want me to also clean up duplicate org-level labels named State/In Progress or to ensure only a single State/In Progress label remains, I can run a dedup/cleanup pass next. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-state-updater

freemo removed the

State

In Progress

label

2026-04-05 08:52:21 +00:00

freemo referenced this issue

2026-04-05 09:08:33 +00:00

[CA-AUTO] Backlog Groomer — Session Tracker — groomer-1 — 2026-04-05 #3182