UAT: MCPToolResult.data typed as dict[str, Any] but MCP protocol returns content as a list — type mismatch causes Pydantic validation error with real MCP servers #2152

Open
opened 2026-04-03 04:28:27 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: fix/mcp-tool-result-data-type-mismatch
  • Commit Message: fix(mcp): widen MCPToolResult.data type to accept MCP 1.4.0 list content responses
  • Milestone: v3.7.0
  • Parent Epic: #399

Bug Report

Severity Assessment

  • Impact: High — all real MCP 1.4.0 server integrations will fail with a Pydantic ValidationError on every tool invocation. The mock transport masks this by returning a non-standard dict, so the bug is invisible in unit tests.
  • Likelihood: Certain — any deployment connecting MCPToolAdapter to a real MCP 1.4.0 server will hit this on the first invoke() call.
  • Priority: High

What Was Tested

Code-level analysis of MCPToolResult model definition and MCPToolAdapter.invoke() return value handling against the MCP 1.4.0 protocol specification.

Expected Behavior

MCPToolResult.data should be typed to accept the actual MCP 1.4.0 protocol response format. Per the MCP 1.4.0 spec, tool call results have content: list[TextContent | ImageContent | EmbeddedResource]. The field type should be widened to list[Any] | dict[str, Any] (or Any) to correctly handle both the mock protocol format (dict) and the real MCP protocol format (list).

Actual Behavior

# src/cleveragents/mcp/adapter.py, line 132
class MCPToolResult(BaseModel):
    data: dict[str, Any] = Field(default_factory=dict, ...)  # Wrong type!

In MCPToolAdapter.invoke():

# src/cleveragents/mcp/adapter.py, lines 523–527
return MCPToolResult(
    success=True,
    data=result.get("content", result),  # content is a list in real MCP 1.4.0!
    duration_ms=elapsed,
)

When a real MCP 1.4.0 server returns {"content": [{"type": "text", "text": "result"}]}, result.get("content", result) returns a list, and MCPToolResult(data=[...]) raises:

pydantic.ValidationError: 1 validation error for MCPToolResult
data
  value is not a valid dict (type=type_error.dict)

The mock transport in features/mocks/mock_mcp_transport.py (line 58) works around this by returning {"content": {"result": "ok"}} — a dict — which is not valid MCP 1.4.0 protocol. This masks the bug in all unit tests.

Code Locations

  • src/cleveragents/mcp/adapter.py, line 132: data: dict[str, Any] — incorrect type annotation
  • src/cleveragents/mcp/adapter.py, lines 523–527: data=result.get("content", result) — passes list to dict-typed field
  • features/mocks/mock_mcp_transport.py, line 58: mock returns dict content (non-standard, masks the bug)

Steps to Reproduce

  1. Connect MCPToolAdapter to a real MCP 1.4.0 server
  2. Call adapter.invoke("some_tool", {})
  3. Observe pydantic.ValidationError: value is not a valid dict when the server returns list content

Subtasks

  • Write a failing Behave scenario (@tdd_expected_fail) demonstrating that MCPToolResult(data=[{"type": "text", "text": "ok"}]) raises ValidationError with the current type
  • Write a failing Behave scenario demonstrating that MCPToolAdapter.invoke() raises ValidationError when the underlying transport returns {"content": [{"type": "text", "text": "ok"}]} (real MCP 1.4.0 format)
  • Update MCPToolResult.data type annotation from dict[str, Any] to list[Any] | dict[str, Any] in src/cleveragents/mcp/adapter.py line 132
  • Update the default_factory for MCPToolResult.data to be consistent with the widened type (e.g., keep dict as default or use a union-compatible default)
  • Update features/mocks/mock_mcp_transport.py line 58 to return {"content": [{"type": "text", "text": "mock result"}]} — a list — to match the real MCP 1.4.0 protocol format
  • Verify all existing Behave scenarios that reference MCPToolResult.data still pass after the type change
  • Update docs/api/mcp.md if the MCPToolResult.data field type is documented there (cross-check with #2118)
  • Run nox -e typecheck and confirm no Pyright errors introduced by the type widening
  • Run nox -e lint and confirm no lint errors
  • Run nox -e unit_tests and confirm all Behave scenarios pass
  • Run nox -e coverage_report and confirm coverage ≥ 97%

Definition of Done

  • MCPToolResult.data is typed as list[Any] | dict[str, Any] (or Any) in src/cleveragents/mcp/adapter.py
  • MCPToolAdapter.invoke() no longer raises ValidationError when a real MCP 1.4.0 server returns content as a list
  • features/mocks/mock_mcp_transport.py returns MCP 1.4.0-compliant list content (not a non-standard dict)
  • Failing Behave scenarios exist before implementation (TDD)
  • All Behave scenarios pass after the fix
  • docs/api/mcp.md updated if MCPToolResult.data type is documented there
  • No Pyright type errors (nox -e typecheck passes)
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(mcp): widen MCPToolResult.data type to accept MCP 1.4.0 list content responses), followed by a blank line, then additional detail lines, ending with ISSUES CLOSED: #<this issue number>
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/mcp-tool-result-data-type-mismatch)
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/mcp-tool-result-data-type-mismatch` - **Commit Message**: `fix(mcp): widen MCPToolResult.data type to accept MCP 1.4.0 list content responses` - **Milestone**: v3.7.0 - **Parent Epic**: #399 ## Bug Report ### Severity Assessment - **Impact**: High — all real MCP 1.4.0 server integrations will fail with a Pydantic `ValidationError` on every tool invocation. The mock transport masks this by returning a non-standard dict, so the bug is invisible in unit tests. - **Likelihood**: Certain — any deployment connecting `MCPToolAdapter` to a real MCP 1.4.0 server will hit this on the first `invoke()` call. - **Priority**: High ### What Was Tested Code-level analysis of `MCPToolResult` model definition and `MCPToolAdapter.invoke()` return value handling against the MCP 1.4.0 protocol specification. ### Expected Behavior `MCPToolResult.data` should be typed to accept the actual MCP 1.4.0 protocol response format. Per the MCP 1.4.0 spec, tool call results have `content: list[TextContent | ImageContent | EmbeddedResource]`. The field type should be widened to `list[Any] | dict[str, Any]` (or `Any`) to correctly handle both the mock protocol format (dict) and the real MCP protocol format (list). ### Actual Behavior ```python # src/cleveragents/mcp/adapter.py, line 132 class MCPToolResult(BaseModel): data: dict[str, Any] = Field(default_factory=dict, ...) # Wrong type! ``` In `MCPToolAdapter.invoke()`: ```python # src/cleveragents/mcp/adapter.py, lines 523–527 return MCPToolResult( success=True, data=result.get("content", result), # content is a list in real MCP 1.4.0! duration_ms=elapsed, ) ``` When a real MCP 1.4.0 server returns `{"content": [{"type": "text", "text": "result"}]}`, `result.get("content", result)` returns a list, and `MCPToolResult(data=[...])` raises: ``` pydantic.ValidationError: 1 validation error for MCPToolResult data value is not a valid dict (type=type_error.dict) ``` The mock transport in `features/mocks/mock_mcp_transport.py` (line 58) works around this by returning `{"content": {"result": "ok"}}` — a dict — which is not valid MCP 1.4.0 protocol. This masks the bug in all unit tests. ### Code Locations - `src/cleveragents/mcp/adapter.py`, line 132: `data: dict[str, Any]` — incorrect type annotation - `src/cleveragents/mcp/adapter.py`, lines 523–527: `data=result.get("content", result)` — passes list to dict-typed field - `features/mocks/mock_mcp_transport.py`, line 58: mock returns dict content (non-standard, masks the bug) ### Steps to Reproduce 1. Connect `MCPToolAdapter` to a real MCP 1.4.0 server 2. Call `adapter.invoke("some_tool", {})` 3. Observe `pydantic.ValidationError: value is not a valid dict` when the server returns list content ## Subtasks - [ ] Write a failing Behave scenario (`@tdd_expected_fail`) demonstrating that `MCPToolResult(data=[{"type": "text", "text": "ok"}])` raises `ValidationError` with the current type - [ ] Write a failing Behave scenario demonstrating that `MCPToolAdapter.invoke()` raises `ValidationError` when the underlying transport returns `{"content": [{"type": "text", "text": "ok"}]}` (real MCP 1.4.0 format) - [ ] Update `MCPToolResult.data` type annotation from `dict[str, Any]` to `list[Any] | dict[str, Any]` in `src/cleveragents/mcp/adapter.py` line 132 - [ ] Update the `default_factory` for `MCPToolResult.data` to be consistent with the widened type (e.g., keep `dict` as default or use a union-compatible default) - [ ] Update `features/mocks/mock_mcp_transport.py` line 58 to return `{"content": [{"type": "text", "text": "mock result"}]}` — a list — to match the real MCP 1.4.0 protocol format - [ ] Verify all existing Behave scenarios that reference `MCPToolResult.data` still pass after the type change - [ ] Update `docs/api/mcp.md` if the `MCPToolResult.data` field type is documented there (cross-check with #2118) - [ ] Run `nox -e typecheck` and confirm no Pyright errors introduced by the type widening - [ ] Run `nox -e lint` and confirm no lint errors - [ ] Run `nox -e unit_tests` and confirm all Behave scenarios pass - [ ] Run `nox -e coverage_report` and confirm coverage ≥ 97% ## Definition of Done - [ ] `MCPToolResult.data` is typed as `list[Any] | dict[str, Any]` (or `Any`) in `src/cleveragents/mcp/adapter.py` - [ ] `MCPToolAdapter.invoke()` no longer raises `ValidationError` when a real MCP 1.4.0 server returns `content` as a list - [ ] `features/mocks/mock_mcp_transport.py` returns MCP 1.4.0-compliant list content (not a non-standard dict) - [ ] Failing Behave scenarios exist before implementation (TDD) - [ ] All Behave scenarios pass after the fix - [ ] `docs/api/mcp.md` updated if `MCPToolResult.data` type is documented there - [ ] No Pyright type errors (`nox -e typecheck` passes) - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(mcp): widen MCPToolResult.data type to accept MCP 1.4.0 list content responses`), followed by a blank line, then additional detail lines, ending with `ISSUES CLOSED: #<this issue number>` - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/mcp-tool-result-data-type-mismatch`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-03 04:28:32 +00:00
freemo self-assigned this 2026-04-03 16:58:00 +00:00
Author
Owner

MoSCoW classification: Must Have

Rationale: This issue addresses a core spec requirement or blocks critical functionality. The project cannot ship without this fix.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

MoSCoW classification: **Must Have** Rationale: This issue addresses a core spec requirement or blocks critical functionality. The project cannot ship without this fix. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo removed this from the v3.7.0 milestone 2026-04-07 01:19:43 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#399 Epic: Post-MVP Server & Clients
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#2152
No description provided.