aditya
ee82d6101e
Implement MCPToolAdapter to connect to external MCP servers, enumerate tools, and register them in ToolRegistry with source="mcp". Includes connect/reconnect/disconnect lifecycle with timeout enforcement, input validation on invoke, capability inference, Behave/Robot/ASV tests, and docs/reference/mcp_adapter.md. ISSUES CLOSED: #159
282 lines
12 KiB
Gherkin
282 lines
12 KiB
Gherkin
Feature: MCP Tool Adapter
|
|
As an actor runtime
|
|
I want an adapter that connects to MCP servers, discovers tools,
|
|
and invokes them with schema validation
|
|
So that external MCP tools are usable like any registered tool
|
|
|
|
# -------------------------------------------------------------------
|
|
# Adapter Lifecycle
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Create adapter from server config
|
|
Given an MCP server config for "github" with stdio transport
|
|
When I create an MCP adapter from the config
|
|
Then the adapter server name should be "github"
|
|
And the adapter transport should be "stdio"
|
|
And the adapter should not be connected
|
|
|
|
Scenario: Create adapter from SSE server config
|
|
Given an MCP server config for "remote-api" with sse transport and url "https://api.example.com/mcp"
|
|
When I create an MCP adapter from the config
|
|
Then the adapter server name should be "remote-api"
|
|
And the adapter transport should be "sse"
|
|
|
|
Scenario: Connect adapter starts handshake
|
|
Given an MCP adapter with a mock transport
|
|
When I connect the adapter
|
|
Then the adapter should be connected
|
|
And the adapter server capabilities should be set
|
|
|
|
Scenario: Disconnect adapter cleans up
|
|
Given a connected MCP adapter with a mock transport
|
|
When I disconnect the adapter
|
|
Then the adapter should not be connected
|
|
|
|
Scenario: Double connect is idempotent
|
|
Given a connected MCP adapter with a mock transport
|
|
When I connect the adapter
|
|
Then the adapter should be connected
|
|
|
|
Scenario: Disconnect when not connected is safe
|
|
Given an MCP adapter with a mock transport
|
|
When I disconnect the adapter
|
|
Then the adapter should not be connected
|
|
|
|
# -------------------------------------------------------------------
|
|
# Tool Discovery
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Discover tools from MCP server
|
|
Given a connected MCP adapter with 3 mock tools
|
|
When I discover tools from the adapter
|
|
Then the adapter should have 3 discovered tools
|
|
And discovered tool 0 should have a name
|
|
And discovered tool 0 should have an input schema
|
|
|
|
Scenario: Discover tools with include filter
|
|
Given a connected MCP adapter with 3 mock tools
|
|
And a tool filter including only "tool_0"
|
|
When I discover tools from the adapter with filter
|
|
Then the adapter should have 1 discovered tools
|
|
|
|
Scenario: Discover tools with exclude filter
|
|
Given a connected MCP adapter with 3 mock tools
|
|
And a tool filter excluding "tool_0"
|
|
When I discover tools from the adapter with filter
|
|
Then the adapter should have 2 discovered tools
|
|
|
|
Scenario: Discover returns empty when server has no tools
|
|
Given a connected MCP adapter with 0 mock tools
|
|
When I discover tools from the adapter
|
|
Then the adapter should have 0 discovered tools
|
|
|
|
Scenario: Discover when not connected raises error
|
|
Given an MCP adapter with a mock transport
|
|
When I discover tools expecting an error
|
|
Then the adapter error should mention "not connected"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Tool Invocation
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Invoke a discovered tool successfully
|
|
Given a connected MCP adapter with a callable mock tool "create_issue"
|
|
When I invoke "create_issue" with arguments {"title": "Bug", "body": "Fix it"}
|
|
Then the invocation should succeed
|
|
And the invocation result should contain key "id"
|
|
|
|
Scenario: Invoke with valid input schema passes validation
|
|
Given a connected MCP adapter with a schema-validated mock tool "create_issue"
|
|
When I invoke "create_issue" with arguments {"title": "Bug", "body": "Fix it"}
|
|
Then the invocation should succeed
|
|
|
|
Scenario: Invoke with invalid input fails validation
|
|
Given a connected MCP adapter with a schema-validated mock tool "create_issue"
|
|
When I invoke "create_issue" with arguments {"wrong_field": 123}
|
|
Then the invocation should fail
|
|
And the invocation error should mention "validation"
|
|
|
|
Scenario: Invoke unknown tool raises error
|
|
Given a connected MCP adapter with a callable mock tool "create_issue"
|
|
When I invoke "nonexistent_tool" with arguments {}
|
|
Then the invocation should fail
|
|
And the invocation error should mention "not found"
|
|
|
|
Scenario: Invoke when not connected raises error
|
|
Given an MCP adapter with a mock transport
|
|
When I invoke MCP tool "create_issue" while disconnected
|
|
Then the adapter error should mention "not connected"
|
|
|
|
Scenario: Invoke tool that returns error from server
|
|
Given a connected MCP adapter with a failing mock tool "bad_tool"
|
|
When I invoke "bad_tool" with arguments {}
|
|
Then the invocation should fail
|
|
And the invocation error should mention "server error"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Tool Registration in ToolRegistry
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Register discovered MCP tools in ToolRegistry
|
|
Given a connected MCP adapter with 2 mock tools
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then the registry should have 2 tools
|
|
And registry tool 0 name should start with "mcp-github/"
|
|
|
|
Scenario: Registered MCP tool is callable via registry
|
|
Given a connected MCP adapter with a callable mock tool "list_repos"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then calling registry tool "mcp-github/list_repos" should succeed
|
|
|
|
Scenario: Re-registration after tool list changed
|
|
Given a connected MCP adapter with 2 mock tools
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
And the MCP server adds a new tool "deploy"
|
|
And I re-register MCP tools in the registry with namespace "mcp-github"
|
|
Then the registry should have 3 tools
|
|
|
|
Scenario: Registered tool handler returns error on failed invoke
|
|
Given a connected MCP adapter with a failing mock tool "bad_tool"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-fail"
|
|
Then calling registry tool "mcp-fail/bad_tool" should return an error
|
|
|
|
# -------------------------------------------------------------------
|
|
# Capability Inference
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Infer read_only for tool named list_repos
|
|
When I infer capabilities for tool name "list_repos"
|
|
Then the inferred capabilities should have read_only true
|
|
And the inferred capabilities should have writes false
|
|
|
|
Scenario: Infer writes for tool named create_issue
|
|
When I infer capabilities for tool name "create_issue"
|
|
Then the inferred capabilities should have read_only false
|
|
And the inferred capabilities should have writes true
|
|
|
|
Scenario: Infer writes for tool named delete_branch
|
|
When I infer capabilities for tool name "delete_branch"
|
|
Then the inferred capabilities should have read_only false
|
|
And the inferred capabilities should have writes true
|
|
|
|
Scenario: Infer default for unknown tool name
|
|
When I infer capabilities for tool name "run_pipeline"
|
|
Then the inferred capabilities should have read_only false
|
|
And the inferred capabilities should have writes false
|
|
|
|
Scenario: Infer read_only for tool with get prefix
|
|
When I infer capabilities for tool name "get-user-details"
|
|
Then the inferred capabilities should have read_only true
|
|
And the inferred capabilities should have writes false
|
|
|
|
Scenario: Write overrides read when both keywords present
|
|
When I infer capabilities for tool name "search_and_update"
|
|
Then the inferred capabilities should have writes true
|
|
|
|
Scenario: Registered MCP tool has inferred capabilities
|
|
Given a connected MCP adapter with a callable mock tool "list_repos"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then registry tool "mcp-github/list_repos" should have read_only capability
|
|
|
|
Scenario: Registered write tool has writes capability
|
|
Given a connected MCP adapter with a callable mock tool "create_issue"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then registry tool "mcp-github/create_issue" should have writes capability
|
|
|
|
# -------------------------------------------------------------------
|
|
# Edge Cases
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Access discovered_tools property after discovery
|
|
Given a connected MCP adapter with 2 mock tools
|
|
When I discover tools from the adapter
|
|
Then the adapter discovered_tools property should have 2 items
|
|
|
|
Scenario: Disconnect handles close errors gracefully
|
|
Given a connected MCP adapter with a transport that errors on close
|
|
When I disconnect the adapter
|
|
Then the adapter should not be connected
|
|
|
|
Scenario: Invoke tool that raises generic exception
|
|
Given a connected MCP adapter with a transport-error mock tool "crash_tool"
|
|
When I invoke "crash_tool" with arguments {}
|
|
Then the invocation should fail
|
|
And the invocation error should mention "server error"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Error Classification
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Classify connection failure
|
|
Given an MCP adapter with a transport that fails to connect
|
|
When I connect the adapter expecting error
|
|
Then the adapter error should mention "connection"
|
|
|
|
Scenario: Classify timeout error
|
|
Given a connected MCP adapter with a tool that times out
|
|
When I invoke "slow_tool" with arguments {}
|
|
Then the invocation should fail
|
|
And the invocation error should mention "timeout"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Server Config Validation
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Stdio transport requires command
|
|
Given an MCP server config for "bad" with stdio transport but no command
|
|
When I create an MCP adapter from the config expecting validation error
|
|
Then the adapter error should mention "command"
|
|
|
|
Scenario: SSE transport requires url
|
|
Given an MCP server config for "bad" with sse transport but no url
|
|
When I create an MCP adapter from the config expecting validation error
|
|
Then the adapter error should mention "url"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Reconnect Lifecycle
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Reconnect restores connection after disconnect
|
|
Given a connected MCP adapter with a mock transport
|
|
When I disconnect the adapter
|
|
And I reconnect the adapter
|
|
Then the adapter should be connected
|
|
|
|
Scenario: Reconnect clears tool cache
|
|
Given a connected MCP adapter with 3 mock tools
|
|
When I discover tools from the adapter
|
|
And I reconnect the adapter
|
|
Then the adapter discovered_tools property should have 0 items
|
|
|
|
Scenario: Reconnect on unavailable server raises connection error
|
|
Given a connected MCP adapter with a transport that fails to reconnect
|
|
When I reconnect the adapter expecting error
|
|
Then the adapter error should mention "connection"
|
|
|
|
Scenario: Connect times out when transport hangs
|
|
Given an MCP adapter with a transport that hangs on connect
|
|
When I connect the adapter with timeout 0.05 expecting error
|
|
Then the adapter error should mention "connection"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Source and Checkpointable Metadata
|
|
# -------------------------------------------------------------------
|
|
|
|
Scenario: Registered MCP tool has source mcp
|
|
Given a connected MCP adapter with a callable mock tool "list_repos"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then registry tool "mcp-github/list_repos" should have source "mcp"
|
|
|
|
Scenario: Registered MCP tool has checkpointable false
|
|
Given a connected MCP adapter with a callable mock tool "create_issue"
|
|
And an empty MCP tool registry
|
|
When I register MCP tools in the registry with namespace "mcp-github"
|
|
Then registry tool "mcp-github/create_issue" should have checkpointable false
|