cleveragents/cleveragents-core

Fork 3

UAT: `ProviderCapabilities.supports_json_mode` is `False` for Anthropic — incorrect capability metadata, Anthropic supports structured output via tool-use #4728

New issue

Open

opened 2026-04-08 18:12:21 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-08 18:12:21 +00:00

Owner

Summary

ProviderRegistry.DEFAULT_CAPABILITIES in src/cleveragents/providers/registry.py marks supports_json_mode=False for the Anthropic provider. This is incorrect — Anthropic Claude models support structured JSON output via tool-use (function calling), which is the standard mechanism for JSON mode in LangChain's ChatAnthropic.

Expected Behavior

Anthropic's supports_json_mode should be True. Claude models support structured output through:

Tool-use / function calling (the primary mechanism)
Prompt-based JSON output (less reliable but supported)

The spec's capability metadata should accurately reflect what providers support to allow the system to make correct routing decisions (e.g., when response_format is specified in an actor config).

Actual Behavior

# src/cleveragents/providers/registry.py
DEFAULT_CAPABILITIES: ClassVar[dict[ProviderType, ProviderCapabilities]] = {
    ProviderType.ANTHROPIC: ProviderCapabilities(
        supports_streaming=True,
        supports_tool_calls=True,
        supports_vision=True,
        max_context_length=200000,
        supports_json_mode=False,  # ← incorrect
    ),
    ...
}

Meanwhile, supports_tool_calls=True for Anthropic — and tool-use is the mechanism through which JSON mode works. The inconsistency means any code that checks supports_json_mode before using structured output will incorrectly skip Anthropic.

Impact

Any code that gates structured output on supports_json_mode will incorrectly exclude Anthropic
Actor configs with response_format set will fail silently or raise errors when using Anthropic
The capability metadata is misleading for users inspecting provider capabilities

Code Location

src/cleveragents/providers/registry.py — DEFAULT_CAPABILITIES class variable, Anthropic entry (lines ~100–110)

Fix Direction

ProviderType.ANTHROPIC: ProviderCapabilities(
    supports_streaming=True,
    supports_tool_calls=True,
    supports_vision=True,
    max_context_length=200000,
    supports_json_mode=True,  # ← fix: Anthropic supports JSON via tool-use
),

Note: If supports_json_mode is intended to mean "native JSON mode" (like OpenAI's response_format: {"type": "json_object"}), then the field semantics should be clarified in the docstring, and a separate supports_structured_output_via_tools field may be needed. Either way, the current False value is misleading.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Summary `ProviderRegistry.DEFAULT_CAPABILITIES` in `src/cleveragents/providers/registry.py` marks `supports_json_mode=False` for the Anthropic provider. This is incorrect — Anthropic Claude models support structured JSON output via tool-use (function calling), which is the standard mechanism for JSON mode in LangChain's `ChatAnthropic`. ## Expected Behavior Anthropic's `supports_json_mode` should be `True`. Claude models support structured output through: 1. Tool-use / function calling (the primary mechanism) 2. Prompt-based JSON output (less reliable but supported) The spec's capability metadata should accurately reflect what providers support to allow the system to make correct routing decisions (e.g., when `response_format` is specified in an actor config). ## Actual Behavior ```python # src/cleveragents/providers/registry.py DEFAULT_CAPABILITIES: ClassVar[dict[ProviderType, ProviderCapabilities]] = { ProviderType.ANTHROPIC: ProviderCapabilities( supports_streaming=True, supports_tool_calls=True, supports_vision=True, max_context_length=200000, supports_json_mode=False, # ← incorrect ), ... } ``` Meanwhile, `supports_tool_calls=True` for Anthropic — and tool-use is the mechanism through which JSON mode works. The inconsistency means any code that checks `supports_json_mode` before using structured output will incorrectly skip Anthropic. ## Impact - Any code that gates structured output on `supports_json_mode` will incorrectly exclude Anthropic - Actor configs with `response_format` set will fail silently or raise errors when using Anthropic - The capability metadata is misleading for users inspecting provider capabilities ## Code Location - `src/cleveragents/providers/registry.py` — `DEFAULT_CAPABILITIES` class variable, Anthropic entry (lines ~100–110) ## Fix Direction ```python ProviderType.ANTHROPIC: ProviderCapabilities( supports_streaming=True, supports_tool_calls=True, supports_vision=True, max_context_length=200000, supports_json_mode=True, # ← fix: Anthropic supports JSON via tool-use ), ``` Note: If `supports_json_mode` is intended to mean "native JSON mode" (like OpenAI's `response_format: {"type": "json_object"}`), then the field semantics should be clarified in the docstring, and a separate `supports_structured_output_via_tools` field may be needed. Either way, the current `False` value is misleading. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-08 18:52:12 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4743

HAL9000 referenced this issue

2026-04-08 18:55:48 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4743

HAL9000 added the

labels

2026-04-08 23:00:50 +00:00

HAL9000 commented

2026-04-09 03:05:40 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — spec compliance bug identified by UAT testing
Story Points: 3 (M) — targeted fix to align implementation with spec
MoSCoW: Must Have — spec compliance is required for correct system behavior

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — spec compliance bug identified by UAT testing - **Story Points**: 3 (M) — targeted fix to align implementation with spec - **MoSCoW**: Must Have — spec compliance is required for correct system behavior --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 added

and removed

labels

2026-04-09 03:05:50 +00:00

HAL9000 added this to the v3.5.0 milestone