Fix v3 actor provider resolution for models without / separator #10926

Closed
opened 2026-04-29 08:13:03 +00:00 by hurui200320 · 1 comment
Member

Metadata

  • Commit Message: fix(actor): resolve provider from explicit field in v3 YAML before inferring from model
  • Branch: bugfix/m3-v3-actor-provider-resolution

Background and context

The v3 actor YAML schema supports an explicit provider field (e.g., provider: anthropic) alongside model (e.g., model: claude-sonnet-4-20250514). The specification (line 21412) defines the resolution order for provider/model as:

  1. CLI override
  2. Top-level provider / model keys
  3. Top-level provider_type / model_id aliases
  4. v2-extracted values from actors.<name>.config.provider / .model

Additionally, the v3 ActorConfigSchema (schema.py line 747) defines provider as an explicit field with description "LLM provider (e.g. openai, anthropic)".

Current behavior

Two bugs prevent built-in LLM actors from working when the model name lacks a / separator (e.g., claude-sonnet-4-20250514):

  1. _generate_builtin_actor_yaml (actor/registry.py line 128) constructs the model field as f"{provider}/{model}", producing "Anthropic/claude-sonnet-4-20250514". The Anthropic API expects just "claude-sonnet-4-20250514", causing a 404 error.

  2. _build_from_v3 (reactive/config_parser.py line 306) always infers the provider from the model string via infer_provider_from_model(). Since claude-sonnet-4-20250514 contains no /, this returns "custom" — which is not a valid ProviderType, causing Unknown provider type: custom.

The net result: uv run agents actor run anthropic/claude-sonnet-4-20250514 hi fails with Unexpected error: Unknown provider type: custom.

Expected behavior

  • The explicit provider field in v3 YAML should be respected per the spec's resolution order (step 2: top-level provider key).
  • Built-in actors should generate YAML with the model identifier as-is (no provider prefix), and the provider field should be lowercase to match ProviderType enum values.
  • uv run agents actor run anthropic/claude-sonnet-4-20250514 hi should successfully invoke the Anthropic API.

Acceptance criteria

  • _build_from_v3 checks for an explicit provider field in the v3 YAML data before falling back to infer_provider_from_model(model).
  • _generate_builtin_actor_yaml generates model as the bare model identifier (e.g., claude-sonnet-4-20250514), not provider/model.
  • _generate_builtin_actor_yaml generates provider in lowercase (e.g., anthropic, not Anthropic).
  • uv run agents actor run anthropic/claude-sonnet-4-20250514 hi succeeds and returns a response from the Anthropic API.
  • Existing actors with / in the model string (e.g., openrouter/anthropic-claude-sonnet-4-20250514) continue to work via the inference fallback.
  • All existing tests pass.

Supporting information

  • Spec reference: line 21412 (provider/model resolution order), line 20586 (config.provider field definition)
  • Schema reference: ActorConfigSchema.provider field at actor/schema.py line 747
  • Affected files:
    • src/cleveragents/reactive/config_parser.py_build_from_v3 method
    • src/cleveragents/actor/registry.py_generate_builtin_actor_yaml method

Subtasks

  • Fix _build_from_v3 to check explicit provider field before inferring from model
  • Fix _generate_builtin_actor_yaml to use bare model identifier and lowercase provider
  • Remove @tdd_expected_fail from features/tdd_actor_run_response.feature (bug #10861 fixed as side effect)
  • Run nox (all default sessions), fix any errors
  • Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
## Metadata - **Commit Message**: `fix(actor): resolve provider from explicit field in v3 YAML before inferring from model` - **Branch**: `bugfix/m3-v3-actor-provider-resolution` ## Background and context The v3 actor YAML schema supports an explicit `provider` field (e.g., `provider: anthropic`) alongside `model` (e.g., `model: claude-sonnet-4-20250514`). The specification (line 21412) defines the resolution order for provider/model as: 1. CLI override 2. Top-level `provider` / `model` keys 3. Top-level `provider_type` / `model_id` aliases 4. v2-extracted values from `actors.<name>.config.provider` / `.model` Additionally, the v3 `ActorConfigSchema` (schema.py line 747) defines `provider` as an explicit field with description `"LLM provider (e.g. openai, anthropic)"`. ## Current behavior Two bugs prevent built-in LLM actors from working when the model name lacks a `/` separator (e.g., `claude-sonnet-4-20250514`): 1. **`_generate_builtin_actor_yaml`** (`actor/registry.py` line 128) constructs the model field as `f"{provider}/{model}"`, producing `"Anthropic/claude-sonnet-4-20250514"`. The Anthropic API expects just `"claude-sonnet-4-20250514"`, causing a 404 error. 2. **`_build_from_v3`** (`reactive/config_parser.py` line 306) always infers the provider from the model string via `infer_provider_from_model()`. Since `claude-sonnet-4-20250514` contains no `/`, this returns `"custom"` — which is not a valid `ProviderType`, causing `Unknown provider type: custom`. The net result: `uv run agents actor run anthropic/claude-sonnet-4-20250514 hi` fails with `Unexpected error: Unknown provider type: custom`. ## Expected behavior - The explicit `provider` field in v3 YAML should be respected per the spec's resolution order (step 2: top-level `provider` key). - Built-in actors should generate YAML with the model identifier as-is (no provider prefix), and the provider field should be lowercase to match `ProviderType` enum values. - `uv run agents actor run anthropic/claude-sonnet-4-20250514 hi` should successfully invoke the Anthropic API. ## Acceptance criteria - [ ] `_build_from_v3` checks for an explicit `provider` field in the v3 YAML data before falling back to `infer_provider_from_model(model)`. - [ ] `_generate_builtin_actor_yaml` generates `model` as the bare model identifier (e.g., `claude-sonnet-4-20250514`), not `provider/model`. - [ ] `_generate_builtin_actor_yaml` generates `provider` in lowercase (e.g., `anthropic`, not `Anthropic`). - [ ] `uv run agents actor run anthropic/claude-sonnet-4-20250514 hi` succeeds and returns a response from the Anthropic API. - [ ] Existing actors with `/` in the model string (e.g., `openrouter/anthropic-claude-sonnet-4-20250514`) continue to work via the inference fallback. - [ ] All existing tests pass. ## Supporting information - Spec reference: line 21412 (provider/model resolution order), line 20586 (`config.provider` field definition) - Schema reference: `ActorConfigSchema.provider` field at `actor/schema.py` line 747 - Affected files: - `src/cleveragents/reactive/config_parser.py` — `_build_from_v3` method - `src/cleveragents/actor/registry.py` — `_generate_builtin_actor_yaml` method ## Subtasks - [x] Fix `_build_from_v3` to check explicit `provider` field before inferring from model - [x] Fix `_generate_builtin_actor_yaml` to use bare model identifier and lowercase provider - [x] Remove `@tdd_expected_fail` from `features/tdd_actor_run_response.feature` (bug #10861 fixed as side effect) - [x] Run `nox` (all default sessions), fix any errors - [x] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.
hurui200320 added this to the v3.2.0 milestone 2026-04-29 08:14:21 +00:00
Author
Member

Implementation Notes

Fix 1: _generate_builtin_actor_yaml in ActorRegistry (actor/registry.py)

Problem: The model field was being constructed as f"{provider}/{model}" (e.g. "anthropic/claude-sonnet-4-20250514"). The Anthropic API expects just the bare model ID "claude-sonnet-4-20250514". Additionally, the provider field was stored with the original case from ProviderInfo.name (e.g. "Anthropic") which didn't match ProviderType enum values (lowercase).

Fix: Changed "model": f"{provider}/{model}" to "model": model and "provider": provider to "provider": provider.lower(). The description still shows provider/model for human readability. This is a minimal, targeted change that doesn't affect the canonical config_blob stored on the actor entity — only the v3 YAML text that is fed to _build_from_v3.

Fix 2: _build_from_v3 in ReactiveConfigParser (reactive/config_parser.py)

Problem: Provider was always inferred from the model string via infer_provider_from_model(model). For bare model IDs like "claude-sonnet-4-20250514" (no /), this returned "custom" — an invalid ProviderType value — causing Unknown provider type: custom when SimpleLLMAgent._resolve_llm() tried to create the LLM.

Fix: Added a check for an explicit top-level provider field in the v3 YAML data before falling back to inference. This follows the spec resolution order (step 2: top-level provider key takes precedence). The fix is: if data.get("provider") is a non-empty string, use it directly; otherwise fall back to infer_provider_from_model(model). This ensures models with / in their name (e.g. OpenRouter's openrouter/anthropic-claude-sonnet) continue to work via inference when no explicit provider field is present.

Side Effect: Issue #10861 Fixed

The TDD capture test features/tdd_actor_run_response.feature was tagged @tdd_expected_fail to capture bug #10861 (agents actor run returning no response). After Fix 1, built-in actor YAML now has:

  • model: claude-sonnet-4-20250514 (bare, correct)
  • provider: anthropic (lowercase, matches ProviderType.ANTHROPIC)

This allows _build_from_v3 to correctly resolve the provider and model, and the agents actor run command now successfully invokes the mock LLM in the test and returns the expected response. The @tdd_expected_fail tag was removed from the test.

Tests Added

features/builtin_actor_v3_yaml.feature — 3 new @tdd_issue_10926 scenarios:

  1. Built-in actor YAML uses bare model identifier without provider prefix — verifies model: claude-sonnet-4-20250514 not model: anthropic/claude-sonnet-4-20250514
  2. Built-in actor YAML has lowercase provider field — verifies provider: anthropic not provider: Anthropic
  3. Built-in actor YAML for model without slash has separate provider and model fields — validates full schema compliance

features/actor_v3_route_synthesis.feature — 3 new @tdd_issue_10926 scenarios:

  1. v3 actor with explicit provider field uses it without inferenceprovider: anthropic + model: claude-sonnet-4-20250514 → agent config has provider: anthropic
  2. v3 actor without explicit provider falls back to inference from model with slashmodel: openrouter/anthropic-claude-sonnet → inferred provider: openrouter
  3. v3 actor with model without slash and no explicit provider gets custom providermodel: gpt-4-turbo, no providerprovider: custom

Quality Gate Results

  • lint: PASS
  • typecheck: PASS (0 errors, 3 warnings from external stubs only)
  • unit_tests: PASS (671 features, 15673 scenarios, 0 failures)
  • integration_tests: PASS (1997/1997)
  • coverage_report: PASS (97.12% ≥ 97%)

PR

See PR #10930: #10930

## Implementation Notes ### Fix 1: `_generate_builtin_actor_yaml` in `ActorRegistry` (actor/registry.py) **Problem:** The `model` field was being constructed as `f"{provider}/{model}"` (e.g. `"anthropic/claude-sonnet-4-20250514"`). The Anthropic API expects just the bare model ID `"claude-sonnet-4-20250514"`. Additionally, the `provider` field was stored with the original case from `ProviderInfo.name` (e.g. `"Anthropic"`) which didn't match `ProviderType` enum values (lowercase). **Fix:** Changed `"model": f"{provider}/{model}"` to `"model": model` and `"provider": provider` to `"provider": provider.lower()`. The description still shows `provider/model` for human readability. This is a minimal, targeted change that doesn't affect the canonical `config_blob` stored on the actor entity — only the v3 YAML text that is fed to `_build_from_v3`. ### Fix 2: `_build_from_v3` in `ReactiveConfigParser` (reactive/config_parser.py) **Problem:** Provider was always inferred from the model string via `infer_provider_from_model(model)`. For bare model IDs like `"claude-sonnet-4-20250514"` (no `/`), this returned `"custom"` — an invalid `ProviderType` value — causing `Unknown provider type: custom` when `SimpleLLMAgent._resolve_llm()` tried to create the LLM. **Fix:** Added a check for an explicit top-level `provider` field in the v3 YAML data before falling back to inference. This follows the spec resolution order (step 2: top-level `provider` key takes precedence). The fix is: if `data.get("provider")` is a non-empty string, use it directly; otherwise fall back to `infer_provider_from_model(model)`. This ensures models with `/` in their name (e.g. OpenRouter's `openrouter/anthropic-claude-sonnet`) continue to work via inference when no explicit `provider` field is present. ### Side Effect: Issue #10861 Fixed The TDD capture test `features/tdd_actor_run_response.feature` was tagged `@tdd_expected_fail` to capture bug #10861 (`agents actor run` returning no response). After Fix 1, built-in actor YAML now has: - `model: claude-sonnet-4-20250514` (bare, correct) - `provider: anthropic` (lowercase, matches `ProviderType.ANTHROPIC`) This allows `_build_from_v3` to correctly resolve the provider and model, and the `agents actor run` command now successfully invokes the mock LLM in the test and returns the expected response. The `@tdd_expected_fail` tag was removed from the test. ### Tests Added **`features/builtin_actor_v3_yaml.feature`** — 3 new `@tdd_issue_10926` scenarios: 1. `Built-in actor YAML uses bare model identifier without provider prefix` — verifies `model: claude-sonnet-4-20250514` not `model: anthropic/claude-sonnet-4-20250514` 2. `Built-in actor YAML has lowercase provider field` — verifies `provider: anthropic` not `provider: Anthropic` 3. `Built-in actor YAML for model without slash has separate provider and model fields` — validates full schema compliance **`features/actor_v3_route_synthesis.feature`** — 3 new `@tdd_issue_10926` scenarios: 1. `v3 actor with explicit provider field uses it without inference` — `provider: anthropic` + `model: claude-sonnet-4-20250514` → agent config has `provider: anthropic` 2. `v3 actor without explicit provider falls back to inference from model with slash` — `model: openrouter/anthropic-claude-sonnet` → inferred `provider: openrouter` 3. `v3 actor with model without slash and no explicit provider gets custom provider` — `model: gpt-4-turbo`, no `provider` → `provider: custom` ### Quality Gate Results - lint: PASS - typecheck: PASS (0 errors, 3 warnings from external stubs only) - unit_tests: PASS (671 features, 15673 scenarios, 0 failures) - integration_tests: PASS (1997/1997) - coverage_report: PASS (97.12% ≥ 97%) ### PR See PR #10930: https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/10930
hurui200320 2026-04-30 02:20:27 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#10926
No description provided.