UAT: `AIProviderInterface` implementation does not match spec — spec defines `provider_name`, `capabilities`, `create_chat_model`, `create_embedding_model` but implementation has `name`, `model_id`, `generate_changes`, `stream_changes` #4108

New issue

Open

opened 2026-04-06 10:22:13 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 10:22:13 +00:00

Owner

Metadata

Branch: fix/providers-ai-provider-interface-spec-alignment
Commit Message: fix(providers): align AIProviderInterface with specification-defined protocol
Milestone: (none — backlog)
Parent Epic: #3365 (Epic: Additional LLM Provider Integrations — Cohere, Groq, Together AI, and Provider Abstraction)

Backlog note: This issue was discovered during autonomous UAT testing
on the LLM Provider Integration feature area. It does not block milestone
completion and has been placed in the backlog for human review and future
milestone assignment.

Bug Report

What was tested: AIProviderInterface implementation against the specification

Expected behavior (from spec, section "Actor Extensibility", line 46289-46313):

The specification defines AIProviderInterface as:

class AIProviderInterface(Protocol):
    """Protocol for LLM provider implementations."""

    @property
    def provider_name(self) -> str: ...

    @property
    def capabilities(self) -> ProviderCapabilities: ...

    def create_chat_model(
        self,
        model: str,
        temperature: float = 0.7,
        **kwargs,
    ) -> BaseChatModel: ...

    def create_embedding_model(
        self,
        model: str,
        **kwargs,
    ) -> BaseEmbeddings: ...

The spec also states: "The registry uses auto-discovery to detect installed langchain-* packages."

Actual behavior (from code analysis):

The implementation in src/cleveragents/domain/providers/ai_provider.py defines a completely different interface:

class AIProviderInterface(Protocol):
    def generate_changes(self, project, plan, contexts, actor_context=None, progress_callback=None) -> ProviderResponse: ...
    def stream_changes(self, project, plan, contexts, actor_context=None, progress_callback=None) -> Iterator[dict]: ...
    @property
    def name(self) -> str: ...
    @property
    def model_id(self) -> str: ...

Differences:

Spec	Implementation
`provider_name` property	`name` property
`capabilities` property → `ProviderCapabilities`	❌ Missing
`create_chat_model(model, temperature, **kwargs)`	❌ Missing
`create_embedding_model(model, **kwargs)`	❌ Missing
❌ Not in spec	`generate_changes()`
❌ Not in spec	`stream_changes()`
❌ Not in spec	`model_id` property

Additionally, the spec states the registry uses auto-discovery to detect installed langchain-* packages, but the implementation uses a hardcoded list of providers in PROVIDER_KEY_ATTRS.

Code location:

src/cleveragents/domain/providers/ai_provider.py — AIProviderInterface Protocol definition
src/cleveragents/providers/registry.py — ProviderRegistry class (no auto-discovery)
Spec: docs/specification.md lines 46289-46313

Steps to reproduce:

Read src/cleveragents/domain/providers/ai_provider.py
Read docs/specification.md lines 46289-46313
Compare the two interface definitions

Severity: Medium — the interface mismatch means third-party provider implementations following the spec will not be compatible with the actual codebase. The spec is the source of truth per CONTRIBUTING.md.

Note: The current generate_changes/stream_changes interface may be intentional for the current implementation phase. This issue should be reviewed by the team to determine whether the spec needs updating or the implementation needs aligning.

Subtasks

Review spec vs implementation to determine which is authoritative
If spec is authoritative: add provider_name, capabilities, create_chat_model, create_embedding_model to AIProviderInterface
If implementation is authoritative: update spec to reflect actual interface
Implement auto-discovery of installed langchain-* packages in ProviderRegistry
Add Behave scenarios for the aligned interface
Verify nox -e typecheck passes
Run nox (all default sessions)

Definition of Done

AIProviderInterface matches the spec definition
ProviderRegistry uses auto-discovery for langchain-* packages (or spec is updated)
All existing provider tests continue to pass
New Behave scenarios cover the aligned interface
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/providers-ai-provider-interface-spec-alignment` - **Commit Message**: `fix(providers): align AIProviderInterface with specification-defined protocol` - **Milestone**: *(none — backlog)* - **Parent Epic**: #3365 (Epic: Additional LLM Provider Integrations — Cohere, Groq, Together AI, and Provider Abstraction) > **Backlog note:** This issue was discovered during autonomous UAT testing > on the LLM Provider Integration feature area. It does not block milestone > completion and has been placed in the backlog for human review and future > milestone assignment. ## Bug Report **What was tested:** `AIProviderInterface` implementation against the specification **Expected behavior (from spec, section "Actor Extensibility", line 46289-46313):** The specification defines `AIProviderInterface` as: ```python class AIProviderInterface(Protocol): """Protocol for LLM provider implementations.""" @property def provider_name(self) -> str: ... @property def capabilities(self) -> ProviderCapabilities: ... def create_chat_model( self, model: str, temperature: float = 0.7, **kwargs, ) -> BaseChatModel: ... def create_embedding_model( self, model: str, **kwargs, ) -> BaseEmbeddings: ... ``` The spec also states: "The registry uses auto-discovery to detect installed `langchain-*` packages." **Actual behavior (from code analysis):** The implementation in `src/cleveragents/domain/providers/ai_provider.py` defines a completely different interface: ```python class AIProviderInterface(Protocol): def generate_changes(self, project, plan, contexts, actor_context=None, progress_callback=None) -> ProviderResponse: ... def stream_changes(self, project, plan, contexts, actor_context=None, progress_callback=None) -> Iterator[dict]: ... @property def name(self) -> str: ... @property def model_id(self) -> str: ... ``` **Differences:** | Spec | Implementation | |------|----------------| | `provider_name` property | `name` property | | `capabilities` property → `ProviderCapabilities` | ❌ Missing | | `create_chat_model(model, temperature, **kwargs)` | ❌ Missing | | `create_embedding_model(model, **kwargs)` | ❌ Missing | | ❌ Not in spec | `generate_changes()` | | ❌ Not in spec | `stream_changes()` | | ❌ Not in spec | `model_id` property | Additionally, the spec states the registry uses **auto-discovery** to detect installed `langchain-*` packages, but the implementation uses a hardcoded list of providers in `PROVIDER_KEY_ATTRS`. **Code location:** - `src/cleveragents/domain/providers/ai_provider.py` — `AIProviderInterface` Protocol definition - `src/cleveragents/providers/registry.py` — `ProviderRegistry` class (no auto-discovery) - Spec: `docs/specification.md` lines 46289-46313 **Steps to reproduce:** 1. Read `src/cleveragents/domain/providers/ai_provider.py` 2. Read `docs/specification.md` lines 46289-46313 3. Compare the two interface definitions **Severity:** Medium — the interface mismatch means third-party provider implementations following the spec will not be compatible with the actual codebase. The spec is the source of truth per CONTRIBUTING.md. **Note:** The current `generate_changes`/`stream_changes` interface may be intentional for the current implementation phase. This issue should be reviewed by the team to determine whether the spec needs updating or the implementation needs aligning. ## Subtasks - [ ] Review spec vs implementation to determine which is authoritative - [ ] If spec is authoritative: add `provider_name`, `capabilities`, `create_chat_model`, `create_embedding_model` to `AIProviderInterface` - [ ] If implementation is authoritative: update spec to reflect actual interface - [ ] Implement auto-discovery of installed `langchain-*` packages in `ProviderRegistry` - [ ] Add Behave scenarios for the aligned interface - [ ] Verify `nox -e typecheck` passes - [ ] Run `nox` (all default sessions) ## Definition of Done - [ ] `AIProviderInterface` matches the spec definition - [ ] `ProviderRegistry` uses auto-discovery for `langchain-*` packages (or spec is updated) - [ ] All existing provider tests continue to pass - [ ] New Behave scenarios cover the aligned interface - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 10:22:22 +00:00

#3365 Epic: Additional LLM Provider Integrations — Cohere, Groq, Together AI, and Provider Abstraction

freemo referenced this issue

2026-04-06 10:32:27 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 10:35:53 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775