cleveragents/cleveragents-core

Fork 3

UAT: `ContextService.analyze_context()` raises `ValueError` when no LLM is configured — context analysis unusable without explicit LLM #3247

New issue

Open

opened 2026-04-05 08:28:02 +00:00 by freemo · 1 comment

freemo commented

2026-04-05 08:28:02 +00:00

Owner

Metadata

Branch: fix/context-service-llm-resolution
Commit Message: fix(context): resolve LLM from configured provider in ContextAnalysisAgent instead of raising ValueError
Milestone: (none — backlog)
Parent Epic: #359

Background and Context

The ACMS spec requires that context analysis produces meaningful summaries. ContextService.analyze_context() is the method that performs this analysis using a LangGraph workflow. However, the implementation raises a ValueError when no LLM is provided, making context analysis completely unusable in the default configuration.

What Was Tested

Code-level analysis of:

src/cleveragents/application/services/context_service.py
src/cleveragents/agents/graphs/context_analysis.py

Expected Behavior (from spec)

The ACMS spec (docs/specification.md, ACMS section) requires that context analysis produces meaningful summaries. The ContextService.analyze_context() method should be callable without requiring callers to explicitly provide an LLM instance — it should use the configured provider from the application settings.

Actual Behavior (from code)

In src/cleveragents/agents/graphs/context_analysis.py, the ContextAnalysisAgent.__init__() method (lines ~120–127):

if llm is None:
    raise ValueError(
        "No LLM provider configured. "
        "Set provider credentials or enable core.mock_providers for testing."
    )

And in src/cleveragents/application/services/context_service.py, analyze_context() calls:

agent = self._get_context_agent(llm)  # llm=None by default

This means any call to ContextService.analyze_context() without an explicit llm parameter will raise a ValueError immediately, before any analysis can occur. The get_context_summary(), get_context_dependencies(), and get_relevant_files() convenience methods all call analyze_context() and are therefore also broken.

Steps to Reproduce

Set up a project with context files.
Call context_service.analyze_context(project) without providing an LLM.
Observe ValueError: No LLM provider configured... is raised immediately.

Code Locations

src/cleveragents/agents/graphs/context_analysis.py, ContextAnalysisAgent.__init__(): raises ValueError when llm is None
src/cleveragents/application/services/context_service.py, _get_context_agent(): passes llm=None by default
src/cleveragents/application/services/context_service.py, analyze_context(): calls _get_context_agent(llm) with llm=None as default

Impact

Context analysis (summaries, dependency extraction, relevance scoring) is completely non-functional in the default configuration. Any feature that relies on ContextService.analyze_context() will fail with a ValueError unless callers explicitly provide an LLM instance.

Proposed Fix

ContextAnalysisAgent should resolve the LLM from the application's configured provider when llm is None, rather than raising a ValueError. Alternatively, ContextService should resolve the LLM from the DI container before calling _get_context_agent().

Subtasks

Investigate the DI container / application settings API for resolving the configured LLM provider
Update ContextAnalysisAgent.__init__() to resolve the LLM from the configured provider when llm is None, instead of raising ValueError
Update ContextService._get_context_agent() / analyze_context() to resolve the LLM from the DI container if not explicitly supplied
Tests (Behave): Add scenario — calling analyze_context() without an explicit LLM uses the configured provider successfully
Tests (Behave): Add scenario — calling analyze_context() without an explicit LLM and with no provider configured raises a clear, actionable error
Tests (Robot): Add integration test for ContextService.analyze_context() in default configuration
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
ContextService.analyze_context() (and all convenience methods that call it) works correctly in the default configuration without requiring callers to supply an explicit LLM instance.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.4.0 (M5: ACMS v1 + Context Scaling). It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/context-service-llm-resolution` - **Commit Message**: `fix(context): resolve LLM from configured provider in ContextAnalysisAgent instead of raising ValueError` - **Milestone**: *(none — backlog)* - **Parent Epic**: #359 ## Background and Context The ACMS spec requires that context analysis produces meaningful summaries. `ContextService.analyze_context()` is the method that performs this analysis using a LangGraph workflow. However, the implementation raises a `ValueError` when no LLM is provided, making context analysis completely unusable in the default configuration. ## What Was Tested Code-level analysis of: - `src/cleveragents/application/services/context_service.py` - `src/cleveragents/agents/graphs/context_analysis.py` ## Expected Behavior (from spec) The ACMS spec (`docs/specification.md`, ACMS section) requires that context analysis produces meaningful summaries. The `ContextService.analyze_context()` method should be callable without requiring callers to explicitly provide an LLM instance — it should use the configured provider from the application settings. ## Actual Behavior (from code) In `src/cleveragents/agents/graphs/context_analysis.py`, the `ContextAnalysisAgent.__init__()` method (lines ~120–127): ```python if llm is None: raise ValueError( "No LLM provider configured. " "Set provider credentials or enable core.mock_providers for testing." ) ``` And in `src/cleveragents/application/services/context_service.py`, `analyze_context()` calls: ```python agent = self._get_context_agent(llm) # llm=None by default ``` This means any call to `ContextService.analyze_context()` without an explicit `llm` parameter will raise a `ValueError` immediately, before any analysis can occur. The `get_context_summary()`, `get_context_dependencies()`, and `get_relevant_files()` convenience methods all call `analyze_context()` and are therefore also broken. ## Steps to Reproduce 1. Set up a project with context files. 2. Call `context_service.analyze_context(project)` without providing an LLM. 3. Observe `ValueError: No LLM provider configured...` is raised immediately. ## Code Locations - `src/cleveragents/agents/graphs/context_analysis.py`, `ContextAnalysisAgent.__init__()`: raises `ValueError` when `llm is None` - `src/cleveragents/application/services/context_service.py`, `_get_context_agent()`: passes `llm=None` by default - `src/cleveragents/application/services/context_service.py`, `analyze_context()`: calls `_get_context_agent(llm)` with `llm=None` as default ## Impact Context analysis (summaries, dependency extraction, relevance scoring) is completely non-functional in the default configuration. Any feature that relies on `ContextService.analyze_context()` will fail with a `ValueError` unless callers explicitly provide an LLM instance. ## Proposed Fix `ContextAnalysisAgent` should resolve the LLM from the application's configured provider when `llm is None`, rather than raising a `ValueError`. Alternatively, `ContextService` should resolve the LLM from the DI container before calling `_get_context_agent()`. ## Subtasks - [ ] Investigate the DI container / application settings API for resolving the configured LLM provider - [ ] Update `ContextAnalysisAgent.__init__()` to resolve the LLM from the configured provider when `llm is None`, instead of raising `ValueError` - [ ] Update `ContextService._get_context_agent()` / `analyze_context()` to resolve the LLM from the DI container if not explicitly supplied - [ ] Tests (Behave): Add scenario — calling `analyze_context()` without an explicit LLM uses the configured provider successfully - [ ] Tests (Behave): Add scenario — calling `analyze_context()` without an explicit LLM and with no provider configured raises a clear, actionable error - [ ] Tests (Robot): Add integration test for `ContextService.analyze_context()` in default configuration - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - `ContextService.analyze_context()` (and all convenience methods that call it) works correctly in the default configuration without requiring callers to supply an explicit LLM instance. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.4.0 (M5: ACMS v1 + Context Scaling). It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added a new dependency

2026-04-05 08:34:28 +00:00

#359 Epic: ACMS v1 + Context Scaling (M5)

freemo added this to the v3.4.0 milestone

2026-04-05 08:42:34 +00:00

freemo commented

2026-04-05 08:44:48 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Backlog
Milestone: v3.4.0 (assigned — ACMS context analysis is M5 scope)
MoSCoW: Should Have — the v3.4.0 acceptance criteria require "Context analysis produces meaningful summaries"; a ValueError when no LLM is configured makes context analysis unusable in default configuration
Parent Epic: #396 (ACMS Context Pipeline)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Backlog - **Milestone**: v3.4.0 (assigned — ACMS context analysis is M5 scope) - **MoSCoW**: Should Have — the v3.4.0 acceptance criteria require "Context analysis produces meaningful summaries"; a ValueError when no LLM is configured makes context analysis unusable in default configuration - **Parent Epic**: #396 (ACMS Context Pipeline) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo modified the milestone from v3.4.0 to v3.7.0

2026-04-05 08:47:16 +00:00

freemo referenced this issue

2026-04-05 08:49:48 +00:00

[CA-AUTO] Project Owner — project-owner-1 — Session Tracker #2793

freemo referenced this issue

2026-04-05 09:01:27 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3094

freemo referenced this issue

2026-04-05 09:08:33 +00:00

[CA-AUTO] Backlog Groomer — Session Tracker — groomer-1 — 2026-04-05 #3182

freemo referenced this issue