Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5b73be9f43 | |||
| f179b57e18 | |||
| 90b06e6308 | |||
| beaeac165d | |||
| 2240a028c8 | |||
| 236499f086 | |||
| 60671306ea | |||
| 79ecaea18f | |||
| 69b7cd3cb2 | |||
| 908e53ea85 | |||
| 7164b04019 | |||
| 988a169831 | |||
| 39175dd284 | |||
| 81935a9cf7 |
@@ -415,10 +415,45 @@ For each finding, invoke `ca-new-issue-creator` with:
|
||||
|
||||
### Duplicate Avoidance
|
||||
|
||||
Before filing any issue:
|
||||
1. Search Forgejo for existing issues with "TEST-INFRA:" prefix
|
||||
2. Check for similar titles/descriptions
|
||||
3. If potential duplicate found, skip
|
||||
> **CRITICAL: When in doubt, SKIP — it is better to miss an improvement
|
||||
> suggestion than to create noise that wastes groomer and implementor time.**
|
||||
|
||||
Before filing ANY issue, you MUST perform rigorous duplicate checking:
|
||||
|
||||
1. **Extract keywords**: From your proposed issue title, extract 3-5 key
|
||||
technical terms (e.g., "parallelize", "E2E", "coverage", "Docker",
|
||||
"nox", "cache").
|
||||
|
||||
2. **Search by keywords**: Search Forgejo for open AND closed issues containing ANY
|
||||
of those key terms. Use the search API, not just the "TEST-INFRA:"
|
||||
prefix.
|
||||
|
||||
3. **Compare semantically**: For each search result, compare your proposed
|
||||
improvement with the existing issue's description. Two issues are
|
||||
**duplicates** if they propose the same optimization for the same
|
||||
component, even if the wording differs. Examples of duplicates:
|
||||
- "Parallelize E2E tests" and "Run E2E tests in parallel"
|
||||
- "Create custom Docker image for CI" and "Pre-built CI base image"
|
||||
- "Cache nox environments" and "Persist nox virtualenvs between runs"
|
||||
|
||||
4. **Check all prefixes**: Search for issues with `TEST-INFRA:`, `BUG-HUNT:`,
|
||||
and `UAT:` prefixes — other agents may have already identified the same
|
||||
improvement opportunity from a different angle.
|
||||
|
||||
5. **If ANY existing issue proposes the same or substantially similar
|
||||
improvement**: **SKIP** — do not file. It is far better to miss one
|
||||
improvement suggestion than to file the 7th duplicate of the same idea.
|
||||
Previous sessions created 48+ TEST-INFRA issues with significant overlap
|
||||
across 8 topic clusters.
|
||||
|
||||
6. **Post-filing verification**: After filing an issue, wait 5 seconds and
|
||||
re-check for duplicates (another parallel worker may have filed the same
|
||||
issue simultaneously). If a duplicate appeared, close your issue as a
|
||||
duplicate of the earlier one.
|
||||
|
||||
When filing, include a `### Duplicate Check` section in the issue body
|
||||
listing the search queries used, result counts, and your justification for
|
||||
why this is not a duplicate.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+37
-8
@@ -50,6 +50,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
- `agents actor context clear` command to reset actor message history and
|
||||
state while preserving the underlying context directory via `ContextManager`
|
||||
(#6370).
|
||||
- **Quick Start Guide** (PR #9245): Added `docs/quickstart.md` with an end-to-end quick start guide covering prerequisites, installation, project creation, resource registration, plan/apply workflow, and troubleshooting. Updated `mkdocs.yml` navigation to include the Quick Start page.
|
||||
|
||||
- **Plan checkpoint management CLI commands** (#8683): Added `agents plan checkpoint-list <plan-id>` and `agents plan checkpoint-delete <checkpoint-id>` commands. Listing output now highlights checkpoint ID, type, created timestamp, reason, phase, and decision linkage with a concise field summary footer across rich/table/json/yaml formats. Deletion supports batch IDs, interactive confirmation (skip with `--yes`), and structured JSON/YAML responses for automation-friendly scripting.
|
||||
- **Invariant Remove CLI Command** (#8530): Implemented `agents invariant remove <id>` command that soft-deletes an invariant by ID. The command displays a confirmation prompt before removal (bypassable with `--yes`/`-y`), outputs the removed invariant ID on success, and shows a clear error message when the invariant ID does not exist. Supports `--format` flag for JSON and YAML output. Full BDD test coverage and Robot Framework integration tests included.
|
||||
@@ -78,6 +79,42 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **git_tools._get_base_env() TOCTOU Race Condition** (#7619): Fixed a
|
||||
Time-Of-Check-To-Time-Of-Use race condition in `git_tools._get_base_env()`
|
||||
where two concurrent threads could both observe `_BASE_ENV is None`, both
|
||||
snapshot `os.environ`, and write potentially different snapshots. The fix
|
||||
adds a module-level `_BASE_ENV_LOCK: threading.Lock` and replaces the bare
|
||||
`if _BASE_ENV is None` assignment with double-checked locking: the outer
|
||||
check keeps the warm-cache path lock-free; the inner check inside
|
||||
`with _BASE_ENV_LOCK` prevents duplicate initialisation on the very first
|
||||
concurrent call. Three new BDD scenarios in `features/git_tools.feature`
|
||||
(with step definitions in
|
||||
`features/steps/git_tools_thread_safety_steps.py`) verify caching identity,
|
||||
content correctness, and thread safety under 20 concurrent threads.
|
||||
|
||||
- **Unified provider factory: eliminate divergence between `create_llm()` and `create_ai_provider()`** (#10949):
|
||||
Introduced `_create_provider_instance()` as the single internal factory so that
|
||||
both public methods delegate to one place. Creating a new provider now
|
||||
requires changes in exactly one method.
|
||||
- **Fixed API key regression**: the unified factory now explicitly passes
|
||||
the validated API key to all LangChain constructors (OpenAI, Anthropic,
|
||||
Google / Gemini, Azure, Groq, Together, Cohere, and OpenRouter). Users
|
||||
who configure providers via `CLEVERAGENTS_`-prefixed variables are no
|
||||
longer silently failed when LangChain falls back to raw environment
|
||||
variable lookup. Pre-validated keys are forwarded through the
|
||||
`api_key` kwarg to avoid a second settings lookup in the factory
|
||||
closure. (Closes #10949)
|
||||
- **Fixed mock provider accessibility in production**: `ProviderType.MOCK`
|
||||
is now gated by the `CLEVERAGENTS_ALLOW_MOCK_PROVIDER=true` sentinel
|
||||
environment variable. Without this flag, both `create_llm()` and
|
||||
`create_ai_provider()` raise `ValueError` when MOCK is requested,
|
||||
preventing accidental or malicious use of the fake LLM in production.
|
||||
`resolve_provider_by_name("mock")` now also respects the guard and
|
||||
`is_provider_configured(ProviderType.MOCK)` returns `True` as expected.
|
||||
- **Fixed type annotation**: `create_llm()` now declares `**kwargs: Any`
|
||||
instead of `**kwargs: object`, restoring correct Pyright inference for
|
||||
forwarded keyword arguments.
|
||||
|
||||
- **`create_llm()` raises `Unsupported provider type: openrouter`** (#10948): Fixed
|
||||
`ProviderRegistry._create_provider_llm()` missing an `OPENROUTER` branch, which
|
||||
caused `agents actor run openrouter/<model>` to fail with `ValueError`. Added a
|
||||
@@ -87,14 +124,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
`default_headers` kwarg with automatic string coercion for non-string keys/values.
|
||||
|
||||
- **LoadingThrobber Widget Restored** (#6357): Restored `LoadingThrobber` widget
|
||||
(`src/cleveragents/tui/widgets/throbber.py`) and its Robot Framework integration
|
||||
tests (`robot/tui_throbber.robot`) that were missing from master. Also restored
|
||||
supporting modules `src/cleveragents/tui/quotes.py` and
|
||||
`src/cleveragents/tui/data/throbber_quotes.txt`. Fixed the `Throbber Rejects
|
||||
Invalid Styles` integration test by using `Fix Python Indentation` to correctly
|
||||
reconstruct indentation stripped by Robot Framework's `Catenate` keyword.
|
||||
Narrowed exception handling in `throbber.py` to specific types
|
||||
(`ImportError`, `AttributeError`) per coding standards.
|
||||
|
||||
- **Built-in actors v3 YAML format** (#10883): Fixed `agents actor run` failing for
|
||||
built-in actors (e.g., `openai/gpt-4`, `anthropic/claude-3-opus`) due to missing
|
||||
|
||||
@@ -25,4 +25,7 @@ Below are some of the specific details of various contributions.
|
||||
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
|
||||
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
|
||||
* HAL 9000 has contributed the file edit encoding parameter fix (PR #8258 / issue #7559).
|
||||
* HAL 9000 has contributed the architecture-pool-supervisor milestone assignment feature (PR #8188 / issue #7521): added `forgejo_update_pull_request` permission and documented the PR workflow for major spec changes, enabling automatic milestone assignment for specification PRs.
|
||||
* HAL 9000 has contributed the git worktree TOCTOU race condition fix (PR #8178 / issue #7507): replaced the unsafe mkdtemp() + rmdir() pattern with a parent-directory approach to eliminate the race window in concurrent git worktree operations.
|
||||
* HAL 9000 has contributed the git_tools TOCTOU race condition fix (PR #8255 / issue #7619): eliminated the Time-Of-Check-To-Time-Of-Use race in `_get_base_env()` by adding double-checked locking with a module-level `threading.Lock`, preventing concurrent threads from writing conflicting environment snapshots.
|
||||
* HAL 9000 has contributed comprehensive milestone documentation for v3.6.0 (Advanced Concepts & Deferred Features) and v3.7.0 (TUI Implementation) (PR #9903): split into sub-documents covering context strategies, LLM backends, resource types, A2A rename, container tool execution, scope chain resolution, cost/safety budgets, E2E workflow tests, code review examples, plugin architecture, TUI layout, persona system, reference/command input, session management, configuration, and TuiMaterializer integration.
|
||||
|
||||
@@ -100,6 +100,130 @@ nodes, and any warnings.
|
||||
|
||||
---
|
||||
|
||||
## `InvariantReconciliationActor`
|
||||
|
||||
**Introduced:** v3.8.0 | **Module:** `cleveragents.actor.reconciliation`
|
||||
|
||||
The built-in `InvariantReconciliationActor` (`builtin/invariant-reconciliation`)
|
||||
is invoked by `PlanLifecycleService` when a plan enters the Strategize phase.
|
||||
This gate must succeed before Strategize work begins; the lifecycle service
|
||||
persists the reconciled invariant view and reuses it for later Execute or Apply
|
||||
transitions instead of re-running the actor. The actor collects invariants
|
||||
from four scopes, resolves conflicts using specificity-based precedence,
|
||||
records `invariant_enforced` decisions, and returns a reconciled
|
||||
`InvariantSet`.
|
||||
|
||||
See [ADR-016 — Invariant System](../adr/ADR-016-invariant-system.md) for the
|
||||
design rationale behind invariant reconciliation.
|
||||
|
||||
```python
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
|
||||
actor = InvariantReconciliationActor(
|
||||
invariant_service=container.invariant_service(),
|
||||
decision_service=container.decision_service(),
|
||||
)
|
||||
result = actor.run(plan_id="01HXYZ...", project_name="local/my-app")
|
||||
# result.reconciled_set — effective InvariantSet
|
||||
# result.conflicts — list[ConflictRecord] with resolution details
|
||||
# result.enforced_decision_ids — list of ULID strings
|
||||
```
|
||||
|
||||
### Reconciliation Algorithm
|
||||
|
||||
1. Collect invariants from four scopes: **global**, **project**, **action**, **plan**.
|
||||
2. Group by normalised text (case-insensitive, stripped).
|
||||
3. Detect conflicts between invariants at different scopes.
|
||||
4. Resolve using specificity: `plan > action > project > global`.
|
||||
Exception: `non_overridable` global invariants always win.
|
||||
5. Record an `invariant_enforced` decision for each active invariant.
|
||||
6. Return a reconciled `InvariantSet`.
|
||||
|
||||
### `InvariantReconciliationActor` Methods
|
||||
|
||||
| Method signature | Description |
|
||||
|------------------|-------------|
|
||||
| `collect_invariants(*, plan_id: str \| None = None, project_name: str \| None = None, action_name: str \| None = None) -> ScopeInvariants` | Collect invariants from all four scopes and return them as a `ScopeInvariants` container |
|
||||
| `run(*, plan_id: str, project_name: str \| None = None, action_name: str \| None = None, parent_decision_id: str \| None = None) -> ReconciliationResult` | Execute the reconciliation lifecycle (collect → reconcile → record) and return a `ReconciliationResult` |
|
||||
|
||||
### `ReconciliationResult`
|
||||
|
||||
```python
|
||||
from cleveragents.actor.reconciliation import ReconciliationResult
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReconciliationResult:
|
||||
reconciled_set: InvariantSet # effective invariants
|
||||
conflicts: list[ConflictRecord] # detected conflicts with resolution details
|
||||
enforced_decision_ids: list[str] # ULIDs of invariant_enforced decisions
|
||||
```
|
||||
|
||||
### `ConflictRecord`
|
||||
|
||||
```python
|
||||
from cleveragents.actor.reconciliation import ConflictRecord
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ConflictRecord:
|
||||
key: str # normalised invariant text used for grouping
|
||||
winner: Invariant # invariant that prevailed
|
||||
losers: list[Invariant] # invariants that were overridden
|
||||
reason: str # human-readable explanation of the resolution
|
||||
```
|
||||
|
||||
### `ScopeInvariants`
|
||||
|
||||
Convenience container grouping invariants by scope tier:
|
||||
|
||||
```python
|
||||
from cleveragents.actor.reconciliation import ScopeInvariants
|
||||
|
||||
scope_invs = ScopeInvariants(
|
||||
global_invariants=[...],
|
||||
project_invariants=[...],
|
||||
action_invariants=[...],
|
||||
plan_invariants=[...],
|
||||
)
|
||||
all_invs = scope_invs.all_invariants() # flat list across all scopes
|
||||
```
|
||||
|
||||
### `reconcile_invariants` (standalone function)
|
||||
|
||||
```python
|
||||
from cleveragents.actor.reconciliation import reconcile_invariants
|
||||
|
||||
reconciled, conflicts = reconcile_invariants(scope_invariants)
|
||||
# reconciled — list[Invariant] (winners only)
|
||||
# conflicts — list[ConflictRecord]
|
||||
```
|
||||
|
||||
### Constructing the actor via dependency injection
|
||||
|
||||
`InvariantReconciliationActor` is instantiated directly with application
|
||||
services resolved from the DI container:
|
||||
|
||||
```python
|
||||
from cleveragents.application.container import Container
|
||||
from cleveragents.actor.reconciliation import InvariantReconciliationActor
|
||||
|
||||
container = Container()
|
||||
actor = InvariantReconciliationActor(
|
||||
invariant_service=container.invariant_service(),
|
||||
decision_service=container.decision_service(),
|
||||
)
|
||||
```
|
||||
|
||||
### Failure Behaviour
|
||||
|
||||
Reconciliation failures block the phase transition with
|
||||
`ReconciliationBlockedError` and emit an `INVARIANT_VIOLATED` event. The
|
||||
exception is raised by `PlanLifecycleService`, importable via
|
||||
`from cleveragents.application.services.plan_lifecycle_service import ReconciliationBlockedError`.
|
||||
Post-correction reconciliation runs via `CORRECTION_APPLIED` event subscription
|
||||
(best-effort; does not block correction completion).
|
||||
|
||||
---
|
||||
|
||||
## Example: Loading and Compiling an Actor
|
||||
|
||||
```python
|
||||
|
||||
@@ -72,7 +72,7 @@ gate features (e.g., tool calls, vision inputs) at runtime.
|
||||
| Cohere | ✓ | ✓ | ✗ | 128 000 | ✗ |
|
||||
| Groq | ✓ | ✓ | ✗ | 32 000 | ✓ |
|
||||
| Together | ✓ | ✓ | ✗ | 32 000 | ✗ |
|
||||
| Mock | ✓ | ✗ | ✗ | 4 096 | ✗ |
|
||||
| Mock | ✗ | ✗ | ✗ | 4 096 | ✗ |
|
||||
|
||||
---
|
||||
|
||||
@@ -175,9 +175,11 @@ unknown.
|
||||
|
||||
#### `create_ai_provider(provider_type=None, model_id=None, max_retries=3) → AIProviderInterface`
|
||||
|
||||
Creates an `AIProviderInterface` implementation. For most providers this
|
||||
returns a `LangChainChatProvider`; Google and OpenRouter return their own
|
||||
specialized implementations.
|
||||
Creates an `AIProviderInterface` implementation. **All** providers are now
|
||||
wrapped uniformly in `LangChainChatProvider`; the raw LLM instantiation is
|
||||
delegated to `_create_provider_instance()` internally. Previously, some
|
||||
providers (Google, OpenRouter) returned specialised adapter classes, but the
|
||||
unified factory now handles every provider type consistently.
|
||||
|
||||
```python
|
||||
provider = registry.create_ai_provider("openai", model_id="gpt-4o-mini")
|
||||
@@ -266,6 +268,7 @@ provider = resolve_provider_by_name("anthropic")
|
||||
| `GROQ_API_KEY` | Groq credentials |
|
||||
| `TOGETHER_API_KEY` | Together AI credentials |
|
||||
| `CLEVERAGENTS_TESTING_USE_MOCK_AI` | Force mock provider in tests |
|
||||
| `CLEVERAGENTS_ALLOW_MOCK_PROVIDER` | Enable MOCK provider for testing (set to true) |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
# Devcontainer Auto-Discovery
|
||||
|
||||
**Package:** `cleveragents.resource.handlers.discovery`
|
||||
**Introduced:** v3.8.0 (issue #2615)
|
||||
|
||||
This module guide covers the devcontainer auto-discovery system, which
|
||||
automatically detects `devcontainer.json` configurations when a
|
||||
`git-checkout` or `fs-directory` resource is linked to a project.
|
||||
It supports both root-level configurations and named configurations
|
||||
for monorepo projects.
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
When a `git-checkout` or `fs-directory` resource is registered, the
|
||||
`discover_devcontainers()` function scans the resource location for
|
||||
`devcontainer.json` files. Each valid configuration is returned as a
|
||||
`DevcontainerDiscoveryResult`, which the resource registry uses to
|
||||
create child `devcontainer-instance` resources automatically.
|
||||
|
||||
---
|
||||
|
||||
## Scan Paths
|
||||
|
||||
The scanner checks three categories of paths:
|
||||
|
||||
| Category | Path | `config_name` |
|
||||
|----------|------|---------------|
|
||||
| Root-level (default) | `.devcontainer/devcontainer.json` | `None` |
|
||||
| Root-level (flat) | `.devcontainer.json` | `None` |
|
||||
| Named configuration | `.devcontainer/<name>/devcontainer.json` | `"<name>"` |
|
||||
|
||||
Named configurations are discovered by iterating one subdirectory level
|
||||
inside `.devcontainer/`. Each subdirectory that contains a
|
||||
`devcontainer.json` produces a separate result with `config_name` set to
|
||||
the subdirectory name (e.g. `"api"`, `"frontend"`).
|
||||
|
||||
---
|
||||
|
||||
## Key Classes and Functions
|
||||
|
||||
### `DevcontainerDiscoveryResult`
|
||||
|
||||
```python
|
||||
from cleveragents.resource.handlers.discovery import DevcontainerDiscoveryResult
|
||||
|
||||
result = DevcontainerDiscoveryResult(
|
||||
config_path=Path("/workspace/.devcontainer/api/devcontainer.json"),
|
||||
config_data={"name": "API Dev Container", "image": "mcr.microsoft.com/devcontainers/python:3.12"},
|
||||
parent_location="/workspace",
|
||||
config_name="api",
|
||||
)
|
||||
```
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `config_path` | `Path` | Absolute path to the `devcontainer.json` file |
|
||||
| `config_data` | `dict[str, object]` | Parsed JSON content of the devcontainer config |
|
||||
| `parent_location` | `str` | Filesystem path of the parent resource |
|
||||
| `config_name` | `str \| None` | Named configuration identifier, or `None` for root-level configs |
|
||||
|
||||
---
|
||||
|
||||
### `discover_devcontainers`
|
||||
|
||||
```python
|
||||
from cleveragents.resource.handlers.discovery import discover_devcontainers
|
||||
|
||||
results = discover_devcontainers(
|
||||
resource_location="/workspace",
|
||||
resource_type="git-checkout",
|
||||
)
|
||||
for result in results:
|
||||
print(result.config_name, result.config_path)
|
||||
```
|
||||
|
||||
Scans a resource location for all valid devcontainer configurations.
|
||||
Invalid JSON files are skipped with a warning log rather than raising.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `resource_location` | `str` | Filesystem path of the parent resource |
|
||||
| `resource_type` | `str` | Type name of the parent resource |
|
||||
|
||||
**Returns:** `list[DevcontainerDiscoveryResult]` — one entry per valid config.
|
||||
|
||||
**Raises:**
|
||||
- `ValueError` — if `resource_location` is empty or `resource_type` is empty/blank.
|
||||
|
||||
---
|
||||
|
||||
### `is_trigger_type`
|
||||
|
||||
```python
|
||||
from cleveragents.resource.handlers.discovery import is_trigger_type
|
||||
|
||||
is_trigger_type("git-checkout") # True
|
||||
is_trigger_type("sqlite") # False
|
||||
```
|
||||
|
||||
Returns `True` if the given resource type triggers devcontainer auto-discovery.
|
||||
Currently only `"git-checkout"` and `"fs-directory"` are trigger types.
|
||||
|
||||
---
|
||||
|
||||
## Monorepo Example
|
||||
|
||||
Given a monorepo with the following layout:
|
||||
|
||||
```
|
||||
/workspace/
|
||||
├── .devcontainer/
|
||||
│ ├── api/
|
||||
│ │ └── devcontainer.json → config_name="api"
|
||||
│ └── frontend/
|
||||
│ └── devcontainer.json → config_name="frontend"
|
||||
└── src/
|
||||
```
|
||||
|
||||
`discover_devcontainers("/workspace", "git-checkout")` returns two results:
|
||||
|
||||
```python
|
||||
results = discover_devcontainers("/workspace", "git-checkout")
|
||||
# results[0].config_name == "api"
|
||||
# results[1].config_name == "frontend"
|
||||
```
|
||||
|
||||
Each result produces a distinct `devcontainer-instance` resource in the
|
||||
registry, allowing plans to target individual containers by name.
|
||||
|
||||
---
|
||||
|
||||
## Root-Level Example
|
||||
|
||||
For a single-container project:
|
||||
|
||||
```
|
||||
/workspace/
|
||||
├── .devcontainer/
|
||||
│ └── devcontainer.json → config_name=None
|
||||
└── src/
|
||||
```
|
||||
|
||||
```python
|
||||
results = discover_devcontainers("/workspace", "git-checkout")
|
||||
# results[0].config_name is None
|
||||
```
|
||||
|
||||
Root-level configs retain `config_name=None` for full backward compatibility.
|
||||
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Non-trigger types return an empty list.** Calling `discover_devcontainers()`
|
||||
with a resource type that is not `"git-checkout"` or `"fs-directory"` returns
|
||||
`[]` immediately without scanning the filesystem.
|
||||
- **Invalid JSON is silently skipped.** Files that cannot be parsed as JSON
|
||||
objects are logged at `WARNING` level and excluded from results.
|
||||
- **Only one subdirectory level is scanned.** Named configurations must be
|
||||
directly inside `.devcontainer/<name>/devcontainer.json`. Deeper nesting
|
||||
is not supported.
|
||||
- **Sorted order.** Named configurations are returned in lexicographic order
|
||||
of their subdirectory names (via `sorted(dc_dir.iterdir())`).
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Resource System API](../api/resource.md) — `ResourceHandler` protocol and built-in handlers
|
||||
- [ADR-043 Devcontainer Integration](../adr/ADR-043-devcontainer-integration.md) — design rationale
|
||||
- [Resource System](../api/resource.md#devcontainerhandler) — `DevcontainerHandler` CRUD operations
|
||||
@@ -0,0 +1,45 @@
|
||||
# Quick Start Guide
|
||||
|
||||
This quick start guide will walk you through creating a new project, registering a resource, running a plan, and applying changes with CleverAgents.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.11+ and virtualenv
|
||||
- Git
|
||||
- A working CleverAgents installation (see development/testing.md for details)
|
||||
|
||||
## Install (local development)
|
||||
|
||||
```bash
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e .[dev]
|
||||
```
|
||||
|
||||
## Create a new project
|
||||
|
||||
```bash
|
||||
# Create a new directory for your project
|
||||
mkdir my-project && cd my-project
|
||||
# Initialize a CleverAgents project (example command)
|
||||
cleveragents init --name my-project
|
||||
```
|
||||
|
||||
## Register a resource
|
||||
|
||||
Create a resource file under `resources/` (example YAML) and register it with the CLI or API.
|
||||
|
||||
## Plan and apply
|
||||
|
||||
```bash
|
||||
# Create a plan using an action on your project
|
||||
cleveragents plan use <action-name> --project my-project
|
||||
# List plans to find the plan ID
|
||||
cleveragents plan list
|
||||
# Review the plan, then apply it by plan ID
|
||||
cleveragents plan apply <plan-id>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you encounter issues running the examples above, consult `docs/development/testing.md` and the project README for local development tips.
|
||||
@@ -578,6 +578,16 @@ def before_scenario(context, scenario):
|
||||
context.acms_error = None
|
||||
context.assemble_error = None
|
||||
|
||||
# --- Env-var save/restore for provider registry ---
|
||||
# Proactively save env vars so they are restored even if a scenario fails
|
||||
# before reaching the step that normally records them.
|
||||
for attr, env_var in [
|
||||
("original_provider_env", "CLEVERAGENTS_DEFAULT_PROVIDER"),
|
||||
("original_model_env", "CLEVERAGENTS_DEFAULT_MODEL"),
|
||||
("original_allow_mock_env", "CLEVERAGENTS_ALLOW_MOCK_PROVIDER"),
|
||||
]:
|
||||
setattr(context, attr, os.environ.get(env_var))
|
||||
|
||||
# Ensure mock AI flag is always set so plan service tests can resolve actors
|
||||
os.environ["CLEVERAGENTS_TESTING_USE_MOCK_AI"] = "true"
|
||||
|
||||
@@ -740,6 +750,7 @@ def after_scenario(context, scenario):
|
||||
for attr, env_var in [
|
||||
("original_provider_env", "CLEVERAGENTS_DEFAULT_PROVIDER"),
|
||||
("original_model_env", "CLEVERAGENTS_DEFAULT_MODEL"),
|
||||
("original_allow_mock_env", "CLEVERAGENTS_ALLOW_MOCK_PROVIDER"),
|
||||
]:
|
||||
if hasattr(context, attr):
|
||||
original = getattr(context, attr)
|
||||
|
||||
@@ -267,3 +267,19 @@ Feature: Built-in Git Tools
|
||||
Given a tool registry
|
||||
When I register all git tools
|
||||
Then all registered git tools should be read-only
|
||||
|
||||
# ---- Thread Safety (_get_base_env TOCTOU fix) ----
|
||||
|
||||
Scenario: _get_base_env returns the same dict object on repeated calls
|
||||
When I call _get_base_env twice
|
||||
Then both calls should return the same dict object
|
||||
|
||||
Scenario: _get_base_env returns a dict containing git override keys
|
||||
When I call _get_base_env
|
||||
Then the result should contain key "GIT_PAGER"
|
||||
And the result should contain key "NO_COLOR"
|
||||
And the result should contain key "GIT_TERMINAL_PROMPT"
|
||||
|
||||
Scenario: _get_base_env is safe under concurrent initialisation
|
||||
When I call _get_base_env concurrently from 20 threads
|
||||
Then all threads should receive the same dict object
|
||||
|
||||
@@ -54,29 +54,26 @@ Feature: Provider Registry Coverage
|
||||
And the is_configured should be False by default
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Optional string helper coerces non-string values
|
||||
Given I import the optional string coercion helper
|
||||
When I coerce optional string with integer 42
|
||||
Then the coerced optional string result should be "42"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Get all providers from registry
|
||||
Given I have a ProviderRegistry instance
|
||||
When I call get_all_providers
|
||||
Then the provider registry result should be a list of ProviderInfo
|
||||
And the provider registry result should include core provider types
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Get configured providers when none configured
|
||||
Given I have a ProviderRegistry with no API keys
|
||||
Scenario: Get configured providers when none configured and mock not allowed
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is not set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I call get_configured_providers
|
||||
Then the provider registry result should be an empty list
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Get configured providers when none configured
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I call get_configured_providers
|
||||
Then the provider registry result should contain only the MOCK provider
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Get configured providers with OpenAI key
|
||||
Given I have a ProviderRegistry with OpenAI API key set
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with OpenAI API key set
|
||||
When I call get_configured_providers
|
||||
Then the provider registry result should contain OpenAI provider
|
||||
And the provider registry result should contain the MOCK provider
|
||||
And the provider registry OpenAI entry should be configured
|
||||
|
||||
@unit @providers @registry
|
||||
@@ -295,7 +292,8 @@ Feature: Provider Registry Coverage
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create LLM for mock provider skips API key requirement
|
||||
Given I have a ProviderRegistry with no API keys
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
And the provider registry LLM factory is stubbed
|
||||
@@ -303,17 +301,34 @@ Feature: Provider Registry Coverage
|
||||
Then the stubbed LLM factory should be called for ProviderType.MOCK with model "mock-gpt"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm builds stubbed OpenAI client
|
||||
Scenario: Create LLM for mock provider is rejected without env var
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is not set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I try to create an LLM for provider "mock"
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention "not allowed"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create LLM for mock provider returns FakeListLLM end-to-end
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I call create_llm for provider "mock" without specifying a model
|
||||
Then the created LLM should be a FakeListLLM instance
|
||||
And the FakeListLLM produces response "mock response" on invoke
|
||||
And the FakeListLLM produces response "mock response" on invoke
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_instance builds stubbed OpenAI client
|
||||
Given I have a ProviderRegistry with OpenAI API key set
|
||||
And langchain OpenAI client is stubbed
|
||||
When I call _create_provider_llm for provider "openai" with model "custom-openai"
|
||||
When I call _create_provider_instance for provider "openai" with model "custom-openai"
|
||||
Then the stubbed ChatOpenAI should be created with model "custom-openai"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario Outline: _create_provider_llm builds <ProviderName> client
|
||||
Scenario Outline: _create_provider_instance builds <ProviderName> client
|
||||
Given I have a ProviderRegistry with "<provider_key>" API key set
|
||||
And the "<ProviderName>" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "<provider_key>" with model "<model>"
|
||||
When I call _create_provider_instance for provider "<provider_key>" with model "<model>"
|
||||
Then the stubbed <ProviderName> client should be created with argument "model" value "<model>"
|
||||
|
||||
Examples:
|
||||
@@ -326,92 +341,12 @@ Feature: Provider Registry Coverage
|
||||
| Cohere | cohere | command-cover |
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm builds Azure client with deployment override
|
||||
Given I have a ProviderRegistry with "azure" API key set
|
||||
And the "Azure" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "azure" with model "azure-cover"
|
||||
Then the stubbed Azure client should be created with argument "deployment_name" value "azure-cover"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm raises when Azure endpoint is missing
|
||||
Given I have a ProviderRegistry with "azure" API key set
|
||||
And the Azure endpoint setting is cleared
|
||||
When I try to call _create_provider_llm for provider "azure" with model "azure-cover"
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention missing Azure endpoint
|
||||
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm builds OpenRouter client with ChatOpenAI
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
||||
Then the stubbed openrouter client should be created with argument "model" value "claude-openrouter-cover"
|
||||
And the stubbed openrouter client should be created with argument "openai_api_base" value "https://openrouter.ai/api/v1"
|
||||
And the stubbed openrouter client should be created with argument "openai_api_key" value "sk-openrouter-test"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm builds OpenRouter client with sanitized default headers
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
And the OpenRouter default headers include numeric entries
|
||||
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and custom default headers
|
||||
Then the stubbed OpenRouter client headers should include "123" value "456"
|
||||
And the stubbed OpenRouter client headers should include "X-Debug" value "789"
|
||||
And the stubbed openrouter client should be created with argument "openai_api_key" value "sk-openrouter-test"
|
||||
And the stubbed OpenRouter client headers should not include key 123
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: create_llm succeeds for OpenRouter and delegates to _create_provider_llm
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
And the provider registry LLM factory is stubbed
|
||||
When I call create_llm for provider "openrouter" and model "claude-openrouter"
|
||||
Then the stubbed LLM factory should be called for ProviderType.OPENROUTER with model "claude-openrouter"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm adds organization headers for OpenRouter
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the OpenRouter organization setting is "my-org"
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
||||
Then the stubbed OpenRouter client headers should include "HTTP-Referer" value "my-org"
|
||||
And the stubbed OpenRouter client headers should include "X-Title" value "my-org"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm uses default model for OpenRouter when model_id is None
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "openrouter" without specifying a model
|
||||
Then the stubbed openrouter client should be created with argument "model" value "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm treats empty default_headers as None for OpenRouter
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and empty default headers
|
||||
Then the stubbed openrouter client should not have default_headers
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm forwards extra kwargs to ChatOpenAI for OpenRouter
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And the "openrouter" LangChain client is stubbed
|
||||
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and temperature 0.7
|
||||
Then the stubbed openrouter client should be created with argument "temperature" value "0.7"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm raises error when OpenRouter API key is empty
|
||||
Given I have a ProviderRegistry with no API keys
|
||||
When I try to call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention missing "OPENROUTER_API_KEY" environment variable
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_llm raises error for unsupported provider
|
||||
Given I have a ProviderRegistry instance
|
||||
When I try to call _create_provider_llm for provider "mock" with model "mock-gpt"
|
||||
Then a provider registry ValueError should be raised
|
||||
Scenario: _create_provider_instance handles MOCK provider
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
And the "mock" LangChain client is stubbed
|
||||
When I call _create_provider_instance for provider "MOCK" with model "mock-gpt"
|
||||
Then the stubbed mock client should be instantiated
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create AI provider raises error when no provider configured
|
||||
@@ -434,12 +369,30 @@ Feature: Provider Registry Coverage
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention missing "GOOGLE_API_KEY" environment variable
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create AI provider succeeds for MOCK provider without API key
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider for provider "mock"
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "mock"
|
||||
And the provider registry AI provider model_id should be "mock-gpt"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create AI provider for MOCK is rejected without env var
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is not set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I try to create an AI provider for provider "mock"
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention "not allowed"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Create AI provider succeeds for configured provider string
|
||||
Given I have a ProviderRegistry with OpenAI API key set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider for provider "openai"
|
||||
Then the provider registry AI provider should be an OpenAIChatProvider
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "openai"
|
||||
And the provider registry AI provider model_id should be "gpt-4o"
|
||||
|
||||
@@ -448,7 +401,7 @@ Feature: Provider Registry Coverage
|
||||
Given I have a ProviderRegistry with Anthropic API key set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider for provider "anthropic"
|
||||
Then the provider registry AI provider should be an AnthropicChatProvider
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "anthropic"
|
||||
And the provider registry AI provider model_id should be "claude-sonnet-4-20250514"
|
||||
|
||||
@@ -471,7 +424,7 @@ Feature: Provider Registry Coverage
|
||||
Given I have a ProviderRegistry with "google" API key set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider for provider "google"
|
||||
Then the provider registry AI provider should be a GoogleChatProvider
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "google"
|
||||
And the provider registry AI provider model_id should be "gemini-2.0-flash"
|
||||
|
||||
@@ -480,7 +433,7 @@ Feature: Provider Registry Coverage
|
||||
Given I have a ProviderRegistry with "openrouter" API key set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider for provider "openrouter"
|
||||
Then the provider registry AI provider should be an OpenRouterChatProvider
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "openrouter"
|
||||
And the provider registry AI provider model_id should be "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
@@ -490,7 +443,7 @@ Feature: Provider Registry Coverage
|
||||
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
||||
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
||||
When I create an AI provider without specifying a provider
|
||||
Then the provider registry AI provider should be an OpenAIChatProvider
|
||||
Then the provider registry AI provider should be a LangChainChatProvider
|
||||
And the provider registry AI provider name should be "openai"
|
||||
|
||||
|
||||
@@ -509,6 +462,38 @@ Feature: Provider Registry Coverage
|
||||
And I call the AI provider llm factory with model "llama-mini-cover"
|
||||
Then the stubbed LLM factory should be called for ProviderType.GROQ with model "llama-mini-cover"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_instance handles MOCK provider and response list
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
And the "mock" LangChain client is stubbed
|
||||
When I call _create_provider_instance for provider "MOCK" with model "mock-gpt"
|
||||
Then the stubbed mock client should be instantiated
|
||||
And the stubbed mock client responses should be ["mock response"] * 10
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: API key is forwarded through create_ai_provider closure
|
||||
Given I have a ProviderRegistry with OpenAI API key set
|
||||
And the provider registry LLM factory is stubbed
|
||||
When I create an AI provider for provider "openai"
|
||||
And I call the AI provider llm factory with model "gpt-mini"
|
||||
Then the stubbed LLM factory should be called for ProviderType.OPENAI with model "gpt-mini"
|
||||
And the stubbed LLM factory should be called with api_key present
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: resolve_provider_by_name rejects mock without env var
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is not set
|
||||
When I call resolve_provider_by_name with "mock"
|
||||
Then a provider registry ValueError should be raised
|
||||
And the provider registry error should mention "not configured"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: resolve_provider_by_name resolves mock with env var
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And I have a ProviderRegistry with no API keys
|
||||
When I call resolve_provider_by_name with "mock"
|
||||
Then the resolved provider should be a LangChainChatProvider
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: ProviderCapabilities is immutable
|
||||
Given I import the ProviderCapabilities class
|
||||
|
||||
@@ -9,30 +9,26 @@ Feature: Provider Registry Coverage Boost
|
||||
And CLEVERAGENTS_DEFAULT_PROVIDER env var is cleared
|
||||
When I request the default provider type from the boost registry
|
||||
Then the boost result should be ProviderType ANTHROPIC
|
||||
# Covers lines 335-338: settings_default valid ProviderType but not_configured
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Azure deployment resolves from settings when model_id is None
|
||||
Given a registry with azure API key and deployment setting "my-azure-deploy"
|
||||
And the Azure LangChain client is stubbed for boost
|
||||
When I call _create_provider_llm for AZURE with model_id None via boost
|
||||
When I call _create_provider_instance for AZURE with model_id None via boost
|
||||
Then the stubbed Azure client deployment_name should be "my-azure-deploy"
|
||||
# Covers line 502: self._settings.azure_openai_deployment branch
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Azure deployment falls back to gpt-4o when settings deployment is also empty
|
||||
Given a registry with azure API key and no deployment setting
|
||||
And the Azure LangChain client is stubbed for boost
|
||||
When I call _create_provider_llm for AZURE with model_id None via boost
|
||||
When I call _create_provider_instance for AZURE with model_id None via boost
|
||||
Then the stubbed Azure client deployment_name should be "gpt-4o"
|
||||
# Covers line 503: final "gpt-4o" fallback
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Google create_ai_provider uses DEFAULT_MODELS when model_id is empty string
|
||||
Given a registry with google API key configured for boost
|
||||
When I create an AI provider for google with empty string model_id via boost
|
||||
Then the boost AI provider model_id should be "gemini-2.0-flash"
|
||||
# Covers line 620: DEFAULT_MODELS.get fallback in Google branch
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: OpenRouter create_ai_provider raises error when API key is missing
|
||||
@@ -41,11 +37,37 @@ Feature: Provider Registry Coverage Boost
|
||||
Then a boost ValueError should be raised
|
||||
And the boost error message should contain "openrouter"
|
||||
And the boost error message should contain "not configured"
|
||||
# Covers lines 632-637: OpenRouter not-configured error path
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: OpenRouter create_ai_provider uses DEFAULT_MODELS when model_id is empty string
|
||||
Given a registry with openrouter API key configured for boost
|
||||
When I create an AI provider for openrouter with empty string model_id via boost
|
||||
Then the boost AI provider model_id should be "anthropic/claude-sonnet-4-20250514"
|
||||
# Covers lines 643-644: DEFAULT_MODELS.get fallback in OpenRouter branch
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: OpenRouter _create_provider_instance with sanitized numeric headers and organization
|
||||
Given a registry with openrouter API key configured for boost
|
||||
And the openrouter organization is set to "my-org"
|
||||
When I call _create_provider_instance for openrouter with model "claude-or" and numeric headers
|
||||
Then the returned LLM should be recorded with sanitized headers including "X-Debug"
|
||||
And the openrouter kwargs should include HTTP-Referer "my-org"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: _create_provider_instance accepts pre-validated api_key sentinel None
|
||||
Given CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set
|
||||
And a registry with no API keys for boost
|
||||
When I call _create_provider_instance for mock with api_key sentinel None
|
||||
Then the boost result should be a FakeListLLM instance
|
||||
And calling it twice yields the same response
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: Azure endpoint missing raises ValueError
|
||||
Given a registry with azure API key but no endpoint setting
|
||||
When I try to call _create_provider_instance for AZURE with model_id None via boost
|
||||
Then a boost ValueError should be raised
|
||||
And the boost error message should contain "endpoint"
|
||||
|
||||
@unit @providers @registry
|
||||
Scenario: coerce_optional_str coerces non-string values
|
||||
When I coerce optional string with integer 42
|
||||
Then the coerced optional string result should be "42"
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Step definitions for _get_base_env thread-safety (TOCTOU fix) scenarios.
|
||||
|
||||
These steps exercise the double-checked locking pattern introduced in
|
||||
``git_tools._get_base_env()`` to eliminate the TOCTOU race condition
|
||||
where two concurrent threads could both observe ``_BASE_ENV is None``
|
||||
and write potentially different snapshots.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from typing import Any
|
||||
|
||||
import cleveragents.tool.builtins.git_tools as _git_tools_mod
|
||||
from behave import then, when
|
||||
from cleveragents.tool.builtins.git_tools import _get_base_env
|
||||
|
||||
__all__: list[str] = []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Whens
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I call _get_base_env twice")
|
||||
def step_when_call_get_base_env_twice(context: Any) -> None:
|
||||
# Reset the module-level cache so the test is independent of call order.
|
||||
original = _git_tools_mod._BASE_ENV
|
||||
_git_tools_mod._BASE_ENV = None
|
||||
try:
|
||||
context.base_env_first = _get_base_env()
|
||||
context.base_env_second = _get_base_env()
|
||||
finally:
|
||||
_git_tools_mod._BASE_ENV = original
|
||||
|
||||
|
||||
@when("I call _get_base_env")
|
||||
def step_when_call_get_base_env(context: Any) -> None:
|
||||
original = _git_tools_mod._BASE_ENV
|
||||
_git_tools_mod._BASE_ENV = None
|
||||
try:
|
||||
context.base_env_result = _get_base_env()
|
||||
finally:
|
||||
_git_tools_mod._BASE_ENV = original
|
||||
|
||||
|
||||
@when("I call _get_base_env concurrently from 20 threads")
|
||||
def step_when_call_get_base_env_concurrent(context: Any) -> None:
|
||||
n_threads = 20
|
||||
results: list[dict[str, str] | None] = [None] * n_threads
|
||||
barrier = threading.Barrier(n_threads)
|
||||
|
||||
# Reset the module-level cache so all threads race on first initialisation.
|
||||
original = _git_tools_mod._BASE_ENV
|
||||
_git_tools_mod._BASE_ENV = None
|
||||
|
||||
def worker(idx: int) -> None:
|
||||
barrier.wait() # synchronise all threads to maximise race probability
|
||||
results[idx] = _get_base_env()
|
||||
|
||||
threads = [threading.Thread(target=worker, args=(i,)) for i in range(n_threads)]
|
||||
try:
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join(timeout=10)
|
||||
finally:
|
||||
_git_tools_mod._BASE_ENV = original
|
||||
|
||||
context.concurrent_results = results
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thens
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("both calls should return the same dict object")
|
||||
def step_then_same_dict_object(context: Any) -> None:
|
||||
assert context.base_env_first is context.base_env_second, (
|
||||
"_get_base_env() returned different dict objects on repeated calls; "
|
||||
"caching is broken"
|
||||
)
|
||||
|
||||
|
||||
@then('the result should contain key "{key}"')
|
||||
def step_then_result_contains_key(context: Any, key: str) -> None:
|
||||
assert key in context.base_env_result, (
|
||||
f"Expected key '{key}' in _get_base_env() result, "
|
||||
f"got keys: {list(context.base_env_result.keys())}"
|
||||
)
|
||||
|
||||
|
||||
@then("all threads should receive the same dict object")
|
||||
def step_then_all_threads_same_object(context: Any) -> None:
|
||||
results = context.concurrent_results
|
||||
assert all(r is not None for r in results), (
|
||||
"Some threads did not receive a result from _get_base_env()"
|
||||
)
|
||||
first = results[0]
|
||||
for i, r in enumerate(results[1:], start=1):
|
||||
assert r is first, (
|
||||
f"Thread {i} received a different dict object than thread 0; "
|
||||
"double-checked locking is broken"
|
||||
)
|
||||
@@ -24,6 +24,9 @@ from cleveragents.providers.registry import (
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_AZURE_ENDPOINT_DEFAULT = object()
|
||||
|
||||
|
||||
def _make_boost_settings(
|
||||
openai: str | None = None,
|
||||
anthropic: str | None = None,
|
||||
@@ -36,7 +39,7 @@ def _make_boost_settings(
|
||||
together: str | None = None,
|
||||
default_provider: str | None = None,
|
||||
default_model: str | None = None,
|
||||
azure_endpoint: str | None = None,
|
||||
azure_endpoint: str | None = _AZURE_ENDPOINT_DEFAULT,
|
||||
azure_api_version: str | None = None,
|
||||
azure_deployment: str | None = None,
|
||||
openrouter_org: str | None = None,
|
||||
@@ -54,8 +57,8 @@ def _make_boost_settings(
|
||||
settings.together_api_key = together
|
||||
settings.default_provider = default_provider
|
||||
settings.default_model = default_model
|
||||
if azure and not azure_endpoint:
|
||||
azure_endpoint = "https://example.openai.azure.com"
|
||||
if azure_endpoint is _AZURE_ENDPOINT_DEFAULT:
|
||||
azure_endpoint = "https://example.openai.azure.com" if azure else None
|
||||
settings.azure_openai_endpoint = azure_endpoint
|
||||
settings.azure_openai_api_version = azure_api_version
|
||||
settings.azure_openai_deployment = azure_deployment
|
||||
@@ -168,6 +171,18 @@ def step_boost_openrouter_registry(context: Any) -> None:
|
||||
)
|
||||
|
||||
|
||||
@given('the openrouter organization is set to "{org}"')
|
||||
def step_boost_openrouter_org(context: Any, org: str) -> None:
|
||||
context.boost_registry._settings.openrouter_organization = org
|
||||
|
||||
|
||||
@given("a registry with azure API key but no endpoint setting")
|
||||
def step_boost_azure_no_endpoint(context: Any) -> None:
|
||||
context.boost_registry = ProviderRegistry(
|
||||
settings=_make_boost_settings(azure="sk-azure-test", azure_endpoint=None)
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -178,13 +193,63 @@ def step_boost_get_default_provider_type(context: Any) -> None:
|
||||
context.boost_result = context.boost_registry.get_default_provider_type()
|
||||
|
||||
|
||||
@when("I call _create_provider_llm for AZURE with model_id None via boost")
|
||||
def step_boost_create_azure_llm_no_model(context: Any) -> None:
|
||||
context.boost_result = context.boost_registry._create_provider_llm(
|
||||
@when("I call _create_provider_instance for AZURE with model_id None via boost")
|
||||
def step_boost_create_azure_instance_no_model(context: Any) -> None:
|
||||
context.boost_result = context.boost_registry._create_provider_instance(
|
||||
ProviderType.AZURE, None
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I call _create_provider_instance for openrouter with model "{model}" and numeric headers'
|
||||
)
|
||||
def step_boost_create_openrouter_with_headers(context: Any, model: str) -> None:
|
||||
original = sys.modules.get("langchain_openai")
|
||||
calls: list[dict[str, Any]] = []
|
||||
|
||||
class FakeChatOpenAI:
|
||||
def __init__(self, **kwargs: Any) -> None:
|
||||
calls.append(kwargs)
|
||||
|
||||
stub = types.ModuleType("langchain_openai")
|
||||
stub.ChatOpenAI = FakeChatOpenAI
|
||||
stub.AzureChatOpenAI = lambda **_: None
|
||||
sys.modules["langchain_openai"] = stub
|
||||
|
||||
context._boost_openrouter_calls = calls
|
||||
|
||||
def _restore() -> None:
|
||||
if original is None:
|
||||
sys.modules.pop("langchain_openai", None)
|
||||
else:
|
||||
sys.modules["langchain_openai"] = original
|
||||
|
||||
context._boost_cleanup = getattr(context, "_boost_cleanup", [])
|
||||
context._boost_cleanup.append(_restore)
|
||||
|
||||
context.boost_result = context.boost_registry._create_provider_instance(
|
||||
ProviderType.OPENROUTER,
|
||||
model,
|
||||
default_headers={123: 456, "X-Debug": 789},
|
||||
)
|
||||
|
||||
|
||||
@when("I call _create_provider_instance for mock with api_key sentinel None")
|
||||
def step_boost_create_mock_with_sentinel(context: Any) -> None:
|
||||
context.boost_result = context.boost_registry._create_provider_instance(
|
||||
ProviderType.MOCK, None, __api_key_sentinel=None
|
||||
)
|
||||
|
||||
|
||||
@when("I try to call _create_provider_instance for AZURE with model_id None via boost")
|
||||
def step_boost_try_create_azure_no_model(context: Any) -> None:
|
||||
try:
|
||||
context.boost_registry._create_provider_instance(ProviderType.AZURE, None)
|
||||
context.boost_error = None
|
||||
except ValueError as exc:
|
||||
context.boost_error = exc
|
||||
|
||||
|
||||
@when("I create an AI provider for google with empty string model_id via boost")
|
||||
def step_boost_google_ai_provider_empty_model(context: Any) -> None:
|
||||
# Ensure env vars don't interfere
|
||||
@@ -215,6 +280,13 @@ def step_boost_openrouter_ai_provider_empty_model(context: Any) -> None:
|
||||
)
|
||||
|
||||
|
||||
@when("I coerce optional string with integer {value:d}")
|
||||
def step_boost_coerce_optional_str(context: Any, value: int) -> None:
|
||||
from cleveragents.providers.registry import _coerce_optional_str
|
||||
|
||||
context.coerced_value = _coerce_optional_str(value)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -256,3 +328,51 @@ def step_boost_error_contains(context: Any, fragment: str) -> None:
|
||||
assert fragment.lower() in message, (
|
||||
f"Expected error to contain {fragment!r}, got: {message!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the returned LLM should be recorded with sanitized headers including "{header}"')
|
||||
def step_boost_openrouter_sanitized_headers(context: Any, header: str) -> None:
|
||||
calls = getattr(context, "_boost_openrouter_calls", [])
|
||||
assert calls, "Expected ChatOpenAI to be recorded"
|
||||
kwargs = calls[-1]
|
||||
headers = kwargs.get("default_headers", {})
|
||||
assert header in headers, f"Expected {header!r} in headers, got {headers!r}"
|
||||
# Verify numeric keys were coerced to strings
|
||||
assert "123" in headers, "Expected key '123' (coerced from int) in headers"
|
||||
|
||||
|
||||
@then('the openrouter kwargs should include HTTP-Referer "{expected}"')
|
||||
def step_boost_openrouter_referer(context: Any, expected: str) -> None:
|
||||
calls = getattr(context, "_boost_openrouter_calls", [])
|
||||
assert calls, "Expected ChatOpenAI to be recorded"
|
||||
kwargs = calls[-1]
|
||||
headers = kwargs.get("default_headers", {})
|
||||
assert headers.get("HTTP-Referer") == expected, (
|
||||
f"Expected HTTP-Referer={expected!r}, got {headers.get('HTTP-Referer')!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the boost result should be a FakeListLLM instance")
|
||||
def step_boost_result_is_fake_list_llm(context: Any) -> None:
|
||||
from langchain_community.llms import FakeListLLM
|
||||
|
||||
assert isinstance(context.boost_result, FakeListLLM), (
|
||||
f"Expected FakeListLLM, got {type(context.boost_result)}"
|
||||
)
|
||||
|
||||
|
||||
@then("calling it twice yields the same response")
|
||||
def step_boost_llm_invokes_twice(context: Any) -> None:
|
||||
llm = context.boost_result
|
||||
response1 = llm.invoke("test prompt")
|
||||
response2 = llm.invoke("test prompt 2")
|
||||
assert response1 == response2 == "mock response", (
|
||||
f"Expected 'mock response' for both, got {response1!r} and {response2!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the coerced optional string result should be "{expected}"')
|
||||
def step_boost_coerced_result(context: Any, expected: str) -> None:
|
||||
assert getattr(context, "coerced_value", None) == expected, (
|
||||
f"Expected {expected!r}, got {getattr(context, 'coerced_value', None)!r}"
|
||||
)
|
||||
|
||||
@@ -9,11 +9,7 @@ from unittest.mock import MagicMock
|
||||
|
||||
from behave import given, then, when # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.providers.llm.anthropic_provider import AnthropicChatProvider
|
||||
from cleveragents.providers.llm.google_provider import GoogleChatProvider
|
||||
from cleveragents.providers.llm.langchain_chat_provider import LangChainChatProvider
|
||||
from cleveragents.providers.llm.openai_provider import OpenAIChatProvider
|
||||
from cleveragents.providers.llm.openrouter_provider import OpenRouterChatProvider
|
||||
from cleveragents.providers.registry import (
|
||||
ProviderCapabilities,
|
||||
ProviderInfo,
|
||||
@@ -139,6 +135,7 @@ LANGCHAIN_CLIENT_SPECS: dict[str, tuple[str, str]] = {
|
||||
"groq": ("langchain_groq", "ChatGroq"),
|
||||
"together": ("langchain_together", "ChatTogether"),
|
||||
"cohere": ("langchain_cohere", "ChatCohere"),
|
||||
"mock": ("langchain_community.llms", "FakeListLLM"),
|
||||
}
|
||||
|
||||
|
||||
@@ -301,6 +298,18 @@ def step_impl_unset_default_model_env(context: Any) -> None:
|
||||
os.environ.pop("CLEVERAGENTS_DEFAULT_MODEL", None)
|
||||
|
||||
|
||||
@given("CLEVERAGENTS_ALLOW_MOCK_PROVIDER is set")
|
||||
def step_impl_allow_mock_provider_set(context: Any) -> None:
|
||||
context.original_allow_mock_env = os.environ.get("CLEVERAGENTS_ALLOW_MOCK_PROVIDER")
|
||||
os.environ["CLEVERAGENTS_ALLOW_MOCK_PROVIDER"] = "true"
|
||||
|
||||
|
||||
@given("CLEVERAGENTS_ALLOW_MOCK_PROVIDER is not set")
|
||||
def step_impl_allow_mock_provider_unset(context: Any) -> None:
|
||||
context.original_allow_mock_env = os.environ.get("CLEVERAGENTS_ALLOW_MOCK_PROVIDER")
|
||||
os.environ.pop("CLEVERAGENTS_ALLOW_MOCK_PROVIDER", None)
|
||||
|
||||
|
||||
@given("langchain OpenAI client is stubbed")
|
||||
def step_impl_stub_langchain_openai(context: Any) -> None:
|
||||
module_name, class_name = LANGCHAIN_CLIENT_SPECS["openai"]
|
||||
@@ -326,7 +335,9 @@ def step_impl_stub_named_langchain_client(context: Any, provider_name: str) -> N
|
||||
@given("the provider registry LLM factory is stubbed")
|
||||
def step_impl_stub_provider_factory(context: Any) -> None:
|
||||
context.stubbed_llm = object()
|
||||
context.registry._create_provider_llm = MagicMock(return_value=context.stubbed_llm)
|
||||
context.registry._create_provider_instance = MagicMock(
|
||||
return_value=context.stubbed_llm
|
||||
)
|
||||
|
||||
|
||||
# -- When -------------------------------------------------------------------
|
||||
@@ -337,12 +348,6 @@ def step_impl_create_registry_default(context: Any) -> None:
|
||||
context.registry = context.ProviderRegistry()
|
||||
|
||||
|
||||
@when("I coerce optional string with integer {value:d}")
|
||||
def step_impl_coerce_optional_string_with_integer(context: Any, value: int) -> None:
|
||||
coerce_helper = getattr(context, "coerce_optional_str", _coerce_optional_str)
|
||||
context.coerced_value = coerce_helper(value)
|
||||
|
||||
|
||||
@when("I create a ProviderCapabilities with all fields")
|
||||
def step_impl_capabilities_all_fields(context: Any) -> None:
|
||||
context.capabilities = context.ProviderCapabilities(
|
||||
@@ -498,67 +503,75 @@ def step_impl_call_create_llm_no_model(context: Any, value: str) -> None:
|
||||
context.created_llm = context.registry.create_llm(provider_type=value)
|
||||
|
||||
|
||||
@when('I call _create_provider_llm for provider "{value}" with model "{model}"')
|
||||
def step_impl_call_private_llm(context: Any, value: str, model: str) -> None:
|
||||
@when('I call _create_provider_instance for provider "{value}" with model "{model}"')
|
||||
def step_impl_call_private_instance(context: Any, value: str, model: str) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
context.created_llm = context.registry._create_provider_llm(provider_type, model)
|
||||
context.created_llm = context.registry._create_provider_instance(
|
||||
provider_type, model
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I call _create_provider_llm for provider "{value}" with model "{model}" and custom default headers'
|
||||
'I call _create_provider_instance for provider "{value}" with model "{model}" and custom default headers'
|
||||
)
|
||||
def step_impl_call_private_llm_with_headers(
|
||||
def step_impl_call_private_instance_with_headers(
|
||||
context: Any, value: str, model: str
|
||||
) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
default_headers = getattr(context, "openrouter_default_headers", None)
|
||||
assert default_headers is not None, "OpenRouter default headers were not configured"
|
||||
context.created_llm = context.registry._create_provider_llm(
|
||||
context.created_llm = context.registry._create_provider_instance(
|
||||
provider_type, model, default_headers=default_headers
|
||||
)
|
||||
|
||||
|
||||
@when('I call _create_provider_llm for provider "{value}" without specifying a model')
|
||||
def step_impl_call_private_llm_no_model(context: Any, value: str) -> None:
|
||||
@when(
|
||||
'I call _create_provider_instance for provider "{value}" without specifying a model'
|
||||
)
|
||||
def step_impl_call_private_instance_no_model(context: Any, value: str) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
context.created_llm = context.registry._create_provider_llm(provider_type, None)
|
||||
context.created_llm = context.registry._create_provider_instance(
|
||||
provider_type, None
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I call _create_provider_llm for provider "{value}" with model "{model}" and empty default headers'
|
||||
'I call _create_provider_instance for provider "{value}" with model "{model}" and empty default headers'
|
||||
)
|
||||
def step_impl_call_private_llm_empty_headers(
|
||||
def step_impl_call_private_instance_empty_headers(
|
||||
context: Any, value: str, model: str
|
||||
) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
context.created_llm = context.registry._create_provider_llm(
|
||||
context.created_llm = context.registry._create_provider_instance(
|
||||
provider_type, model, default_headers={}
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I call _create_provider_llm for provider "{value}" with model "{model}" and temperature {temperature}'
|
||||
'I call _create_provider_instance for provider "{value}" with model "{model}" and temperature {temperature}'
|
||||
)
|
||||
def step_impl_call_private_llm_with_temperature(
|
||||
def step_impl_call_private_instance_with_temperature(
|
||||
context: Any, value: str, model: str, temperature: str
|
||||
) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
context.created_llm = context.registry._create_provider_llm(
|
||||
context.created_llm = context.registry._create_provider_instance(
|
||||
provider_type, model, temperature=float(temperature)
|
||||
)
|
||||
|
||||
|
||||
@when('I try to call _create_provider_llm for provider "{value}" with model "{model}"')
|
||||
def step_impl_try_private_llm(context: Any, value: str, model: str) -> None:
|
||||
@when(
|
||||
'I try to call _create_provider_instance for provider "{value}" with model "{model}"'
|
||||
)
|
||||
def step_impl_try_private_instance(context: Any, value: str, model: str) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
provider_type = getattr(provider_enum, value.upper())
|
||||
try:
|
||||
context.registry._create_provider_llm(provider_type, model)
|
||||
context.registry._create_provider_instance(provider_type, model)
|
||||
context.error = None
|
||||
except Exception as exc: # pragma: no cover - defensive path
|
||||
context.error = exc
|
||||
@@ -630,13 +643,6 @@ def step_impl_registry_has_providers(context: Any) -> None:
|
||||
assert len(context.registry.get_all_providers()) > 0
|
||||
|
||||
|
||||
@then('the coerced optional string result should be "{expected}"')
|
||||
def step_impl_coerced_optional_string_result(context: Any, expected: str) -> None:
|
||||
assert getattr(context, "coerced_value", None) == expected, (
|
||||
f"Expected coerced value {expected!r}, got {getattr(context, 'coerced_value', None)!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('ProviderType should have {enum_name} value "{value}"')
|
||||
def step_impl_provider_type_values(context: Any, enum_name: str, value: str) -> None:
|
||||
enum_member = getattr(_ensure_provider_type(context), enum_name.upper())
|
||||
@@ -816,24 +822,14 @@ def step_impl_ai_provider_is_langchain(context: Any) -> None:
|
||||
assert isinstance(context.ai_provider, LangChainChatProvider)
|
||||
|
||||
|
||||
@then("the provider registry AI provider should be an OpenAIChatProvider")
|
||||
def step_impl_ai_provider_is_openai(context: Any) -> None:
|
||||
assert isinstance(context.ai_provider, OpenAIChatProvider)
|
||||
|
||||
|
||||
@then("the provider registry AI provider should be an AnthropicChatProvider")
|
||||
def step_impl_ai_provider_is_anthropic(context: Any) -> None:
|
||||
assert isinstance(context.ai_provider, AnthropicChatProvider)
|
||||
|
||||
|
||||
@then("the provider registry AI provider should be a GoogleChatProvider")
|
||||
def step_impl_ai_provider_is_google(context: Any) -> None:
|
||||
assert isinstance(context.ai_provider, GoogleChatProvider)
|
||||
|
||||
|
||||
@then("the provider registry AI provider should be an OpenRouterChatProvider")
|
||||
def step_impl_ai_provider_is_openrouter(context: Any) -> None:
|
||||
assert isinstance(context.ai_provider, OpenRouterChatProvider)
|
||||
@then("the stubbed mock client should be instantiated")
|
||||
def step_impl_stubbed_mock_client_instantiated(context: Any) -> None:
|
||||
stubbed_clients = getattr(context, "stubbed_clients", {})
|
||||
client_data = stubbed_clients.get("mock")
|
||||
assert client_data, "No stubbed client recorded for mock"
|
||||
calls = client_data["calls"]
|
||||
assert calls, "Expected FakeListLLM to be instantiated"
|
||||
assert isinstance(context.created_llm, client_data["class"])
|
||||
|
||||
|
||||
@then('the provider registry AI provider name should be "{value}"')
|
||||
@@ -854,8 +850,8 @@ def step_impl_stubbed_llm_factory_called(
|
||||
) -> None:
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
expected_type = getattr(provider_enum, enum_name.upper())
|
||||
context.registry._create_provider_llm.assert_called_once()
|
||||
args, kwargs = context.registry._create_provider_llm.call_args
|
||||
context.registry._create_provider_instance.assert_called_once()
|
||||
args, kwargs = context.registry._create_provider_instance.call_args
|
||||
assert args[:2] == (expected_type, model)
|
||||
if kwargs:
|
||||
# max_retries is propagated by create_ai_provider to keep retry semantics aligned
|
||||
@@ -863,6 +859,10 @@ def step_impl_stubbed_llm_factory_called(
|
||||
assert max_retries is None or isinstance(max_retries, int), (
|
||||
f"Expected max_retries int or None, got {max_retries!r}"
|
||||
)
|
||||
# api_key should be forwarded (even None) so _get_api_key is not called twice
|
||||
assert "__api_key_sentinel" in kwargs or "api_key" in kwargs, (
|
||||
f"Expected api_key or __api_key_sentinel in factory kwargs, got {sorted(kwargs.keys())!r}"
|
||||
)
|
||||
assert context.created_llm is context.stubbed_llm
|
||||
|
||||
|
||||
@@ -999,6 +999,100 @@ def step_impl_default_models(context: Any, provider: str, model: str) -> None:
|
||||
assert context.default_models[enum_member] == model
|
||||
|
||||
|
||||
# -- Hooks ------------------------------------------------------------------
|
||||
@then('the provider registry error should mention "{message}"')
|
||||
def step_impl_error_mentions_message(context: Any, message: str) -> None:
|
||||
assert context.error is not None
|
||||
assert message.lower() in str(context.error).lower(), (
|
||||
f"Expected error to mention {message!r}, got {str(context.error)!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the created LLM should be a FakeListLLM instance")
|
||||
def step_impl_created_llm_is_fake_list_llm(context: Any) -> None:
|
||||
from langchain_community.llms import FakeListLLM
|
||||
|
||||
assert isinstance(context.created_llm, FakeListLLM), (
|
||||
f"Expected FakeListLLM, got {type(context.created_llm)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the provider registry result should contain only the MOCK provider")
|
||||
def step_impl_result_contains_only_mock(context: Any) -> None:
|
||||
provider_types = [p.provider_type for p in context.result]
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
assert provider_types == [provider_enum.MOCK], (
|
||||
f"Expected only MOCK, got {provider_types}"
|
||||
)
|
||||
|
||||
|
||||
@then("the provider registry result should contain the MOCK provider")
|
||||
def step_impl_result_contains_mock(context: Any) -> None:
|
||||
provider_types = [p.provider_type for p in context.result]
|
||||
provider_enum = _ensure_provider_type(context)
|
||||
assert provider_enum.MOCK in provider_types, (
|
||||
f"Expected MOCK in list, got {provider_types}"
|
||||
)
|
||||
|
||||
|
||||
@then("the stubbed {provider_name} client responses should be {expected_responses}")
|
||||
def step_impl_stubbed_client_responses(
|
||||
context: Any, provider_name: str, expected_responses: str
|
||||
) -> None:
|
||||
provider_key = provider_name.lower()
|
||||
stubbed_clients = getattr(context, "stubbed_clients", {})
|
||||
assert provider_key in stubbed_clients, (
|
||||
f"No stubbed client recorded for {provider_name}"
|
||||
)
|
||||
client_data = stubbed_clients[provider_key]
|
||||
calls = client_data["calls"]
|
||||
assert calls, f"Expected {provider_name} client to be instantiated"
|
||||
last_call = calls[-1]
|
||||
actual = last_call["kwargs"].get("responses")
|
||||
if actual is None:
|
||||
actual = last_call.get("args")
|
||||
expected = eval(expected_responses) # e.g. ["mock response"] * 10
|
||||
assert actual == expected, f"Expected responses={expected!r}, got {actual!r}"
|
||||
assert isinstance(context.created_llm, client_data["class"])
|
||||
|
||||
|
||||
@then("the stubbed LLM factory should be called with api_key present")
|
||||
def step_impl_stubbed_llm_factory_api_key(context: Any) -> None:
|
||||
context.registry._create_provider_instance.assert_called()
|
||||
_args, kwargs = context.registry._create_provider_instance.call_args
|
||||
assert "__api_key_sentinel" in kwargs or "api_key" in kwargs, (
|
||||
f"Expected api_key or __api_key_sentinel in factory kwargs, got {sorted(kwargs.keys())!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the FakeListLLM produces response "{response}" on invoke')
|
||||
def step_impl_fake_list_llm_invokes(context: Any, response: str) -> None:
|
||||
from langchain_community.llms import FakeListLLM
|
||||
|
||||
llm = context.created_llm
|
||||
assert isinstance(llm, FakeListLLM), f"Expected FakeListLLM, got {type(llm)}"
|
||||
assert llm.invoke("test prompt") == response, (
|
||||
f"Expected FakeListLLM to produce {response!r}"
|
||||
)
|
||||
|
||||
|
||||
@when('I call resolve_provider_by_name with "{name}"')
|
||||
def step_impl_resolve_provider_by_name(context: Any, name: str) -> None:
|
||||
from cleveragents.providers.registry import (
|
||||
resolve_provider_by_name,
|
||||
reset_provider_registry,
|
||||
)
|
||||
|
||||
reset_provider_registry()
|
||||
try:
|
||||
context.resolved_provider = resolve_provider_by_name(name)
|
||||
context.error = None
|
||||
except ValueError as exc:
|
||||
context.error = exc
|
||||
|
||||
|
||||
@then("the resolved provider should be a LangChainChatProvider")
|
||||
def step_impl_resolved_provider_is_langchain(context: Any) -> None:
|
||||
assert isinstance(context.resolved_provider, LangChainChatProvider)
|
||||
|
||||
|
||||
# Hook logic lives in features/environment.py to ensure scenarios reset state.
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Step definitions for features/tdd_mcp_adapter_rlock_concurrency.feature.
|
||||
|
||||
Tests that MCPToolAdapter releases its RLock before making transport calls,
|
||||
allowing concurrent operations to proceed without being blocked.
|
||||
|
||||
Bug #10512: MCPToolAdapter holds RLock during entire transport call,
|
||||
blocking concurrent operations.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.mcp.adapter import (
|
||||
MCPServerConfig,
|
||||
MCPToolAdapter,
|
||||
MCPToolResult,
|
||||
)
|
||||
from features.mocks.mock_mcp_transport import MockMCPTransport
|
||||
|
||||
|
||||
def _mock_tool(
|
||||
name: str, desc: str = "", schema: dict[str, Any] | None = None
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"name": name,
|
||||
"description": desc or f"Mock tool {name}",
|
||||
"inputSchema": schema or {},
|
||||
}
|
||||
|
||||
|
||||
class SlowMCPTransport(MockMCPTransport):
|
||||
"""Transport that introduces a configurable delay during call()."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
delay: float,
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
super().__init__(tools=tools)
|
||||
self._delay = delay
|
||||
|
||||
def call(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
# Use the real time.sleep (not the capped test version) to ensure
|
||||
# the delay is meaningful for concurrency testing.
|
||||
_real_sleep = getattr(time, "_original_sleep", time.sleep)
|
||||
_real_sleep(self._delay)
|
||||
return super().call(method, params)
|
||||
|
||||
|
||||
class ConcurrencyTrackingTransport(MockMCPTransport):
|
||||
"""Transport that tracks the maximum number of concurrent in-flight calls.
|
||||
|
||||
Uses a threading.Lock-protected counter to record how many calls are
|
||||
executing simultaneously. The ``max_concurrent`` attribute reflects the
|
||||
peak concurrency observed across all calls.
|
||||
|
||||
If the adapter holds its RLock during transport calls, only one call can
|
||||
be in-flight at a time (max_concurrent == 1). If the lock is released
|
||||
before the transport call, multiple calls can overlap (max_concurrent > 1).
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
delay: float,
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
super().__init__(tools=tools)
|
||||
self._delay = delay
|
||||
self._counter_lock = threading.Lock()
|
||||
self._current_concurrent: int = 0
|
||||
self.max_concurrent: int = 0
|
||||
|
||||
def call(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
with self._counter_lock:
|
||||
self._current_concurrent += 1
|
||||
if self._current_concurrent > self.max_concurrent:
|
||||
self.max_concurrent = self._current_concurrent
|
||||
|
||||
# Use the real time.sleep (not the capped test version) to ensure
|
||||
# the delay is meaningful for concurrency testing.
|
||||
_real_sleep = getattr(time, "_original_sleep", time.sleep)
|
||||
_real_sleep(self._delay)
|
||||
|
||||
with self._counter_lock:
|
||||
self._current_concurrent -= 1
|
||||
|
||||
return super().call(method, params)
|
||||
|
||||
|
||||
class LockCheckingInvokeTransport(MockMCPTransport):
|
||||
"""Transport that checks whether the adapter's RLock is held during call()."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
adapter_lock: threading.RLock,
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
super().__init__(tools=tools)
|
||||
self._adapter_lock = adapter_lock
|
||||
self.lock_was_held_during_call = False
|
||||
|
||||
def call(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if method == "tools/call":
|
||||
# Try to acquire the lock from a different thread.
|
||||
# If the lock is held by the calling thread, a different thread
|
||||
# will fail to acquire it (non-blocking).
|
||||
acquired_event = threading.Event()
|
||||
lock_held = [True]
|
||||
|
||||
def _try_acquire() -> None:
|
||||
got_it = self._adapter_lock.acquire(blocking=False)
|
||||
if got_it:
|
||||
lock_held[0] = False
|
||||
self._adapter_lock.release()
|
||||
acquired_event.set()
|
||||
|
||||
t = threading.Thread(target=_try_acquire, daemon=True)
|
||||
t.start()
|
||||
t.join(timeout=1.0)
|
||||
acquired_event.wait(timeout=1.0)
|
||||
self.lock_was_held_during_call = lock_held[0]
|
||||
|
||||
return super().call(method, params)
|
||||
|
||||
|
||||
class LockCheckingDiscoveryTransport(MockMCPTransport):
|
||||
"""Transport that checks whether the adapter's RLock is held during tools/list."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
adapter_lock: threading.RLock,
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
super().__init__(tools=tools)
|
||||
self._adapter_lock = adapter_lock
|
||||
self.lock_was_held_during_call = False
|
||||
|
||||
def call(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if method == "tools/list":
|
||||
acquired_event = threading.Event()
|
||||
lock_held = [True]
|
||||
|
||||
def _try_acquire() -> None:
|
||||
got_it = self._adapter_lock.acquire(blocking=False)
|
||||
if got_it:
|
||||
lock_held[0] = False
|
||||
self._adapter_lock.release()
|
||||
acquired_event.set()
|
||||
|
||||
t = threading.Thread(target=_try_acquire, daemon=True)
|
||||
t.start()
|
||||
t.join(timeout=1.0)
|
||||
acquired_event.wait(timeout=1.0)
|
||||
self.lock_was_held_during_call = lock_held[0]
|
||||
|
||||
return super().call(method, params)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Given Steps
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
'a connected MCP adapter with a slow transport tool "{tool_name}" taking {delay:f} seconds'
|
||||
)
|
||||
def step_slow_transport_tool(context: Context, tool_name: str, delay: float) -> None:
|
||||
tools = [_mock_tool(tool_name)]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
context.mcp_transport = SlowMCPTransport(delay=delay, tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=context.mcp_transport)
|
||||
context.mcp_adapter.connect()
|
||||
context.mcp_adapter.discover_tools()
|
||||
|
||||
|
||||
@given(
|
||||
'a connected MCP adapter with a concurrency-tracking transport tool "{tool_name}" taking {delay:f} seconds'
|
||||
)
|
||||
def step_concurrency_tracking_transport_tool(
|
||||
context: Context, tool_name: str, delay: float
|
||||
) -> None:
|
||||
tools = [_mock_tool(tool_name)]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
context.mcp_transport = ConcurrencyTrackingTransport(delay=delay, tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=context.mcp_transport)
|
||||
context.mcp_adapter.connect()
|
||||
context.mcp_adapter.discover_tools()
|
||||
|
||||
|
||||
@given(
|
||||
"a connected MCP adapter with a slow discovery transport taking {delay:f} seconds"
|
||||
)
|
||||
def step_slow_discovery_transport(context: Context, delay: float) -> None:
|
||||
tools = [_mock_tool("tool_0"), _mock_tool("tool_1")]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
context.mcp_transport = SlowMCPTransport(delay=delay, tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=context.mcp_transport)
|
||||
context.mcp_adapter.connect()
|
||||
|
||||
|
||||
@given(
|
||||
"a connected MCP adapter with a concurrency-tracking discovery transport taking {delay:f} seconds"
|
||||
)
|
||||
def step_concurrency_tracking_discovery_transport(
|
||||
context: Context, delay: float
|
||||
) -> None:
|
||||
tools = [_mock_tool("tool_0"), _mock_tool("tool_1")]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
context.mcp_transport = ConcurrencyTrackingTransport(delay=delay, tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=context.mcp_transport)
|
||||
context.mcp_adapter.connect()
|
||||
|
||||
|
||||
@given('a connected MCP adapter with a lock-checking transport tool "{tool_name}"')
|
||||
def step_lock_checking_invoke_transport(context: Context, tool_name: str) -> None:
|
||||
tools = [_mock_tool(tool_name)]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
# Create adapter first to get its lock, then set up transport
|
||||
transport_placeholder = MockMCPTransport(tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=transport_placeholder)
|
||||
# Now create the lock-checking transport with the adapter's actual lock
|
||||
lock_transport = LockCheckingInvokeTransport(
|
||||
adapter_lock=context.mcp_adapter._lock,
|
||||
tools=tools,
|
||||
)
|
||||
# Replace the transport
|
||||
context.mcp_adapter._transport = lock_transport
|
||||
context.mcp_transport = lock_transport
|
||||
context.mcp_adapter._connected = True
|
||||
context.mcp_adapter._capabilities = {"tools": True}
|
||||
context.mcp_adapter.discover_tools()
|
||||
|
||||
|
||||
@given("a connected MCP adapter with a lock-checking discovery transport")
|
||||
def step_lock_checking_discovery_transport(context: Context) -> None:
|
||||
tools = [_mock_tool("tool_0"), _mock_tool("tool_1")]
|
||||
config = MCPServerConfig(name="test-server", transport="stdio", command="echo")
|
||||
transport_placeholder = MockMCPTransport(tools=tools)
|
||||
context.mcp_adapter = MCPToolAdapter(config=config, transport=transport_placeholder)
|
||||
lock_transport = LockCheckingDiscoveryTransport(
|
||||
adapter_lock=context.mcp_adapter._lock,
|
||||
tools=tools,
|
||||
)
|
||||
context.mcp_adapter._transport = lock_transport
|
||||
context.mcp_transport = lock_transport
|
||||
context.mcp_adapter._connected = True
|
||||
context.mcp_adapter._capabilities = {"tools": True}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# When Steps
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I invoke "{tool_name}" concurrently from {count:d} threads')
|
||||
def step_concurrent_invoke(context: Context, tool_name: str, count: int) -> None:
|
||||
# Use threading.Thread directly instead of ThreadPoolExecutor to avoid
|
||||
# deadlocks in forked worker processes (behave-parallel uses fork).
|
||||
# ThreadPoolExecutor holds internal locks that can be in a locked state
|
||||
# after fork(), causing child processes to deadlock on acquisition.
|
||||
results: list[MCPToolResult | None] = [None] * count
|
||||
errors: list[BaseException | None] = [None] * count
|
||||
errors_lock = threading.Lock()
|
||||
|
||||
def _invoke(idx: int) -> None:
|
||||
try:
|
||||
results[idx] = context.mcp_adapter.invoke(tool_name, {})
|
||||
except BaseException as exc:
|
||||
with errors_lock:
|
||||
errors[idx] = exc
|
||||
|
||||
threads = [threading.Thread(target=_invoke, args=(i,)) for i in range(count)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
# Re-raise the first error encountered, if any
|
||||
for err in errors:
|
||||
if err is not None:
|
||||
raise err
|
||||
|
||||
context.concurrent_invoke_results = [r for r in results if r is not None]
|
||||
|
||||
|
||||
@when("I call discover_tools concurrently from {count:d} threads")
|
||||
def step_concurrent_discover(context: Context, count: int) -> None:
|
||||
# Use threading.Thread directly instead of ThreadPoolExecutor to avoid
|
||||
# deadlocks in forked worker processes (behave-parallel uses fork).
|
||||
# ThreadPoolExecutor holds internal locks that can be in a locked state
|
||||
# after fork(), causing child processes to deadlock on acquisition.
|
||||
results: list[Any] = []
|
||||
errors: list[Exception] = []
|
||||
collect_lock = threading.Lock()
|
||||
|
||||
def _discover() -> None:
|
||||
try:
|
||||
result = context.mcp_adapter.discover_tools()
|
||||
with collect_lock:
|
||||
results.append(result)
|
||||
except Exception as exc:
|
||||
with collect_lock:
|
||||
errors.append(exc)
|
||||
|
||||
threads = [threading.Thread(target=_discover) for _ in range(count)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
context.concurrent_discover_results = results
|
||||
context.concurrent_discover_errors = errors
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Then Steps
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
|
||||
@then("all {count:d} concurrent invocations should succeed")
|
||||
def step_all_invocations_succeed(context: Context, count: int) -> None:
|
||||
results = context.concurrent_invoke_results
|
||||
assert len(results) == count, f"Expected {count} results, got {len(results)}"
|
||||
for i, result in enumerate(results):
|
||||
assert result.success, f"Invocation {i} failed: {result.error}"
|
||||
|
||||
|
||||
@then(
|
||||
"at least {min_concurrent:d} invocations should have been in-flight simultaneously"
|
||||
)
|
||||
def step_at_least_n_invocations_concurrent(
|
||||
context: Context, min_concurrent: int
|
||||
) -> None:
|
||||
transport = context.mcp_transport
|
||||
actual = transport.max_concurrent
|
||||
assert actual >= min_concurrent, (
|
||||
f"Expected at least {min_concurrent} concurrent in-flight invocations, "
|
||||
f"but observed max_concurrent={actual}. "
|
||||
f"The adapter's RLock appears to be held during transport calls, "
|
||||
f"serializing concurrent operations."
|
||||
)
|
||||
|
||||
|
||||
@then("all {count:d} concurrent discoveries should succeed")
|
||||
def step_all_discoveries_succeed(context: Context, count: int) -> None:
|
||||
results = context.concurrent_discover_results
|
||||
errors = context.concurrent_discover_errors
|
||||
assert not errors, f"Expected no errors, got {len(errors)}: {errors}"
|
||||
assert len(results) == count, f"Expected {count} results, got {len(results)}"
|
||||
|
||||
|
||||
@then(
|
||||
"at least {min_concurrent:d} discoveries should have been in-flight simultaneously"
|
||||
)
|
||||
def step_at_least_n_discoveries_concurrent(
|
||||
context: Context, min_concurrent: int
|
||||
) -> None:
|
||||
transport = context.mcp_transport
|
||||
actual = transport.max_concurrent
|
||||
assert actual >= min_concurrent, (
|
||||
f"Expected at least {min_concurrent} concurrent in-flight discoveries, "
|
||||
f"but observed max_concurrent={actual}. "
|
||||
f"The adapter's RLock appears to be held during transport calls, "
|
||||
f"serializing concurrent operations."
|
||||
)
|
||||
|
||||
|
||||
@then("the lock should not have been held during the transport call")
|
||||
def step_lock_not_held_during_invoke(context: Context) -> None:
|
||||
transport = context.mcp_transport
|
||||
assert not transport.lock_was_held_during_call, (
|
||||
"The adapter's RLock was held during the transport.call() invocation. "
|
||||
"The lock should be released before making the transport call."
|
||||
)
|
||||
|
||||
|
||||
@then("the lock should not have been held during the discovery transport call")
|
||||
def step_lock_not_held_during_discovery(context: Context) -> None:
|
||||
transport = context.mcp_transport
|
||||
assert not transport.lock_was_held_during_call, (
|
||||
"The adapter's RLock was held during the transport.call('tools/list') invocation. "
|
||||
"The lock should be released before making the transport call."
|
||||
)
|
||||
@@ -0,0 +1,52 @@
|
||||
@tdd_issue @tdd_issue_10512
|
||||
Feature: MCPToolAdapter releases RLock during transport calls
|
||||
As a developer using MCPToolAdapter concurrently
|
||||
I want the adapter to release its RLock before making transport calls
|
||||
So that concurrent operations are not blocked by slow transport calls
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Bug #10512: MCPToolAdapter holds RLock during entire transport call,
|
||||
# blocking concurrent operations.
|
||||
#
|
||||
# The fix releases the lock before the transport.call() and
|
||||
# re-acquires it only when shared state must be mutated.
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
Scenario: Concurrent invoke calls are not blocked by a slow transport
|
||||
Given a connected MCP adapter with a concurrency-tracking transport tool "slow_op" taking 0.05 seconds
|
||||
When I invoke "slow_op" concurrently from 3 threads
|
||||
Then all 3 concurrent invocations should succeed
|
||||
And at least 2 invocations should have been in-flight simultaneously
|
||||
|
||||
Scenario: Concurrent discover_tools calls are not blocked by a slow transport
|
||||
Given a connected MCP adapter with a concurrency-tracking discovery transport taking 0.05 seconds
|
||||
When I call discover_tools concurrently from 3 threads
|
||||
Then all 3 concurrent discoveries should succeed
|
||||
And at least 2 discoveries should have been in-flight simultaneously
|
||||
|
||||
Scenario: Lock is not held during invoke transport call
|
||||
Given a connected MCP adapter with a lock-checking transport tool "check_lock"
|
||||
When I invoke "check_lock" with arguments {}
|
||||
Then the invocation should succeed
|
||||
And the lock should not have been held during the transport call
|
||||
|
||||
Scenario: Lock is not held during discover_tools transport call
|
||||
Given a connected MCP adapter with a lock-checking discovery transport
|
||||
When I discover tools from the adapter
|
||||
Then the lock should not have been held during the discovery transport call
|
||||
|
||||
Scenario: Invoke still validates under lock before transport call
|
||||
Given a connected MCP adapter with a slow transport tool "slow_op" taking 0.1 seconds
|
||||
When I invoke "nonexistent_tool" with arguments {}
|
||||
Then the invocation should fail
|
||||
And the invocation error should mention "not found"
|
||||
|
||||
Scenario: Invoke on disconnected adapter still raises under lock
|
||||
Given an MCP adapter with a mock transport
|
||||
When I invoke MCP tool "any_tool" while disconnected
|
||||
Then the adapter error should mention "not connected"
|
||||
|
||||
Scenario: Discover on disconnected adapter still raises under lock
|
||||
Given an MCP adapter with a mock transport
|
||||
When I discover tools expecting an error
|
||||
Then the adapter error should mention "not connected"
|
||||
+3
-1
@@ -30,6 +30,7 @@ nav:
|
||||
- Invariant Reconciliation: modules/invariant-reconciliation.md
|
||||
- ACMS Context Hydration: modules/context-hydration.md
|
||||
- Git Worktree Sandbox: modules/git-worktree-sandbox.md
|
||||
- Devcontainer Auto-Discovery: modules/devcontainer-discovery.md
|
||||
- Showcase:
|
||||
- Overview: showcase/index.md
|
||||
- CLI Tools: showcase/cli-tools/
|
||||
@@ -60,6 +61,7 @@ nav:
|
||||
- Reference/Command Input & Sessions: tui/input-and-sessions.md
|
||||
- Configuration, Key Bindings & Integration: tui/configuration-and-integration.md
|
||||
- FAQ: faq.md
|
||||
- Quick Start: quickstart.md
|
||||
- Changelog: CHANGELOG.md
|
||||
- Contributing: CONTRIBUTING.md
|
||||
- Reference: reference/
|
||||
@@ -92,7 +94,7 @@ nav:
|
||||
- ADR-025 Observability & Logging: adr/ADR-025-observability-and-logging.md
|
||||
- ADR-026 Agent-to-Agent Protocol (A2A): adr/ADR-026-agent-client-protocol.md
|
||||
- ADR-027 Language Server Protocol (LSP) Integration: adr/ADR-027-language-server-protocol.md
|
||||
- ADR-028 Agent Skills Standard (AgentSkills.io): adr/ADR-028-agent-skills-standard.md
|
||||
- ADR-028 Skill/Tool Abstraction Definition: adr/ADR-028-agent-skills-standard.md
|
||||
- ADR-029 Model Context Protocol (MCP) Adoption: adr/ADR-029-model-context-protocol.md
|
||||
- ADR-030 Skill Abstraction Definition: adr/ADR-030-skill-abstraction-definition.md
|
||||
- ADR-031 Actor Abstraction Definition: adr/ADR-031-actor-abstraction-definition.md
|
||||
|
||||
@@ -9,7 +9,7 @@ Suite Teardown Cleanup Test Environment
|
||||
*** Test Cases ***
|
||||
Noxfile Contains Coverage Threshold Constant
|
||||
[Documentation] Verify COVERAGE_THRESHOLD = 96.5 is defined in noxfile.py
|
||||
[Tags] coverage config
|
||||
[Tags] coverage config tdd_issue tdd_issue_4305
|
||||
|
||||
${content}= Get File ${WORKSPACE}/noxfile.py
|
||||
Should Contain ${content} COVERAGE_THRESHOLD = 96.5
|
||||
|
||||
@@ -174,6 +174,12 @@ class MCPTransport:
|
||||
class MCPToolAdapter:
|
||||
"""Adapter bridging an MCP server into the CleverAgents ToolRegistry.
|
||||
|
||||
Thread-safety strategy: the adapter's ``RLock`` protects shared
|
||||
mutable state (``_connected``, ``_capabilities``, ``_tools``,
|
||||
``_notification_listeners``). The lock is **released** before
|
||||
making transport calls (``_transport.call()``) so that concurrent
|
||||
operations are not blocked by slow network I/O. See bug #10512.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
config:
|
||||
@@ -411,6 +417,10 @@ class MCPToolAdapter:
|
||||
) -> list[MCPToolDescriptor]:
|
||||
"""Enumerate tools from the connected MCP server.
|
||||
|
||||
The adapter's RLock is held only for state validation and
|
||||
mutation — it is released before the transport call so that
|
||||
concurrent operations are not blocked by slow network I/O.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tool_filter:
|
||||
@@ -421,6 +431,7 @@ class MCPToolAdapter:
|
||||
RuntimeError
|
||||
If the adapter is not connected.
|
||||
"""
|
||||
# Phase 1: Validate connection state under lock.
|
||||
with self._lock:
|
||||
if not self._connected:
|
||||
msg = (
|
||||
@@ -428,25 +439,30 @@ class MCPToolAdapter:
|
||||
f"Call connect() first."
|
||||
)
|
||||
raise RuntimeError(msg)
|
||||
transport = self._transport
|
||||
|
||||
result = self._transport.call("tools/list", {})
|
||||
raw_tools = result.get("tools", [])
|
||||
# Phase 2: Transport call WITHOUT lock — allows concurrent access.
|
||||
result = transport.call("tools/list", {})
|
||||
raw_tools = result.get("tools", [])
|
||||
|
||||
descriptors: list[MCPToolDescriptor] = []
|
||||
for raw in raw_tools:
|
||||
desc = MCPToolDescriptor(
|
||||
name=raw.get("name", ""),
|
||||
description=raw.get("description", ""),
|
||||
input_schema=raw.get("inputSchema", {}),
|
||||
annotations=raw.get("annotations", {}),
|
||||
)
|
||||
descriptors.append(desc)
|
||||
descriptors: list[MCPToolDescriptor] = []
|
||||
for raw in raw_tools:
|
||||
desc = MCPToolDescriptor(
|
||||
name=raw.get("name", ""),
|
||||
description=raw.get("description", ""),
|
||||
input_schema=raw.get("inputSchema", {}),
|
||||
annotations=raw.get("annotations", {}),
|
||||
)
|
||||
descriptors.append(desc)
|
||||
|
||||
if tool_filter:
|
||||
descriptors = self._apply_filter(descriptors, tool_filter)
|
||||
if tool_filter:
|
||||
descriptors = self._apply_filter(descriptors, tool_filter)
|
||||
|
||||
# Phase 3: Update shared state under lock.
|
||||
with self._lock:
|
||||
self._tools = {d.name: d for d in descriptors}
|
||||
return descriptors
|
||||
|
||||
return descriptors
|
||||
|
||||
def invoke(
|
||||
self,
|
||||
@@ -456,7 +472,9 @@ class MCPToolAdapter:
|
||||
"""Invoke a discovered MCP tool by name.
|
||||
|
||||
Validates inputs against the tool's JSON Schema before calling
|
||||
the server.
|
||||
the server. The adapter's RLock is held only for state
|
||||
validation — it is released before the transport call so that
|
||||
concurrent invocations are not blocked by slow network I/O.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -470,6 +488,7 @@ class MCPToolAdapter:
|
||||
MCPToolResult
|
||||
Invocation outcome with success flag, data, and timing.
|
||||
"""
|
||||
# Phase 1: Validate state and inputs under lock.
|
||||
with self._lock:
|
||||
if not self._connected:
|
||||
msg = (
|
||||
@@ -494,57 +513,61 @@ class MCPToolAdapter:
|
||||
f"{validation_error}",
|
||||
)
|
||||
|
||||
start = time.monotonic()
|
||||
try:
|
||||
result = self._transport.call(
|
||||
"tools/call",
|
||||
{"name": tool_name, "arguments": arguments},
|
||||
)
|
||||
except TimeoutError as exc:
|
||||
elapsed = (time.monotonic() - start) * 1000
|
||||
return MCPToolResult(
|
||||
success=False,
|
||||
error=f"Tool '{tool_name}' timeout: {exc}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
except Exception as exc:
|
||||
elapsed = (time.monotonic() - start) * 1000
|
||||
return MCPToolResult(
|
||||
success=False,
|
||||
error=f"MCP server error invoking '{tool_name}': {exc}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
transport = self._transport
|
||||
|
||||
# Phase 2: Transport call WITHOUT lock — allows concurrent access.
|
||||
start = time.monotonic()
|
||||
try:
|
||||
result = transport.call(
|
||||
"tools/call",
|
||||
{"name": tool_name, "arguments": arguments},
|
||||
)
|
||||
except TimeoutError as exc:
|
||||
elapsed = (time.monotonic() - start) * 1000
|
||||
|
||||
if result.get("isError"):
|
||||
content = result.get("content", [])
|
||||
if content and isinstance(content, list) and len(content) > 0:
|
||||
error_text = content[0].get("text", "unknown error")
|
||||
else:
|
||||
error_text = "unknown error"
|
||||
return MCPToolResult(
|
||||
success=False,
|
||||
error=f"MCP server error: {error_text}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
|
||||
# MCP 1.4.0 returns ``content`` as a list of ContentItem dicts
|
||||
# (e.g. ``[{"type": "text", "text": "..."}]``). Normalise to a
|
||||
# plain dict so that ``MCPToolResult.data`` is always
|
||||
# ``dict[str, Any]`` and downstream code can rely on dict access.
|
||||
raw_content = result.get("content", [])
|
||||
if isinstance(raw_content, list):
|
||||
normalised: dict[str, Any] = {"content": raw_content}
|
||||
elif isinstance(raw_content, dict):
|
||||
normalised = raw_content
|
||||
else:
|
||||
# Unexpected type — wrap it so the dict contract holds.
|
||||
normalised = {"content": raw_content}
|
||||
return MCPToolResult(
|
||||
success=True,
|
||||
data=normalised,
|
||||
success=False,
|
||||
error=f"Tool '{tool_name}' timeout: {exc}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
except Exception as exc:
|
||||
elapsed = (time.monotonic() - start) * 1000
|
||||
return MCPToolResult(
|
||||
success=False,
|
||||
error=f"MCP server error invoking '{tool_name}': {exc}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
elapsed = (time.monotonic() - start) * 1000
|
||||
|
||||
# Phase 3: Process result (no lock needed — local variables only).
|
||||
if result.get("isError"):
|
||||
content = result.get("content", [])
|
||||
if content and isinstance(content, list) and len(content) > 0:
|
||||
error_text = content[0].get("text", "unknown error")
|
||||
else:
|
||||
error_text = "unknown error"
|
||||
return MCPToolResult(
|
||||
success=False,
|
||||
error=f"MCP server error: {error_text}",
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
|
||||
# MCP 1.4.0 returns ``content`` as a list of ContentItem dicts
|
||||
# (e.g. ``[{"type": "text", "text": "..."}]``). Normalise to a
|
||||
# plain dict so that ``MCPToolResult.data`` is always
|
||||
# ``dict[str, Any]`` and downstream code can rely on dict access.
|
||||
raw_content = result.get("content", [])
|
||||
if isinstance(raw_content, list):
|
||||
normalised: dict[str, Any] = {"content": raw_content}
|
||||
elif isinstance(raw_content, dict):
|
||||
normalised = raw_content
|
||||
else:
|
||||
# Unexpected type — wrap it so the dict contract holds.
|
||||
normalised = {"content": raw_content}
|
||||
return MCPToolResult(
|
||||
success=True,
|
||||
data=normalised,
|
||||
duration_ms=elapsed,
|
||||
)
|
||||
|
||||
def register_tools(
|
||||
self,
|
||||
|
||||
@@ -39,6 +39,21 @@ def _coerce_optional_str(value: object | None) -> str | None:
|
||||
return str(value)
|
||||
|
||||
|
||||
def _coerce_env_bool(value: str) -> bool:
|
||||
"""Consistent boolean coercion for environment variables.
|
||||
|
||||
Accepts ``true``, ``1``, ``yes``, and ``on`` (case-insensitive) as truthy.
|
||||
"""
|
||||
return value.strip().lower() in ("true", "1", "yes", "on")
|
||||
|
||||
|
||||
class _ApiKeyMissing:
|
||||
"""Sentinel distinguishing "api_key was not provided" from ``None``."""
|
||||
|
||||
|
||||
_API_KEY_MISSING = _ApiKeyMissing()
|
||||
|
||||
|
||||
class ProviderType(StrEnum):
|
||||
"""Supported AI provider types."""
|
||||
|
||||
@@ -165,7 +180,7 @@ class ProviderRegistry:
|
||||
supports_json_mode=False,
|
||||
),
|
||||
ProviderType.MOCK: ProviderCapabilities(
|
||||
supports_streaming=True,
|
||||
supports_streaming=False,
|
||||
supports_tool_calls=False,
|
||||
supports_vision=False,
|
||||
max_context_length=4096,
|
||||
@@ -240,6 +255,25 @@ class ProviderRegistry:
|
||||
)
|
||||
self._providers[provider_type] = provider_info
|
||||
|
||||
# Register MOCK with conditional ``is_configured``. The guard field
|
||||
# ``api_key_env_var`` points to ``CLEVERAGENTS_ALLOW_MOCK_PROVIDER``
|
||||
# semantically (it is the activation flag) even though it is not a
|
||||
# traditional API key.
|
||||
mock_allowed = _coerce_env_bool(
|
||||
os.environ.get("CLEVERAGENTS_ALLOW_MOCK_PROVIDER", "")
|
||||
)
|
||||
mock_info = ProviderInfo(
|
||||
provider_type=ProviderType.MOCK,
|
||||
name=ProviderType.MOCK.value.title(),
|
||||
api_key_env_var="CLEVERAGENTS_ALLOW_MOCK_PROVIDER",
|
||||
default_model=self.DEFAULT_MODELS.get(ProviderType.MOCK, "unknown"),
|
||||
capabilities=self.DEFAULT_CAPABILITIES.get(
|
||||
ProviderType.MOCK, ProviderCapabilities()
|
||||
),
|
||||
is_configured=mock_allowed,
|
||||
)
|
||||
self._providers[ProviderType.MOCK] = mock_info
|
||||
|
||||
def get_configured_providers(self) -> list[ProviderInfo]:
|
||||
"""Get list of all configured providers.
|
||||
|
||||
@@ -401,45 +435,30 @@ class ProviderRegistry:
|
||||
|
||||
return self.DEFAULT_MODELS.get(provider_type)
|
||||
|
||||
def create_llm(
|
||||
self,
|
||||
provider_type: ProviderType | str | None = None,
|
||||
model_id: str | None = None,
|
||||
**kwargs: object,
|
||||
) -> BaseLanguageModel:
|
||||
"""Create a LangChain LLM instance for a provider.
|
||||
def _get_api_key(self, provider_type: ProviderType) -> str | None:
|
||||
"""Get and validate the API key for a provider.
|
||||
|
||||
Args:
|
||||
provider_type: The provider type. Uses default if not specified.
|
||||
model_id: The model ID. Uses default for provider if not specified.
|
||||
**kwargs: Additional arguments passed to the LLM constructor.
|
||||
provider_type: The provider type to look up.
|
||||
|
||||
Returns:
|
||||
A LangChain BaseLanguageModel instance.
|
||||
The API key string, or ``None`` if the provider has no key
|
||||
requirement (e.g. ``MOCK`` when explicitly allowed).
|
||||
|
||||
Raises:
|
||||
ValueError: If no provider is configured or provider is invalid.
|
||||
ValueError: If the provider requires an API key but it is not
|
||||
configured, or if ``MOCK`` is requested without the
|
||||
``CLEVERAGENTS_ALLOW_MOCK_PROVIDER`` environment variable,
|
||||
or if the provider type has no API key mapping.
|
||||
"""
|
||||
# Resolve provider type
|
||||
if provider_type is None:
|
||||
provider_type = self.get_default_provider_type()
|
||||
if provider_type is None:
|
||||
raise ValueError(
|
||||
"No AI provider configured. Please set one of the following "
|
||||
"environment variables: OPENAI_API_KEY, ANTHROPIC_API_KEY, "
|
||||
"GOOGLE_API_KEY, or another supported provider key."
|
||||
)
|
||||
elif not isinstance(provider_type, ProviderType):
|
||||
try:
|
||||
provider_type = ProviderType(provider_type.lower())
|
||||
except ValueError as e:
|
||||
raise ValueError(f"Unknown provider type: {provider_type}") from e
|
||||
if provider_type == ProviderType.MOCK:
|
||||
if _coerce_env_bool(os.environ.get("CLEVERAGENTS_ALLOW_MOCK_PROVIDER", "")):
|
||||
return None
|
||||
raise ValueError(
|
||||
"Mock provider is not allowed in production. "
|
||||
"Set CLEVERAGENTS_ALLOW_MOCK_PROVIDER=true to enable it."
|
||||
)
|
||||
|
||||
# Resolve model ID
|
||||
if model_id is None:
|
||||
model_id = self.get_default_model(provider_type)
|
||||
|
||||
# Get API key
|
||||
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
|
||||
if key_attr:
|
||||
api_key = getattr(self._settings, key_attr, None)
|
||||
@@ -448,37 +467,62 @@ class ProviderRegistry:
|
||||
f"Provider {provider_type.value} is not configured. "
|
||||
f"Please set the {key_attr.upper()} environment variable."
|
||||
)
|
||||
return api_key
|
||||
|
||||
# Create the appropriate LLM
|
||||
return self._create_provider_llm(provider_type, model_id, **kwargs)
|
||||
# ProviderType is in the enum but not in PROVIDER_KEY_ATTRS
|
||||
raise ValueError(
|
||||
f"Provider {provider_type.value} has no API key mapping in "
|
||||
"PROVIDER_KEY_ATTRS."
|
||||
)
|
||||
|
||||
def _create_provider_llm(
|
||||
def _create_provider_instance(
|
||||
self,
|
||||
provider_type: ProviderType,
|
||||
model_id: str | None,
|
||||
**kwargs: Any,
|
||||
) -> BaseLanguageModel:
|
||||
"""Create a LangChain LLM for the specified provider.
|
||||
"""Single internal factory for provider instantiation.
|
||||
|
||||
This is the single source of truth for raw LLM instantiation. Both
|
||||
``create_llm()`` and ``create_ai_provider()`` delegate here so that
|
||||
adding a new provider requires touching exactly one place.
|
||||
|
||||
Args:
|
||||
provider_type: The provider type.
|
||||
model_id: The model ID.
|
||||
**kwargs: Additional arguments for the LLM.
|
||||
model_id: The model ID (provider default used when ``None``).
|
||||
**kwargs: Extra constructor arguments forwarded to the LLM.
|
||||
If ``api_key`` is present, it is consumed here instead of
|
||||
being fetched from settings. ``max_retries`` is consumed
|
||||
if present and never forwarded to LangChain constructors.
|
||||
|
||||
Returns:
|
||||
A LangChain BaseLanguageModel instance.
|
||||
A ``BaseLanguageModel`` instance for the requested provider.
|
||||
|
||||
Raises:
|
||||
ValueError: If the provider is not configured or not supported.
|
||||
"""
|
||||
# Import the appropriate LangChain module based on provider
|
||||
# Consume retry metadata so it never leaks into provider constructors.
|
||||
kwargs.pop("max_retries", None)
|
||||
|
||||
# If the caller explicitly passed api_key (even ``None``), use it
|
||||
# directly; otherwise fall back to _get_api_key.
|
||||
sentinel = kwargs.pop("__api_key_sentinel", _API_KEY_MISSING)
|
||||
if sentinel is _API_KEY_MISSING:
|
||||
api_key: str | None = self._get_api_key(provider_type)
|
||||
else:
|
||||
api_key = sentinel
|
||||
|
||||
if provider_type == ProviderType.OPENAI:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
return ChatOpenAI(model=model_id or "gpt-4o", **kwargs)
|
||||
return ChatOpenAI(model=model_id or "gpt-4o", api_key=api_key, **kwargs)
|
||||
|
||||
if provider_type == ProviderType.ANTHROPIC:
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
|
||||
return ChatAnthropic(
|
||||
model=model_id or "claude-sonnet-4-20250514",
|
||||
api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -487,6 +531,7 @@ class ProviderRegistry:
|
||||
|
||||
return ChatGoogleGenerativeAI(
|
||||
model=model_id or "gemini-2.0-flash",
|
||||
google_api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -522,6 +567,7 @@ class ProviderRegistry:
|
||||
deployment_name=deployment_name,
|
||||
azure_endpoint=azure_endpoint,
|
||||
api_version=api_version,
|
||||
api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -530,6 +576,7 @@ class ProviderRegistry:
|
||||
|
||||
return ChatGroq(
|
||||
model=model_id or "llama-3.1-70b-versatile",
|
||||
api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -538,6 +585,7 @@ class ProviderRegistry:
|
||||
|
||||
return ChatTogether(
|
||||
model=model_id or "meta-llama/Llama-3.1-70B-Instruct-Turbo",
|
||||
api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -546,17 +594,13 @@ class ProviderRegistry:
|
||||
|
||||
return ChatCohere(
|
||||
model=model_id or "command-r-plus",
|
||||
api_key=api_key,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
if provider_type == ProviderType.OPENROUTER:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
api_key = self._settings.openrouter_api_key
|
||||
if not api_key:
|
||||
raise ValueError(
|
||||
"Provider openrouter requires OPENROUTER_API_KEY to be set."
|
||||
)
|
||||
raw_headers = kwargs.pop("default_headers", None)
|
||||
sanitized_headers: dict[str, str] | None = (
|
||||
{str(k): str(v) for k, v in raw_headers.items()}
|
||||
@@ -581,8 +625,68 @@ class ProviderRegistry:
|
||||
openrouter_kwargs.update(kwargs)
|
||||
return ChatOpenAI(**openrouter_kwargs)
|
||||
|
||||
if provider_type == ProviderType.MOCK:
|
||||
from langchain_community.llms import FakeListLLM
|
||||
|
||||
return FakeListLLM(responses=["mock response"] * 10)
|
||||
|
||||
raise ValueError(f"Unsupported provider type: {provider_type}")
|
||||
|
||||
def _resolve_provider_type(
|
||||
self,
|
||||
provider_type: ProviderType | str | None = None,
|
||||
) -> ProviderType:
|
||||
"""Resolve a possibly ``None`` or string provider type to ``ProviderType``.
|
||||
|
||||
Uses the configured fallback chain when ``None`` and performs
|
||||
case-insensitive string matching otherwise.
|
||||
|
||||
Raises:
|
||||
ValueError: If no provider is configured or the name is unknown.
|
||||
"""
|
||||
if provider_type is None:
|
||||
resolved = self.get_default_provider_type()
|
||||
if resolved is None:
|
||||
raise ValueError(
|
||||
"No AI provider configured. Please set one of the following "
|
||||
"environment variables: OPENAI_API_KEY, ANTHROPIC_API_KEY, "
|
||||
"GOOGLE_API_KEY, or another supported provider key."
|
||||
)
|
||||
return resolved
|
||||
if not isinstance(provider_type, ProviderType):
|
||||
try:
|
||||
return ProviderType(provider_type.lower())
|
||||
except ValueError as e:
|
||||
raise ValueError(f"Unknown provider type: {provider_type}") from e
|
||||
return provider_type
|
||||
|
||||
def create_llm(
|
||||
self,
|
||||
provider_type: ProviderType | str | None = None,
|
||||
model_id: str | None = None,
|
||||
**kwargs: Any,
|
||||
) -> BaseLanguageModel:
|
||||
"""Create a LangChain LLM instance for a provider.
|
||||
|
||||
Args:
|
||||
provider_type: The provider type. Uses default if not specified.
|
||||
model_id: The model ID. Uses default for provider if not specified.
|
||||
**kwargs: Additional arguments passed to the LLM constructor.
|
||||
|
||||
Returns:
|
||||
A LangChain BaseLanguageModel instance.
|
||||
|
||||
Raises:
|
||||
ValueError: If no provider is configured or provider is invalid.
|
||||
"""
|
||||
resolved = self._resolve_provider_type(provider_type)
|
||||
|
||||
if model_id is None:
|
||||
model_id = self.get_default_model(resolved)
|
||||
|
||||
# Delegate to the unified factory (API key validation happens inside)
|
||||
return self._create_provider_instance(resolved, model_id, **kwargs)
|
||||
|
||||
def create_ai_provider(
|
||||
self,
|
||||
provider_type: ProviderType | str | None = None,
|
||||
@@ -591,8 +695,9 @@ class ProviderRegistry:
|
||||
) -> AIProviderInterface:
|
||||
"""Create an AIProviderInterface instance for a provider.
|
||||
|
||||
This creates a LangChainChatProvider that wraps the LangChain LLM
|
||||
with the AIProviderInterface protocol.
|
||||
All providers are wrapped uniformly in ``LangChainChatProvider``.
|
||||
Raw LLM instantiation is delegated to ``_create_provider_instance()``
|
||||
so that adding a new provider requires touching exactly one place.
|
||||
|
||||
Args:
|
||||
provider_type: The provider type. Uses default if not specified.
|
||||
@@ -609,135 +714,34 @@ class ProviderRegistry:
|
||||
LangChainChatProvider,
|
||||
)
|
||||
|
||||
# Resolve types
|
||||
if provider_type is None:
|
||||
provider_type = self.get_default_provider_type()
|
||||
if provider_type is None:
|
||||
raise ValueError(
|
||||
"No AI provider configured. Please set one of the following "
|
||||
"environment variables: OPENAI_API_KEY, ANTHROPIC_API_KEY, "
|
||||
"GOOGLE_API_KEY, or another supported provider key."
|
||||
)
|
||||
elif not isinstance(provider_type, ProviderType):
|
||||
try:
|
||||
provider_type = ProviderType(provider_type.lower())
|
||||
except ValueError as e:
|
||||
raise ValueError(f"Unknown provider type: {provider_type}") from e
|
||||
resolved = self._resolve_provider_type(provider_type)
|
||||
|
||||
if model_id is None:
|
||||
model_id = self.get_default_model(provider_type)
|
||||
model_id = self.get_default_model(resolved)
|
||||
|
||||
provider_info = self.get_provider_info(provider_type)
|
||||
provider_info = self.get_provider_info(resolved)
|
||||
capabilities = (
|
||||
provider_info.capabilities if provider_info else ProviderCapabilities()
|
||||
)
|
||||
|
||||
if provider_type == ProviderType.OPENAI:
|
||||
from cleveragents.providers.llm.openai_provider import OpenAIChatProvider
|
||||
|
||||
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
|
||||
api_key = getattr(self._settings, key_attr, None) if key_attr else None
|
||||
if not api_key:
|
||||
missing_env = (
|
||||
key_attr.upper() if key_attr else provider_type.value.upper()
|
||||
)
|
||||
raise ValueError(
|
||||
f"Provider {provider_type.value} is not configured. "
|
||||
f"Please set the {missing_env} environment variable."
|
||||
)
|
||||
|
||||
return OpenAIChatProvider(
|
||||
api_key=api_key,
|
||||
model=model_id or self.DEFAULT_MODELS.get(provider_type, "gpt-4o"),
|
||||
max_retries=max_retries,
|
||||
)
|
||||
|
||||
if provider_type == ProviderType.ANTHROPIC:
|
||||
from cleveragents.providers.llm.anthropic_provider import (
|
||||
AnthropicChatProvider,
|
||||
)
|
||||
|
||||
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
|
||||
api_key = getattr(self._settings, key_attr, None) if key_attr else None
|
||||
if not api_key:
|
||||
missing_env = (
|
||||
key_attr.upper() if key_attr else provider_type.value.upper()
|
||||
)
|
||||
raise ValueError(
|
||||
f"Provider {provider_type.value} is not configured. "
|
||||
f"Please set the {missing_env} environment variable."
|
||||
)
|
||||
|
||||
return AnthropicChatProvider(
|
||||
api_key=api_key,
|
||||
model=model_id
|
||||
or self.DEFAULT_MODELS.get(provider_type, "claude-sonnet-4-20250514"),
|
||||
max_retries=max_retries,
|
||||
)
|
||||
|
||||
if provider_type == ProviderType.GOOGLE:
|
||||
from cleveragents.providers.llm.google_provider import GoogleChatProvider
|
||||
|
||||
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
|
||||
api_key = getattr(self._settings, key_attr, None) if key_attr else None
|
||||
if not api_key:
|
||||
missing_env = (
|
||||
key_attr.upper() if key_attr else provider_type.value.upper()
|
||||
)
|
||||
raise ValueError(
|
||||
f"Provider {provider_type.value} is not configured. "
|
||||
f"Please set the {missing_env} environment variable."
|
||||
)
|
||||
|
||||
return GoogleChatProvider(
|
||||
api_key=api_key,
|
||||
model=model_id
|
||||
or self.DEFAULT_MODELS.get(provider_type, "gemini-2.0-flash"),
|
||||
max_retries=max_retries,
|
||||
)
|
||||
|
||||
if provider_type == ProviderType.OPENROUTER:
|
||||
from cleveragents.providers.llm.openrouter_provider import (
|
||||
OpenRouterChatProvider,
|
||||
)
|
||||
|
||||
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
|
||||
api_key = getattr(self._settings, key_attr, None) if key_attr else None
|
||||
if not api_key:
|
||||
missing_env = (
|
||||
key_attr.upper() if key_attr else provider_type.value.upper()
|
||||
)
|
||||
raise ValueError(
|
||||
f"Provider {provider_type.value} is not configured. "
|
||||
f"Please set the {missing_env} environment variable."
|
||||
)
|
||||
|
||||
return OpenRouterChatProvider(
|
||||
api_key=api_key,
|
||||
model=model_id
|
||||
or self.DEFAULT_MODELS.get(
|
||||
provider_type, "anthropic/claude-sonnet-4-20250514"
|
||||
),
|
||||
organization=self._settings.openrouter_organization,
|
||||
max_retries=max_retries,
|
||||
)
|
||||
|
||||
# Create factory function for the LLM.
|
||||
# Capture the narrowed ProviderType in a local variable so the closure
|
||||
# holds a ProviderType rather than the wider ProviderType | str | None.
|
||||
resolved_provider_type: ProviderType = provider_type
|
||||
# Eager validation: fail fast before building the factory closure,
|
||||
# then reuse the validated key to avoid a second lookup.
|
||||
api_key: str | None = self._get_api_key(resolved)
|
||||
|
||||
def llm_factory(mid: str) -> BaseLanguageModel:
|
||||
factory_kwargs: dict[str, Any] = {"max_retries": max_retries}
|
||||
return self._create_provider_llm(
|
||||
resolved_provider_type,
|
||||
# Use sentinel so _create_provider_instance knows the key was
|
||||
# pre-validated and should not trigger a second _get_api_key.
|
||||
factory_kwargs["__api_key_sentinel"] = api_key
|
||||
return self._create_provider_instance(
|
||||
resolved,
|
||||
mid,
|
||||
**factory_kwargs,
|
||||
)
|
||||
|
||||
return LangChainChatProvider(
|
||||
name=provider_type.value,
|
||||
model_id=model_id or self.DEFAULT_MODELS.get(provider_type, "unknown"),
|
||||
name=resolved.value,
|
||||
model_id=model_id or self.DEFAULT_MODELS.get(resolved, "unknown"),
|
||||
llm_factory=llm_factory,
|
||||
max_retries=max_retries,
|
||||
supports_streaming=capabilities.supports_streaming,
|
||||
|
||||
@@ -37,6 +37,7 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -65,16 +66,30 @@ _GIT_ENV: dict[str, str] = {
|
||||
# Cached merge of ``os.environ`` with ``_GIT_ENV``, populated on first use.
|
||||
_BASE_ENV: dict[str, str] | None = None
|
||||
|
||||
# Lock that serialises the one-time initialisation of ``_BASE_ENV``.
|
||||
# The outer ``if _BASE_ENV is None`` check keeps the warm-cache path
|
||||
# lock-free; the inner check inside the ``with`` block prevents duplicate
|
||||
# initialisation when two threads race on the very first call.
|
||||
_BASE_ENV_LOCK: threading.Lock = threading.Lock()
|
||||
|
||||
|
||||
def _get_base_env() -> dict[str, str]:
|
||||
"""Return a cached merge of the process environment with git overrides.
|
||||
|
||||
The snapshot is taken on first call and reused thereafter, avoiding a
|
||||
full ``os.environ`` copy on every git invocation.
|
||||
|
||||
Thread safety: double-checked locking eliminates the TOCTOU race where
|
||||
two concurrent threads could both observe ``_BASE_ENV is None``, both
|
||||
snapshot ``os.environ``, and write potentially different snapshots.
|
||||
The outer check keeps the warm-cache path lock-free; the inner check
|
||||
inside the lock prevents duplicate initialisation on the first call.
|
||||
"""
|
||||
global _BASE_ENV
|
||||
if _BASE_ENV is None:
|
||||
_BASE_ENV = {**os.environ, **_GIT_ENV}
|
||||
with _BASE_ENV_LOCK:
|
||||
if _BASE_ENV is None:
|
||||
_BASE_ENV = {**os.environ, **_GIT_ENV}
|
||||
return _BASE_ENV
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user