Compare commits
13 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b692894c88 | |||
| 2f01e7d625 | |||
| 63746b4d30 | |||
| 05acc26c40 | |||
| a1ea40d2e7 | |||
| 87a7ce35d7 | |||
|
78be08870c
|
|||
| 5ee08ea946 | |||
| 6e1646d565 | |||
| 815f546bd2 | |||
| f78c1c2c98 | |||
| 3f0ce3d20a | |||
| 2cba7d41bc |
@@ -5,6 +5,17 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- **`agents session tell` invokes real LLM orchestrator actor** (#5784): Replaced the
|
||||
M3 echo-stub with real actor invocation via `SessionWorkflow`, routing through
|
||||
`LangChainSessionCaller` → `ToolCallingRuntime.run_tool_loop()`. The user prompt
|
||||
is sent to the session's bound orchestrator actor (or `--actor` override), the
|
||||
assistant's response is persisted via `SessionService.append_message()`, and token
|
||||
usage is tracked via `SessionService.update_token_usage()`. Output includes a
|
||||
**Usage** panel (Rich/Plain) or `usage` object (JSON/YAML) with input tokens,
|
||||
output tokens, estimated cost, and duration. The `--stream` flag produces real
|
||||
LLM streaming output. A `SessionActorNotConfiguredError` is raised with exit
|
||||
code 1 when no actor is configured.
|
||||
|
||||
- Fixed `ReactiveEventBus.emit()` exception handler to log the full exception
|
||||
message (`str(exc)`) and enable traceback forwarding (`exc_info=True`).
|
||||
Previously the handler logged only the exception type name (e.g.
|
||||
@@ -13,7 +24,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
traceback in the structlog warning entry. Removed `@tdd_expected_fail` tag
|
||||
from the TDD test so both scenarios run as normal regression guards. (#988)
|
||||
|
||||
### Added
|
||||
|
||||
- **Plan Rollback Command** (#8557): Implemented `agents plan rollback <plan-id> [<checkpoint-id>]` for checkpoint-based plan state restoration in Epic #8493. The command restores a plan's sandbox to the state captured at a given checkpoint, discarding all decisions made after that checkpoint. The checkpoint can be specified as an optional positional second argument or via the `--to-checkpoint` named option. Supports `--yes/-y` flag to skip confirmation prompts and `--format/-f` for output format selection (rich/plain/json/yaml). Included with comprehensive BDD test coverage (>= 97%) and spec-aligned output formatting showing rollback summary, changes reverted, impact analysis, and post-rollback state panels.
|
||||
|
||||
### Fixed
|
||||
- **Actor configuration validation incorrectly requires top-level provider field** (#4300):
|
||||
Actor configuration in V3 is now obtained from the nested configuration
|
||||
parameter, according to the specification.
|
||||
Removed the legacy V2 fallback support and the tests affected by that
|
||||
removal. Mocked existing steps to allow remaining V2 features to be
|
||||
covered/tested.
|
||||
|
||||
- **TUI Prompt Symbol Mode Awareness** (#6431): The prompt widget now displays a
|
||||
mode-dependent symbol (`❯` normal, `/` command, `$` shell, `☰` multi-line),
|
||||
implemented via `_PromptSymbolMixin` and `InputMode.MULTILINE`. The widget uses
|
||||
@@ -96,6 +118,8 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Changed
|
||||
|
||||
- Fixed stale `AUTO-BUG-POOL` tracking prefix references in automation-tracking.md documentation and agent-system-specification.md spec document, replaced with correct `AUTO-BUG-SUP` prefix used by the bug-hunt-pool-supervisor agent (#7875).
|
||||
|
||||
- **`agents session list` now displays full 26-character session ULIDs** (#10970): The Rich table
|
||||
and Summary panel ("Most Recent" / "Oldest") previously showed only the first 8 characters of
|
||||
each session ULID. This made the output unusable for copy-paste into `session tell`,
|
||||
@@ -120,6 +144,11 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Security
|
||||
|
||||
- **PyYAML upgraded to >=6.0.3 to address known security vulnerability** (#9055):
|
||||
Added an explicit `pyyaml>=6.0.3` dependency constraint to `pyproject.toml` to
|
||||
prevent installation of vulnerable older versions with known YAML parsing security
|
||||
issues.
|
||||
|
||||
- **aiohttp upgraded to >=3.13.4 to remediate CVE-2026-34513 and CVE-2026-34515** (#1549, #1544):
|
||||
Added an explicit `aiohttp>=3.13.4` dependency constraint to `pyproject.toml` to remediate
|
||||
two high-severity open redirect vulnerabilities. Both CVEs affect the CleverAgents platform's
|
||||
@@ -220,6 +249,25 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
failure paths. Comprehensive BDD test coverage validates the fix under concurrent
|
||||
execution and confirms proper cleanup behavior.
|
||||
|
||||
- **Database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy** (#8608):
|
||||
Implemented comprehensive database resource support enabling users to interact with
|
||||
PostgreSQL and SQLite backends through a unified resource interface. Introduces
|
||||
`DatabaseResourceHandler` providing full CRUD operations (`read`, `write`, `delete`,
|
||||
`list_children`), connection validation with automatic credential masking via
|
||||
:mod:`cleveragents.shared.redaction`, and transaction-based sandbox strategy using
|
||||
BEGIN/COMMIT/ROLLBACK wrappers for safe, isolated database operations. SQLite-specific
|
||||
checkpoint and rollback support with SAVEPOINT semantics. Support for multiple backends (PostgreSQL, SQLite, MySQL, DuckDB) via unified "DatabaseResourceHandler" and type-specific routing. BDD test
|
||||
coverage in ``features/database_resources.feature`` (connection validation, CRUD workflows,
|
||||
transaction/rollback behavior, error handling, credential masking verification) and
|
||||
Robot Framework integration tests in ``robot/database_resources.robot``.
|
||||
|
||||
- **TransactionSandbox infrastructure for database resource isolation** (#8608):
|
||||
Implemented ``TransactionSandbox`` class with BEGIN/COMMIT/ROLLBACK lifecycle
|
||||
management for transaction-based sandbox strategy. Wired into ``SandboxFactory``
|
||||
as the strategy resolver for database resource types. Added ``database`` resource type
|
||||
registration in bootstrap builtin types and updated ``_resource_registry_data.py``
|
||||
to recognize database resource categories.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **fix(repositories): derive PlanResult.success from result_success column instead of error_message** (#7501):
|
||||
@@ -420,6 +468,13 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
MCP logger thread-safety in `session.py` using a threading lock. Created
|
||||
migration guide for users transitioning from legacy to V3 workflow.
|
||||
|
||||
- **Plan Tree JSON/YAML Command Envelope** (#9163): `agents plan tree --format json/yaml`
|
||||
now wraps output in the spec-required command envelope with `command`, `status`,
|
||||
`exit_code`, `data`, `timing`, and `messages` fields. The `data` field contains
|
||||
`plan_id`, `tree`, `summary` (nodes, depth, child_plans, invariants, superseded),
|
||||
`child_plans` list, and `decision_ids` mapping. Timing now reflects actual elapsed
|
||||
milliseconds from command start to envelope construction.
|
||||
|
||||
- **Automation Profile Silent Fallback** (#8232): `_resolve_profile_for_plan` in
|
||||
`PlanLifecycleService` now raises a clear `ValidationError` when a plan's
|
||||
automation profile name is not a known built-in profile, instead of silently
|
||||
|
||||
+7
-4
@@ -17,12 +17,12 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed automated implementation, bug fixes, and feature development as part of the CleverAgents automation pool.
|
||||
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
|
||||
* HAL 9000 has contributed the plan concurrency race-condition fix (#7989): wired `LockService` into the plan lifecycle, guarding `execute_plan()` and `apply_plan()` with plan-level advisory locks and unique per-invocation owner identities to prevent silent concurrent state corruption.
|
||||
* HAL 9000 has contributed the bug-hunt-pool-supervisor non-blocking tracking fix: updated step 5 to be best-effort and added rule 9 to prevent the automation-tracking-manager call from blocking the main supervisor loop.
|
||||
<<<<<<< HEAD
|
||||
* HAL 9000 has contributed the bug-hunt-pool-supervisor non-blocking tracking fix (#7875 / PR #7957): updated step 5 to be best-effort and added rule 9 to prevent the automation-tracking-manager call from blocking the main supervisor loop.
|
||||
* Jeffrey Phillips Freeman has contributed the complete AUTO-BUG-POOL to AUTO-BUG-SUP tracking prefix fix across agent-system-specification.md, automation-tracking.md documentation and agent-system-specification.md spec document, replaced with correct `AUTO-BUG-SUP` prefix used by the bug-hunt-pool-supervisor agent (#7875).
|
||||
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
|
||||
* HAL 9000 has contributed the benchmark workflow separation (#9040): moved the benchmark-regression job out of the default PR workflow into a dedicated scheduled workflow, reducing median PR CI turnaround time from 99-132 minutes to under 30 minutes.
|
||||
* HAL 9000 has contributed the agent-evolution-pool-supervisor PR metadata assignment (#7888): the supervisor now automatically looks up the Type/Automation label and earliest open milestone before dispatching improvement PR creation workers, ensuring all generated improvement PRs have correct Type labels and milestone assignments.
|
||||
* HAL 9000 has contributed the decision recording hook for the Strategize phase (issue #8522): captures every decision point with question, chosen option, alternatives, confidence, rationale, and full context snapshot for replay and correction.
|
||||
* HAL 9000 has contributed automated specification maintenance, documentation updates, and bot-driven PR authorship.
|
||||
* HAL 9000 has contributed the plan tree JSON/YAML command envelope fix (#9163): wrapped `agents plan tree --format json/yaml` output in the spec-required command envelope structure, added summary statistics, decision_ids mapping, child_plans list, and accurate timing measurement.
|
||||
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
|
||||
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
|
||||
* HAL 9000 has contributed the file edit encoding parameter fix (PR #8258 / issue #7559).
|
||||
@@ -38,3 +38,6 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed the error-suppression removal fix (PR #9247 / issue #9060): removed both `try...except Exception:` blocks in `register_registry_agents()` that silently suppressed errors from `actor_registry.list_actors()` and the route bridge refresh, enabling exceptions to propagate per CONTRIBUTING.md fail-fast policy. Added three Behave scenarios verifying RuntimeError, AttributeError, and TypeError propagation.
|
||||
* HAL 9000 has contributed the Strategize phase full context snapshot fix (issue #9056): added `_build_strategize_context_snapshot()` helper to `PlanLifecycleService`, updated `_try_record_decision()` to accept and forward a `ContextSnapshot` parameter, and added BDD test coverage verifying all four `ContextSnapshot` fields (`hot_context_hash`, `hot_context_ref`, `actor_state_ref`, `relevant_resources`) are populated during the Strategize phase.
|
||||
* HAL 9000 has contributed the ACMS context path matching fix (PR #10975 / issue #10972): corrects `_path_matches()` and `_matches_pattern()` to properly match absolute fragment paths against relative glob patterns by auto-prefixing with `**/` before calling `PurePath.full_match()`, preventing silent inefficacy of include/exclude filters for absolute paths in fragment metadata.
|
||||
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
|
||||
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
|
||||
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
|
||||
|
||||
@@ -665,7 +665,7 @@ The Automation Tracking Manager provides eleven operations:
|
||||
| `AUTO-IMP-POOL` | Implementation Pool Supervisor |
|
||||
| `AUTO-REV-POOL` | PR Review Pool Supervisor |
|
||||
| `AUTO-UAT-POOL` | UAT Test Pool Supervisor |
|
||||
| `AUTO-BUG-POOL` | Bug Hunt Pool Supervisor |
|
||||
| `AUTO-BUG-SUP` | Bug Hunt Pool Supervisor |
|
||||
| `AUTO-INF-POOL` | Test Infrastructure Pool Supervisor |
|
||||
| `AUTO-ARCH` | Architecture Supervisor |
|
||||
| `AUTO-EPIC` | Epic Planning Supervisor |
|
||||
@@ -693,7 +693,7 @@ The following table maps between the two schemes for agents where they differ:
|
||||
| Implementation Pool | `AUTO-IMP-SUP` | `AUTO-IMP-POOL` |
|
||||
| PR Review Pool | `AUTO-REV-SUP` | `AUTO-REV-POOL` |
|
||||
| UAT Test Pool | `AUTO-UAT-SUP` | `AUTO-UAT-POOL` |
|
||||
| Bug Hunt Pool | `AUTO-BUG-SUP` | `AUTO-BUG-POOL` |
|
||||
| Bug Hunt Pool | `AUTO-BUG-SUP` | `AUTO-BUG-SUP` |
|
||||
| Test Infrastructure Pool | `AUTO-INF-SUP` | `AUTO-INF-POOL` |
|
||||
| Human Liaison | `AUTO-HUMAN` | `AUTO-LIAISON` |
|
||||
| Grooming | `AUTO-GROOM` | `AUTO-GROOMER` |
|
||||
@@ -1869,7 +1869,7 @@ Only critical bugs get assigned to the active milestone. Non-critical issues go
|
||||
| **Mode** | `subagent` |
|
||||
| **Model** | `google/gemini-2.5-pro` |
|
||||
| **Temperature** | 0.1 |
|
||||
| **Tracking Prefix** | `AUTO-BUG-POOL` |
|
||||
| **Tracking Prefix** | `AUTO-BUG-SUP` |
|
||||
| **Worker Count** | N/4 (quarter allocation) |
|
||||
|
||||
#### 8.6.1 Purpose
|
||||
@@ -3164,7 +3164,7 @@ Defines how ALL agents discover, interact with, and coordinate through the autom
|
||||
|
||||
### 13.9 Bug Hunter Tracking Update (`shared/bug_hunter_tracking_update.md`)
|
||||
|
||||
Defines the migration guide for converting the Bug Hunt supervisor from the old session state system to the new individual tracking issue system. Documents the tracking issue format `[AUTO-BUG-POOL] Bug Detection Pool Status (Cycle N)`, the cleanup protocol (one issue per cycle, preserve announcements), and worker coordination via announcement issues.
|
||||
Defines the migration guide for converting the Bug Hunt supervisor from the old session state system to the new individual tracking issue system. Documents the tracking issue format `[AUTO-BUG-SUP] Bug Hunt Status (Cycle 25)`, the cleanup protocol (one issue per cycle, preserve announcements), and worker coordination via announcement issues.
|
||||
|
||||
---
|
||||
|
||||
@@ -6767,7 +6767,7 @@ The system uses five distinct redundancy patterns:
|
||||
| 4 | pr-merge-pool-supervisor | subagent | claude-sonnet-4-6 | AUTO-MERGE | Singleton Supervisor |
|
||||
| 5 | pr-fix-pool-supervisor | *(removed — absorbed into implementation-pool-supervisor)* | -- | -- | -- |
|
||||
| 6 | uat-test-pool-supervisor | subagent | claude-sonnet-4-6 | AUTO-UAT-POOL | Pool Supervisor |
|
||||
| 7 | bug-hunt-pool-supervisor | subagent | gemini-2.5-pro | AUTO-BUG-POOL | Pool Supervisor |
|
||||
| 7 | bug-hunt-pool-supervisor | subagent | gemini-2.5-pro | AUTO-BUG-SUP | Pool Supervisor |
|
||||
| 8 | test-infra-pool-supervisor | subagent | gemini-2.5-pro | AUTO-INF-POOL | Pool Supervisor |
|
||||
| 9 | architecture-pool-supervisor | subagent | claude-sonnet-4-6 | AUTO-ARCH | Singleton Supervisor |
|
||||
| 10 | epic-planning-pool-supervisor | subagent | claude-sonnet-4-6 | AUTO-EPIC | Singleton Supervisor |
|
||||
|
||||
@@ -61,7 +61,7 @@ All prefixes are registered in the `automation-tracking-manager` subagent, which
|
||||
| uat-test-pool-supervisor | `AUTO-UAT-POOL` | `[AUTO-UAT-POOL] UAT Status (Cycle 6)` |
|
||||
| project-owner-pool-supervisor | `AUTO-PROJ-OWN` | `[AUTO-PROJ-OWN] Project Status (Cycle 11)` |
|
||||
| agent-evolution-pool-supervisor | `AUTO-EVLV` | `[AUTO-EVLV] Agent Evolution Report (Cycle 10)` |
|
||||
| bug-hunt-pool-supervisor | `AUTO-BUG-POOL` | `[AUTO-BUG-POOL] Bug Detection Pool Status (Cycle 60)` |
|
||||
| bug-hunt-pool-supervisor | `AUTO-BUG-SUP` | `[AUTO-BUG-SUP] Bug Hunt Status (Cycle 25)` |
|
||||
| spec-update-pool-supervisor | `AUTO-SPEC` | `[AUTO-SPEC] Specification Update Report (Cycle 1)` |
|
||||
| test-infra-pool-supervisor | `AUTO-INF-POOL` | `[AUTO-INF-POOL] Infrastructure Analysis Report (Cycle 2)` |
|
||||
| epic-planner | `AUTO-EPIC` | `[AUTO-EPIC] Epic Planning Update (Cycle 9)` |
|
||||
@@ -394,7 +394,7 @@ label:"Automation Tracking" [AUTO-UAT-POOL] in:title
|
||||
label:"Automation Tracking" [AUTO-PROJ-OWN] in:title
|
||||
label:"Automation Tracking" [AUTO-EVLV] in:title
|
||||
label:"Automation Tracking" [AUTO-EPIC] in:title
|
||||
label:"Automation Tracking" [AUTO-BUG-POOL] in:title
|
||||
label:"Automation Tracking" [AUTO-BUG-SUP] in:title
|
||||
label:"Automation Tracking" [AUTO-SPEC] in:title
|
||||
label:"Automation Tracking" [AUTO-INF-POOL] in:title
|
||||
```
|
||||
|
||||
@@ -279,140 +279,10 @@ Feature: Actor configuration uncovered lines coverage
|
||||
When I build an actor configuration from blob {"provider": "only-provider"}
|
||||
Then a ValueError should be raised containing "model is required"
|
||||
|
||||
Scenario: from_blob merges default and v2 and override options
|
||||
When I build an actor configuration from structured blob with defaults and overrides:
|
||||
"""
|
||||
{
|
||||
"blob": {
|
||||
"provider": "cli-provider",
|
||||
"model": "cli-model",
|
||||
"options": {"user": "blob"},
|
||||
"agents": {
|
||||
"writer": {
|
||||
"config": {
|
||||
"options": {"v2": "v2-option"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"default_options": {"base": "default"},
|
||||
"option_overrides": {"user": "override", "extra": "added"}
|
||||
}
|
||||
"""
|
||||
Then the actor configuration should have provider "cli-provider" and model "cli-model"
|
||||
And the actor configuration options should equal {"base": "default", "v2": "v2-option", "user": "override", "extra": "added"}
|
||||
|
||||
Scenario: from_blob with None blob defaults to empty data
|
||||
When I build actor config from None blob with provider and model overrides
|
||||
Then the actor configuration should have provider "override-provider" and model "override-model"
|
||||
|
||||
# --- _extract_v2_actor: lines 275-307 ---
|
||||
|
||||
Scenario: v2 YAML actor config infers provider and model
|
||||
Given an actor config file "v2.yaml" with content:
|
||||
"""
|
||||
cleveragents:
|
||||
default_router: main_router
|
||||
agents:
|
||||
paper_writer:
|
||||
type: llm
|
||||
config:
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
unsafe: true
|
||||
options:
|
||||
temperature: 0.5
|
||||
routes:
|
||||
main_router:
|
||||
type: stream
|
||||
operators:
|
||||
- type: map
|
||||
params:
|
||||
agent: paper_writer
|
||||
publications:
|
||||
- __output__
|
||||
"""
|
||||
When I parse the actor configuration from file "v2.yaml" with overrides:
|
||||
"""
|
||||
{}
|
||||
"""
|
||||
Then the actor configuration should have provider "openai" and model "gpt-4"
|
||||
And the actor configuration unsafe flag should be true
|
||||
And the actor configuration graph descriptor should include key "routes"
|
||||
And the actor configuration graph descriptor should include key "agents"
|
||||
And the actor configuration graph descriptor should include key "cleveragents"
|
||||
And the actor configuration options should equal {"temperature": 0.5}
|
||||
|
||||
Scenario: v2 extraction includes merges and templates keys when present
|
||||
Given an actor config file "v2_merges.yaml" with content:
|
||||
"""
|
||||
agents:
|
||||
writer:
|
||||
config:
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
merges:
|
||||
combined:
|
||||
type: join
|
||||
templates:
|
||||
default:
|
||||
system_prompt: "hello"
|
||||
"""
|
||||
When I parse the actor configuration from file "v2_merges.yaml" with overrides:
|
||||
"""
|
||||
{}
|
||||
"""
|
||||
Then the actor configuration should have provider "openai" and model "gpt-4"
|
||||
And the actor configuration graph descriptor should include key "merges"
|
||||
And the actor configuration graph descriptor should include key "templates"
|
||||
|
||||
Scenario: v2 extraction handles agent entry without config block
|
||||
Given an actor config file "v2_no_config.yaml" with content:
|
||||
"""
|
||||
provider: fallback-provider
|
||||
model: fallback-model
|
||||
agents:
|
||||
first:
|
||||
type: llm
|
||||
"""
|
||||
When I parse the actor configuration from file "v2_no_config.yaml" with overrides:
|
||||
"""
|
||||
{}
|
||||
"""
|
||||
Then the actor configuration should have provider "fallback-provider" and model "fallback-model"
|
||||
|
||||
# --- _extract_v2_options: lines 316-326 ---
|
||||
|
||||
Scenario: v2 extraction skips non-dict agent entries
|
||||
Given an actor config file "invalid_v2.yaml" with content:
|
||||
"""
|
||||
provider: fallback-provider
|
||||
model: fallback-model
|
||||
agents:
|
||||
first: not-a-dict
|
||||
"""
|
||||
When I parse the actor configuration from file "invalid_v2.yaml" with overrides:
|
||||
"""
|
||||
{}
|
||||
"""
|
||||
Then the actor configuration should have provider "fallback-provider" and model "fallback-model"
|
||||
|
||||
Scenario: v2 options extraction returns None for agents without options
|
||||
Given an actor config file "v2_no_opts.yaml" with content:
|
||||
"""
|
||||
agents:
|
||||
writer:
|
||||
config:
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
When I parse the actor configuration from file "v2_no_opts.yaml" with overrides:
|
||||
"""
|
||||
{}
|
||||
"""
|
||||
Then the actor configuration should have provider "openai" and model "gpt-4"
|
||||
And the actor configuration options should equal {}
|
||||
|
||||
Scenario: from_blob reads provider_type and model_id aliases
|
||||
When I build an actor configuration from blob {"provider_type": "aliased-provider", "model_id": "aliased-model"}
|
||||
Then the actor configuration should have provider "aliased-provider" and model "aliased-model"
|
||||
|
||||
@@ -15,6 +15,7 @@ Feature: ActorRegistry.add() accepts spec-compliant actor YAML formats
|
||||
And the registered actor name should be "local/my-assistant"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
@tdd_issue @tdd_issue_4300
|
||||
Scenario: registry.add() accepts spec-compliant actors: map with separate provider and model
|
||||
When I add a spec-compliant YAML with actors map and separate provider model
|
||||
Then the actor should be registered with provider "anthropic" and model "claude-3"
|
||||
@@ -53,24 +54,13 @@ Feature: ActorRegistry.add() accepts spec-compliant actor YAML formats
|
||||
Then the actor should be registered with provider "nested-provider" and model "top-level-model"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
# ── Graph descriptor preserved from nested config ──────────────────
|
||||
|
||||
Scenario: registry.add() preserves graph descriptor from nested spec-compliant config
|
||||
When I add a spec-compliant YAML with actors map and combined actor field
|
||||
Then the registered actor graph descriptor should contain key "actors"
|
||||
|
||||
# ── Top-level provider+model still picks up nested unsafe/graph ──────
|
||||
# ── Top-level provider+model still picks up nested unsafe ───────────
|
||||
|
||||
Scenario: registry.add() with top-level provider and model detects nested unsafe flag
|
||||
When I add a YAML with top-level provider and model and nested actors map with unsafe flag
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should be marked unsafe
|
||||
|
||||
Scenario: registry.add() with top-level provider and model detects nested graph descriptor
|
||||
When I add a YAML with top-level provider and model and nested actors map with graph descriptor
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor graph descriptor should contain key "actors"
|
||||
|
||||
# ── Unsafe confirmation gate ───────────────────────────────────────
|
||||
|
||||
Scenario: registry.add() rejects unsafe actor without confirmation
|
||||
@@ -94,12 +84,6 @@ Feature: ActorRegistry.add() accepts spec-compliant actor YAML formats
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should not be marked unsafe
|
||||
|
||||
# ── Missing provider/model still rejected for non-v3 YAML ─────────
|
||||
|
||||
Scenario: registry.add() rejects non-v3 YAML without any provider or model
|
||||
When I attempt to add a YAML with no provider or model anywhere
|
||||
Then a spec-yaml ValidationError should be raised containing "provider"
|
||||
|
||||
# ── update=True path ───────────────────────────────────────────────
|
||||
|
||||
Scenario: registry.add() with update=True overwrites an existing actor using spec-compliant YAML
|
||||
@@ -139,366 +123,6 @@ Feature: ActorRegistry.add() accepts spec-compliant actor YAML formats
|
||||
And the registered actor should be marked unsafe
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
# ── Multi-actor YAML rejection ───────────────────────────────────
|
||||
|
||||
Scenario: registry.add() rejects multi-actor YAML with a ValidationError
|
||||
When I attempt to add a multi-actor YAML with two actor entries
|
||||
Then a spec-yaml ValidationError should be raised containing "single-actor"
|
||||
|
||||
Scenario: registry.add() rejects multi-actor YAML via agents: fallback when actors: is null
|
||||
When I attempt to add a YAML with actors null and multi-entry agents map
|
||||
Then a spec-yaml ValidationError should be raised containing "single-actor"
|
||||
|
||||
# ── _extract_v2_actor handles actors: key ──────────────────────────
|
||||
|
||||
Scenario: _extract_v2_actor extracts provider/model from actors: map
|
||||
When I call _extract_v2_actor with an actors map containing combined actor field
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: _extract_v2_actor extracts from actors: map with separate fields
|
||||
When I call _extract_v2_actor with an actors map containing separate provider model
|
||||
Then the spec-yaml extracted provider should be "anthropic"
|
||||
And the spec-yaml extracted model should be "claude-3"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: _extract_v2_actor prefers actors: key over agents: key
|
||||
When I call _extract_v2_actor with both actors and agents maps
|
||||
Then the spec-yaml extracted provider should be "actors-provider"
|
||||
And the spec-yaml extracted model should be "actors-model"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: _extract_v2_actor graph descriptor contains agent key matching the actor entry name
|
||||
When I call _extract_v2_actor with an actors map containing combined actor field
|
||||
Then the spec-yaml extracted graph descriptor should contain key "agent"
|
||||
And the spec-yaml extracted graph descriptor agent value should be "my_assistant"
|
||||
|
||||
Scenario: _extract_v2_actor with actors map containing unsafe flag
|
||||
When I call _extract_v2_actor with an actors map containing unsafe true
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted unsafe flag should be True
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: _extract_v2_actor returns None for empty data
|
||||
When I call _extract_v2_actor with an empty dict
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
|
||||
Scenario: _extract_v2_actor returns None for actors key with empty map
|
||||
When I call _extract_v2_actor with actors key containing empty map
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
|
||||
Scenario: _extract_v2_actor returns None for actors key with None value
|
||||
When I call _extract_v2_actor with actors key containing None value
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
|
||||
# ── _extract_v2_actor handles agents: key ─────────────────────────
|
||||
|
||||
Scenario: _extract_v2_actor returns graph descriptor with agents map_key
|
||||
When I call _extract_v2_actor with an agents map containing combined actor field
|
||||
Then the spec-yaml extracted graph descriptor should contain key "agents"
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
# ── _extract_v2_actor edge cases: non-dict / missing config ────────
|
||||
|
||||
Scenario: _extract_v2_actor returns None for non-dict first entry
|
||||
When I call _extract_v2_actor with a non-dict first entry in actors map
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
Scenario: _extract_v2_actor returns None for dict entry missing config block
|
||||
When I call _extract_v2_actor with a dict entry missing config block
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
# ── _extract_v2_actor edge case: actors: {} blocks agents: fallback ─
|
||||
|
||||
Scenario: _extract_v2_actor with empty actors dict blocks agents fallback
|
||||
When I call _extract_v2_actor with empty actors dict and valid agents map
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
# ── _extract_v2_actor edge case: actors: [] (list type) ────────────
|
||||
|
||||
Scenario: _extract_v2_actor with actors as list returns None
|
||||
When I call _extract_v2_actor with actors as a list
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should be None
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
# ── _extract_v2_options handles actors: and agents: keys ───────────
|
||||
|
||||
Scenario: _extract_v2_options extracts options from actors: map
|
||||
When I call _extract_v2_options with an actors map containing options
|
||||
Then the spec-yaml extracted options should contain key "temperature" with value 0.7
|
||||
|
||||
Scenario: _extract_v2_options extracts options from agents: map
|
||||
When I call _extract_v2_options with an agents map containing options
|
||||
Then the spec-yaml extracted options should contain key "max_tokens" with value 1024
|
||||
|
||||
Scenario: _extract_v2_options returns None for actors key with empty map
|
||||
When I call _extract_v2_options with actors key containing empty map
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options returns None for actors key with None value
|
||||
When I call _extract_v2_options with actors key containing None value
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options returns None for actors key with list value
|
||||
When I call _extract_v2_options with actors key containing list value
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options returns None when config block has no options key
|
||||
When I call _extract_v2_options with an actors map where config has no options key
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options returns None for non-dict first entry in actors map
|
||||
When I call _extract_v2_options with a non-dict first entry in actors map
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options returns None for dict entry missing config block
|
||||
When I call _extract_v2_options with a dict entry missing config block
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
Scenario: _extract_v2_options prefers actors: key over agents: key
|
||||
When I call _extract_v2_options with both actors and agents maps containing options
|
||||
Then the spec-yaml extracted options should contain key "source" with value "actors"
|
||||
|
||||
# ── Unsafe coercion edge cases (unsafe coercion) ────────────────
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: "no" as False (not truthy string)
|
||||
When I call _extract_v2_actor with unsafe value "no"
|
||||
Then the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: "yes" as False (not truthy string)
|
||||
When I call _extract_v2_actor with unsafe value "yes"
|
||||
Then the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: 1 (integer) as True
|
||||
When I call _extract_v2_actor with unsafe value 1
|
||||
Then the spec-yaml extracted unsafe flag should be True
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
Scenario: registry.add() accepts actors map with unsafe: 1 (integer) and unsafe flag
|
||||
When I add a YAML with actors map where unsafe is integer 1 and the unsafe flag set
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should be marked unsafe
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: 1.0 (float) as True
|
||||
When I call _extract_v2_actor with unsafe value 1.0
|
||||
Then the spec-yaml extracted unsafe flag should be True
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: 2 (integer > 1) as False
|
||||
When I call _extract_v2_actor with unsafe value 2
|
||||
Then the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
Scenario: _extract_v2_actor treats unsafe: 0 (integer zero) as False
|
||||
When I call _extract_v2_actor with unsafe value 0
|
||||
Then the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
|
||||
# ── Top-level unsafe: 1 (integer) through registry.add() ────────────
|
||||
|
||||
Scenario: registry.add() rejects top-level unsafe: 1 (integer) without confirmation
|
||||
When I attempt to add a YAML with top-level unsafe integer 1 and no flag
|
||||
Then a spec-yaml ValidationError should be raised containing "unsafe"
|
||||
|
||||
Scenario: registry.add() accepts top-level unsafe: 1 (integer) with unsafe flag
|
||||
When I add a YAML with top-level unsafe integer 1 and the unsafe flag set
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should be marked unsafe
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
# ── Top-level graph_descriptor key through registry.add() (T7) ──────
|
||||
|
||||
Scenario: registry.add() resolves graph descriptor from top-level graph_descriptor key
|
||||
When I add a YAML with top-level provider model and graph_descriptor key
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor graph descriptor should contain key "workflow"
|
||||
|
||||
Scenario: _extract_v2_actor includes top-level routes key in graph descriptor
|
||||
When I call _extract_v2_actor with an actors map and a top-level routes key
|
||||
Then the spec-yaml extracted graph descriptor should contain key "routes"
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
# ── Legacy graph key fallback (M3) ─────────────────────────────────
|
||||
|
||||
Scenario: registry.add() resolves graph descriptor from legacy top-level graph key
|
||||
When I add a YAML with top-level provider model and legacy graph key
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor graph descriptor should contain key "workflow"
|
||||
|
||||
# ── Empty actors map through registry.add() (M4) ───────────────────
|
||||
|
||||
Scenario: registry.add() with empty actors map does not fall back to agents map for provider/model
|
||||
When I attempt to add a YAML with empty actors map and valid agents map but no top-level provider
|
||||
Then a spec-yaml ValidationError should be raised containing "provider"
|
||||
|
||||
# ── provider_type / model_id aliases in nested config (m1) ─────────
|
||||
|
||||
Scenario: _extract_v2_actor extracts provider from provider_type alias in nested config
|
||||
When I call _extract_v2_actor with an actors map using provider_type alias
|
||||
Then the spec-yaml extracted provider should be "alias-provider"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: _extract_v2_actor extracts model from model_id alias in nested config
|
||||
When I call _extract_v2_actor with an actors map using model_id alias
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "alias-model"
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
# ── _extract_v2_options with empty dict (m4) ────────────────────────
|
||||
|
||||
Scenario: _extract_v2_options returns None for empty dict input
|
||||
When I call _extract_v2_options with an empty dict
|
||||
Then the spec-yaml extracted options should be None
|
||||
|
||||
# ── compiled_metadata value assertion (m5) ──────────────────────────
|
||||
|
||||
Scenario: registry.add() forwards compiled_metadata with correct values
|
||||
When I add a spec-compliant YAML with schema_version "2.0" and compiled_metadata
|
||||
Then the registered actor compiled metadata key "key" should have value "val"
|
||||
|
||||
# ── Combined actor field edge cases ────────────────────────────────
|
||||
|
||||
Scenario: Combined actor field without slash is ignored
|
||||
When I call _extract_v2_actor with an actors map where actor field has no slash
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
|
||||
Scenario: Combined actor field does not override explicit provider but fills missing model
|
||||
When I call _extract_v2_actor with an actors map where both actor and provider exist
|
||||
Then the spec-yaml extracted provider should be "explicit-provider"
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: Combined actor field does not override explicit model but fills missing provider
|
||||
When I call _extract_v2_actor with an actors map where both actor and model exist
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "explicit-model"
|
||||
And the spec-yaml extracted unsafe flag should be False
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
# ── Combined actor field malformed input edge cases ────────────────
|
||||
|
||||
Scenario: Combined actor field with empty provider part yields no provider
|
||||
When I call _extract_v2_actor with an actors map where actor field has empty provider
|
||||
Then the spec-yaml extracted provider should be None
|
||||
And the spec-yaml extracted model should be "gpt-4"
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
Scenario: Combined actor field with empty model part yields no model
|
||||
When I call _extract_v2_actor with an actors map where actor field has empty model
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be None
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
# ── Combined actor field with multiple slashes (L1) ────────────────
|
||||
|
||||
Scenario: Combined actor field with multiple slashes splits on first slash only
|
||||
When I call _extract_v2_actor with an actors map where actor field has multiple slashes
|
||||
Then the spec-yaml extracted provider should be "openai"
|
||||
And the spec-yaml extracted model should be "gpt-4/extra"
|
||||
And the spec-yaml extracted graph descriptor should contain key "actors"
|
||||
|
||||
# ── actors: null + valid agents: through registry.add() (L2) ───────
|
||||
|
||||
Scenario: registry.add() with actors: null falls back to valid agents: map
|
||||
When I add a YAML with actors null and valid agents map
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
# ── Top-level provider_type / model_id aliases through registry.add() (T8) ──
|
||||
|
||||
Scenario: registry.add() accepts top-level provider_type alias
|
||||
When I add a YAML with top-level provider_type alias and model
|
||||
Then the actor should be registered with provider "alias-provider" and model "gpt-4"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
Scenario: registry.add() accepts top-level model_id alias
|
||||
When I add a YAML with top-level provider and model_id alias
|
||||
Then the actor should be registered with provider "openai" and model "alias-model"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
# ── Top-level unsafe string coercion through registry.add() (T9) ───
|
||||
|
||||
Scenario: registry.add() treats top-level unsafe: "yes" as not unsafe (no gate rejection)
|
||||
When I add a YAML with top-level unsafe string "yes" and provider model
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should not be marked unsafe
|
||||
|
||||
Scenario: registry.add() treats top-level unsafe: "no" as not unsafe (no gate rejection)
|
||||
When I add a YAML with top-level unsafe string "no" and provider model
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor should not be marked unsafe
|
||||
|
||||
# ── Nested options extraction through registry.add() (M1) ──────────
|
||||
|
||||
Scenario: registry.add() extracts and preserves nested config options
|
||||
When I add a spec-compliant YAML with actors map and nested options
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor config blob should contain options key "temperature" with value 0.9
|
||||
And the registered actor config blob should contain options key "max_tokens" with value 2000
|
||||
|
||||
Scenario: registry.add() merges nested and top-level options (nested base, top-level overrides)
|
||||
When I add a YAML with both top-level and nested options
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor config blob should contain options key "temperature" with value 0.7
|
||||
And the registered actor config blob should contain options key "max_tokens" with value 2000
|
||||
And the registered actor config blob should contain options key "top_p" with value 0.95
|
||||
|
||||
Scenario: registry.add() with non-dict top-level options uses nested options
|
||||
When I add a YAML with non-dict top-level options and nested options
|
||||
Then the actor should be registered with provider "openai" and model "gpt-4"
|
||||
And the registered actor config blob should contain options key "temperature" with value 0.9
|
||||
|
||||
Scenario: registry.add() sets source: "yaml" default in config_blob
|
||||
When I add a spec-compliant YAML with actors map and combined actor field
|
||||
Then the registered actor config blob should contain source "yaml"
|
||||
|
||||
# ── _extract_v2_options shallow copy mutation isolation (NIT-2) ─────
|
||||
|
||||
Scenario: _extract_v2_options returns a shallow copy that does not mutate the original blob
|
||||
When I call _extract_v2_options and mutate the returned dict
|
||||
Then the original blob options should be unmodified
|
||||
|
||||
# ── M4: update=True creates actor when it doesn't exist ────────────
|
||||
|
||||
Scenario: registry.add() with update=True creates actor when it doesn't exist
|
||||
@@ -518,25 +142,5 @@ Feature: ActorRegistry.add() accepts spec-compliant actor YAML formats
|
||||
When upsert_actor is configured to raise RuntimeError and I add a valid YAML
|
||||
Then a RuntimeError should have been propagated from add
|
||||
|
||||
# ── M7: actors: false (boolean) blocks agents fallback ─────────────
|
||||
|
||||
|
||||
Scenario: registry.add() with actors: false blocks agents fallback and raises provider error
|
||||
When I attempt to add a YAML with actors false and valid agents map
|
||||
Then a spec-yaml ValidationError should be raised containing "provider"
|
||||
|
||||
# ── M8: non-dict compiled_metadata causes Pydantic error ───────────
|
||||
|
||||
Scenario: registry.add() with non-dict compiled_metadata raises a Pydantic validation error
|
||||
When I attempt to add a valid YAML with compiled_metadata as a non-dict string
|
||||
Then a Pydantic validation error should be raised for compiled_metadata
|
||||
|
||||
# ── M9: provider: 0 (integer zero) falls through to provider_type ──
|
||||
|
||||
Scenario: registry.add() with provider: 0 falls through to provider_type fallback
|
||||
When I attempt to add a YAML with provider integer 0 and no provider_type
|
||||
Then a spec-yaml ValidationError should be raised containing "provider"
|
||||
|
||||
Scenario: registry.add() with provider: 0 and valid provider_type uses provider_type
|
||||
When I add a YAML with provider integer 0 and a valid provider_type
|
||||
Then the actor should be registered with provider "fallback-provider" and model "gpt-4"
|
||||
And the registered actor should exist in the actor service
|
||||
|
||||
@@ -26,12 +26,6 @@ Feature: v3 Actor YAML schema support in CLI and registry
|
||||
When I call ActorConfiguration.from_blob with the v3 blob
|
||||
Then the configuration should have a graph_descriptor with type "graph"
|
||||
|
||||
Scenario: ActorConfiguration.from_blob falls through to v2 for legacy YAML
|
||||
Given a v2 actor blob with agents and routes
|
||||
When I call ActorConfiguration.from_blob with the v2 blob
|
||||
Then the configuration should have provider "openai"
|
||||
And the configuration should have model "gpt-4"
|
||||
|
||||
Scenario: ActorRegistry.add validates v3 LLM actor YAML
|
||||
Given a mock actor service for v3 testing
|
||||
And a v3 LLM actor YAML text
|
||||
@@ -67,11 +61,6 @@ Feature: v3 Actor YAML schema support in CLI and registry
|
||||
And the reactive config should have a graph route
|
||||
And the graph route edges should use source and target keys
|
||||
|
||||
Scenario: v3 format detection returns false for v2 data
|
||||
Given a v2 actor config dict with agents key
|
||||
When I check if the config is v3 format
|
||||
Then it should not be detected as v3
|
||||
|
||||
# M14: test update=True path
|
||||
Scenario: ActorRegistry.add with update=True overwrites existing actor
|
||||
Given a mock actor service for v3 testing
|
||||
@@ -174,27 +163,6 @@ Feature: v3 Actor YAML schema support in CLI and registry
|
||||
When I call ActorRegistry.add with a non-mapping YAML string
|
||||
Then a ValidationError should be raised mentioning mapping
|
||||
|
||||
# Coverage: registry.py — _add_legacy v3 schema validation failure
|
||||
Scenario: ActorRegistry._add_legacy rejects invalid v3 YAML via schema validation
|
||||
Given a mock actor service for v3 testing
|
||||
And a legacy v3 YAML text with invalid schema
|
||||
When I call ActorRegistry.add with the legacy invalid v3 YAML
|
||||
Then a ValidationError should be raised mentioning invalid v3 actor
|
||||
|
||||
# Coverage: registry.py — _add_legacy missing provider/model in non-v3 blob
|
||||
Scenario: ActorRegistry._add_legacy rejects non-v3 blob missing provider and model
|
||||
Given a mock actor service for v3 testing
|
||||
And a legacy non-v3 YAML text missing provider and model
|
||||
When I call ActorRegistry.add with the legacy non-v3 YAML
|
||||
Then a ValidationError should be raised mentioning provider and model
|
||||
|
||||
# Coverage: registry.py — _add_legacy unsafe flag rejection
|
||||
Scenario: ActorRegistry._add_legacy rejects unsafe legacy actor without allow_unsafe
|
||||
Given a mock actor service for v3 testing
|
||||
And a legacy actor YAML text marked unsafe
|
||||
When I call ActorRegistry.add with the legacy unsafe YAML
|
||||
Then a ValidationError should be raised mentioning unsafe
|
||||
|
||||
# Coverage: registry.py — upsert_actor v3 validation failure
|
||||
Scenario: ActorRegistry.upsert_actor rejects invalid v3 config_blob
|
||||
Given a mock actor service for v3 testing
|
||||
|
||||
@@ -459,22 +459,6 @@ Feature: Consolidated Actor
|
||||
Then the config options should include the overrides
|
||||
|
||||
|
||||
Scenario: _extract_v2_actor parses v2 agents block
|
||||
When I extract v2 actor from a v2-style config with agents block
|
||||
Then the extracted provider and model should be set
|
||||
And the graph descriptor should contain agent info
|
||||
|
||||
|
||||
Scenario: _extract_v2_actor returns None for missing agents
|
||||
When I extract v2 actor from a config without agents block
|
||||
Then all extracted values should be None
|
||||
|
||||
|
||||
Scenario: _extract_v2_options extracts options from v2 config
|
||||
When I extract v2 options from a v2-style config
|
||||
Then the extracted options should contain expected keys
|
||||
|
||||
|
||||
Scenario: from_file loads and parses config from disk
|
||||
Given a temporary JSON config file with provider and model
|
||||
When I create an ActorConfiguration from the file
|
||||
@@ -876,6 +860,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/test-actor
|
||||
type: llm
|
||||
description: A test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
@@ -888,6 +874,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text with schema_version "2.0":
|
||||
"""
|
||||
name: local/versioned
|
||||
type: llm
|
||||
description: A versioned actor
|
||||
provider: anthropic
|
||||
model: claude-3
|
||||
"""
|
||||
@@ -899,6 +887,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text with compiled_metadata:
|
||||
"""
|
||||
name: local/compiled
|
||||
type: llm
|
||||
description: A compiled actor
|
||||
provider: openai
|
||||
model: gpt-4o
|
||||
"""
|
||||
@@ -910,12 +900,16 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/updatable
|
||||
type: llm
|
||||
description: An updatable actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
And I update actor from YAML text:
|
||||
"""
|
||||
name: local/updatable
|
||||
type: llm
|
||||
description: An updatable actor
|
||||
provider: openai
|
||||
model: gpt-4-turbo
|
||||
"""
|
||||
@@ -927,6 +921,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/removable
|
||||
type: llm
|
||||
description: A removable actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
@@ -939,12 +935,16 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/alpha
|
||||
type: llm
|
||||
description: An alpha actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
And I add actor from YAML text with update:
|
||||
"""
|
||||
name: local/beta
|
||||
type: llm
|
||||
description: A beta actor
|
||||
provider: anthropic
|
||||
model: claude-3
|
||||
"""
|
||||
@@ -957,6 +957,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/auto-prefixed
|
||||
type: llm
|
||||
description: An auto-prefixed actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
@@ -968,6 +970,8 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/fetchable
|
||||
type: llm
|
||||
description: A fetchable actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
@@ -978,6 +982,8 @@ Feature: Consolidated Actor
|
||||
Given an actor registry with no configured providers for persistence
|
||||
When I attempt to add actor from YAML text without name:
|
||||
"""
|
||||
type: llm
|
||||
description: No name actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
@@ -989,12 +995,16 @@ Feature: Consolidated Actor
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/duplicate
|
||||
type: llm
|
||||
description: A duplicate actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
And I attempt to add duplicate actor from YAML text:
|
||||
"""
|
||||
name: local/duplicate
|
||||
type: llm
|
||||
description: A duplicate actor
|
||||
provider: openai
|
||||
model: gpt-4-turbo
|
||||
"""
|
||||
@@ -1008,17 +1018,7 @@ Feature: Consolidated Actor
|
||||
And the actor "local/legacy-yaml" should have schema_version "1.5"
|
||||
|
||||
|
||||
Scenario: Default schema version is applied when not specified
|
||||
Given an actor registry with no configured providers for persistence
|
||||
When I add actor from YAML text:
|
||||
"""
|
||||
name: local/default-version
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
Then the actor "local/default-version" should have schema_version "1.0"
|
||||
|
||||
|
||||
|
||||
# ============================================================
|
||||
# Originally from: actor_runtime.feature
|
||||
# Feature: Tool-Calling Actor Runtime
|
||||
|
||||
@@ -140,7 +140,7 @@ Feature: Consolidated Misc
|
||||
And the operations list should contain "registry.list_tools"
|
||||
And the operations list should contain "context.get"
|
||||
And the operations list should contain "event.subscribe"
|
||||
And the operations list should have 42 items
|
||||
And the operations list should have 44 items
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# A2aHttpTransport — all stubs raise A2aNotAvailableError
|
||||
@@ -639,7 +639,7 @@ Feature: Consolidated Misc
|
||||
Then the m6 smoke operations should include "session.create"
|
||||
And the m6 smoke operations should include "plan.execute"
|
||||
And the m6 smoke operations should include "event.subscribe"
|
||||
And the m6 smoke operations count should be 42
|
||||
And the m6 smoke operations count should be 44
|
||||
|
||||
# --- A2A event queue ---
|
||||
|
||||
|
||||
@@ -109,7 +109,7 @@ Feature: M6 autonomy acceptance smoke tests
|
||||
Then the m6 smoke operations should include "session.create"
|
||||
And the m6 smoke operations should include "plan.execute"
|
||||
And the m6 smoke operations should include "event.subscribe"
|
||||
And the m6 smoke operations count should be 42
|
||||
And the m6 smoke operations count should be 44
|
||||
|
||||
# --- A2A event queue (AC-2: event queue publish/subscribe) ---
|
||||
|
||||
|
||||
@@ -107,12 +107,12 @@ Feature: Plan explain and decision tree CLI commands
|
||||
# plan tree - json format
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@tdd_issue @tdd_issue_4254 @tdd_expected_fail
|
||||
@tdd_issue @tdd_issue_4254
|
||||
Scenario: Tree with json format
|
||||
Given a set of test decisions forming a tree
|
||||
When I format the tree as json
|
||||
Then the json tree output should be valid json
|
||||
And the json tree output should contain "decision_id"
|
||||
Then the json tree output should be a valid json envelope
|
||||
And the json tree output should contain "command"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# plan tree - yaml format
|
||||
|
||||
@@ -65,13 +65,13 @@ Feature: Plan explain and tree CLI command coverage
|
||||
Given pec a mock DecisionService returning a list of decisions
|
||||
When pec I invoke "tree" with format "json"
|
||||
Then pec the exit code should be 0
|
||||
And pec the output should be valid json list
|
||||
And pec the output should be valid json envelope
|
||||
|
||||
Scenario: Tree CLI renders yaml format
|
||||
Given pec a mock DecisionService returning a list of decisions
|
||||
When pec I invoke "tree" with format "yaml"
|
||||
Then pec the exit code should be 0
|
||||
And pec the output should contain "decision_id:"
|
||||
And pec the output should contain "command:"
|
||||
|
||||
Scenario: Tree CLI renders table format
|
||||
Given pec a mock DecisionService returning a list of decisions
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
@tdd_issue @tdd_issue_9055
|
||||
Feature: PyYAML is a declared project dependency with minimum secure version
|
||||
As a CleverAgents developer
|
||||
I want the PyYAML dependency to be listed in pyproject.toml with a minimum secure version
|
||||
So that vulnerable older versions of PyYAML cannot be installed
|
||||
|
||||
Background:
|
||||
Given the pyproject.toml file exists at "pyproject.toml"
|
||||
|
||||
@tdd_issue @tdd_issue_9055
|
||||
Scenario: PyYAML is listed in project dependencies with minimum version constraint
|
||||
When I read the project dependencies from pyproject.toml
|
||||
Then the dependency list should include a package matching "pyyaml>="
|
||||
|
||||
@tdd_issue @tdd_issue_9055
|
||||
Scenario: PyYAML minimum version is >=6.0.3
|
||||
When I read the PyYAML dependency specification from pyproject.toml
|
||||
Then the minimum version should be at least "6.0.3"
|
||||
|
||||
@tdd_issue @tdd_issue_9055
|
||||
Scenario: yaml module is importable as a project dependency
|
||||
When I attempt to import the "yaml" module
|
||||
Then the import should succeed without errors
|
||||
@@ -0,0 +1,58 @@
|
||||
Feature: Real LLM actor invocation in session tell
|
||||
As a user of CleverAgents
|
||||
I want `agents session tell` to invoke the real orchestrator actor
|
||||
So that I get meaningful AI responses instead of stub acknowledgements
|
||||
|
||||
Background:
|
||||
Given a session tell LLM mock environment
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: Real LLM response is returned and persisted
|
||||
Given a session with actor "openai/gpt-4" exists
|
||||
When I invoke session tell with prompt "What can you do?"
|
||||
Then the tell command returns the LLM response
|
||||
And the assistant message is persisted to the session
|
||||
And token usage is recorded
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: --stream flag yields incremental tokens
|
||||
Given a session with actor "openai/gpt-4" exists
|
||||
When I invoke session tell with --stream and prompt "Hello"
|
||||
Then the streamed output contains the LLM response tokens
|
||||
And the assistant message is persisted after streaming
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: No actor configured raises clear error with exit code 1
|
||||
Given a session with no actor exists
|
||||
When I invoke session tell with prompt "Hello"
|
||||
Then the tell command exits with code 1
|
||||
And the error output mentions actor configuration
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: --actor flag overrides the session's bound actor
|
||||
Given a session with actor "anthropic/claude-3-haiku" exists
|
||||
When I invoke session tell with --actor "openai/gpt-4" and prompt "Hello"
|
||||
Then the tell command uses the override actor "openai/gpt-4"
|
||||
And the tell command returns the LLM response
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: --format json output includes usage object
|
||||
Given a session with actor "openai/gpt-4" exists
|
||||
When I invoke session tell with --format json and prompt "What can you do?"
|
||||
Then the tell output is valid JSON with a data envelope
|
||||
And the data section contains the session_id
|
||||
And the data section contains a usage object with expected keys
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: --stream --format json produces valid JSON with usage
|
||||
Given a session with actor "openai/gpt-4" exists
|
||||
When I invoke session tell with --stream --format json and prompt "Hello"
|
||||
Then the tell output is valid JSON with a data envelope
|
||||
And the data section contains a usage object with expected keys
|
||||
|
||||
@session_tell_llm
|
||||
Scenario: Usage panel appears in Rich output
|
||||
Given a session with actor "openai/gpt-4" exists
|
||||
When I invoke session tell with prompt "What can you do?"
|
||||
Then the tell command returns the LLM response
|
||||
And the output contains a Usage panel
|
||||
@@ -0,0 +1,105 @@
|
||||
Feature: Session workflow coverage boost
|
||||
Coverage boost for session_workflow.py helper functions (M1, n4).
|
||||
|
||||
Background:
|
||||
Given the session workflow coverage environment is set up
|
||||
|
||||
# _extract_content
|
||||
Scenario: _extract_content extracts text from content attribute
|
||||
Given coverage boost a mock response with content "hello world"
|
||||
When coverage boost _extract_content is called
|
||||
Then coverage boost the extracted result should be "hello world"
|
||||
|
||||
Scenario: _extract_content falls back to text attribute
|
||||
Given coverage boost a mock response with text "from text attr" and no content
|
||||
When coverage boost _extract_content is called
|
||||
Then coverage boost the extracted result should be "from text attr"
|
||||
|
||||
Scenario: _extract_content handles list content
|
||||
Given coverage boost a mock response with list content containing text dicts and plain strings
|
||||
When coverage boost _extract_content is called
|
||||
Then coverage boost the result should contain the concatenated texts
|
||||
|
||||
Scenario: _extract_content falls back to str for unknown types
|
||||
Given coverage boost a mock response with no content or text attribute
|
||||
When coverage boost _extract_content is called
|
||||
Then coverage boost the result should be a string
|
||||
|
||||
# _extract_token_usage
|
||||
Scenario: _extract_token_usage reads from response_metadata.usage
|
||||
Given coverage boost a mock response with response_metadata usage input_tokens=100 output_tokens=50
|
||||
When coverage boost _extract_token_usage is called
|
||||
Then coverage boost input tokens should be 100 and output tokens should be 50
|
||||
|
||||
Scenario: _extract_token_usage reads from response_metadata.token_usage
|
||||
Given coverage boost a mock response with response_metadata token_usage input_tokens=200 output_tokens=100
|
||||
When coverage boost _extract_token_usage is called
|
||||
Then coverage boost input tokens should be 200 and output tokens should be 100
|
||||
|
||||
Scenario: _extract_token_usage reads prompt_tokens and completion_tokens
|
||||
Given coverage boost a mock response with response_metadata usage prompt_tokens=300 completion_tokens=150
|
||||
When coverage boost _extract_token_usage is called
|
||||
Then coverage boost input tokens should be 300 and output tokens should be 150
|
||||
|
||||
Scenario: _extract_token_usage reads from usage_metadata
|
||||
Given coverage boost a mock response with usage_metadata input_tokens=400 output_tokens=200
|
||||
When coverage boost _extract_token_usage is called
|
||||
Then coverage boost input tokens should be 400 and output tokens should be 200
|
||||
|
||||
Scenario: _extract_token_usage returns zeros for no metadata
|
||||
Given coverage boost a mock response with no usage metadata
|
||||
When coverage boost _extract_token_usage is called
|
||||
Then coverage boost input tokens should be 0 and output tokens should be 0
|
||||
|
||||
# _estimate_cost
|
||||
Scenario: _estimate_cost computes cost from token counts
|
||||
Given coverage boost input tokens 1000 and output tokens 500
|
||||
When coverage boost _estimate_cost is called
|
||||
Then coverage boost the estimated cost should be positive
|
||||
|
||||
# _history_to_langchain_messages
|
||||
Scenario: _history_to_langchain_messages converts all roles
|
||||
Given coverage boost session messages with roles SYSTEM, USER, ASSISTANT, and TOOL
|
||||
When coverage boost _history_to_langchain_messages is called
|
||||
Then coverage boost the result should contain SystemMessage, HumanMessage, AIMessage, and ToolMessage
|
||||
|
||||
Scenario: _history_to_langchain_messages treats unknown role as human
|
||||
Given coverage boost a session message with an unknown role
|
||||
When coverage boost _history_to_langchain_messages is called
|
||||
Then coverage boost the result should contain a HumanMessage
|
||||
|
||||
Scenario: _history_to_langchain_messages handles empty list
|
||||
Given coverage boost an empty list of session messages
|
||||
When coverage boost _history_to_langchain_messages is called
|
||||
Then coverage boost the result should be an empty list
|
||||
|
||||
# LangChainSessionCaller.invoke() tool_results branch
|
||||
Scenario: LangChainSessionCaller.invoke appends tool results
|
||||
Given coverage boost a LangChainSessionCaller with a stub LLM and empty history
|
||||
When coverage boost invoke is called with tool_results containing one success and one failure
|
||||
Then coverage boost the accumulated messages should include tool result messages
|
||||
|
||||
# LangChainSessionCaller.invoke() with tool_calls in response
|
||||
Scenario: LangChainSessionCaller.invoke extracts tool calls from response
|
||||
Given coverage boost a LangChainSessionCaller with a stub LLM that returns tool calls
|
||||
When coverage boost invoke is called for the first time
|
||||
Then coverage boost the LLMResponse should contain the extracted tool calls
|
||||
|
||||
# _build_lc_messages_from_history
|
||||
Scenario: _build_lc_messages_from_history adds system prompt when absent
|
||||
Given coverage boost a SessionWorkflow with a stub service and no registry
|
||||
And coverage boost session history without a system message
|
||||
When coverage boost _build_lc_messages_from_history is called with a prompt
|
||||
Then coverage boost the first message should be a SystemMessage with the session system prompt
|
||||
|
||||
# _MinimalStubLLM
|
||||
Scenario: _MinimalStubLLM.invoke returns stub response
|
||||
Given coverage boost a _MinimalStubLLM instance
|
||||
When coverage boost invoke on the stub is called
|
||||
Then coverage boost the stub response content should be "(no LLM configured)"
|
||||
And coverage boost the stub response should have empty tool_calls
|
||||
|
||||
Scenario: _MinimalStubLLM.stream yields stub chunk
|
||||
Given coverage boost a _MinimalStubLLM instance
|
||||
When coverage boost stream on the stub is called
|
||||
Then coverage boost it should yield a chunk with content "(no LLM configured)"
|
||||
@@ -127,7 +127,8 @@ def step_call_notify_facade(context: Any, operation: str) -> None:
|
||||
@then("the facade should support all 42 operations")
|
||||
def step_check_42_operations(context: Any) -> None:
|
||||
ops = context.facade.list_operations()
|
||||
assert len(ops) == 42, f"Expected 42 operations, got {len(ops)}: {ops}"
|
||||
# 42 original ops + 2 standard A2A ops (message/send, message/stream) = 44
|
||||
assert len(ops) == 44, f"Expected 44 operations, got {len(ops)}: {ops}"
|
||||
|
||||
|
||||
@then("the facade should be cached on subsequent calls")
|
||||
|
||||
@@ -17,11 +17,6 @@ from cleveragents.actor.yaml_loader import (
|
||||
interpolate_env_vars,
|
||||
load_yaml_text,
|
||||
)
|
||||
from cleveragents.actor.yaml_loader import (
|
||||
_restore_template_syntax,
|
||||
interpolate_env_vars,
|
||||
load_yaml_text,
|
||||
)
|
||||
|
||||
|
||||
@then('an actor config ValueError should mention "{text}"')
|
||||
@@ -314,76 +309,6 @@ def step_assert_options(context: Context) -> None:
|
||||
assert context.actor_cfg.options["temperature"] == 0.9
|
||||
|
||||
|
||||
@when("I extract v2 actor from a v2-style config with agents block")
|
||||
def step_extract_v2_actor(context: Context) -> None:
|
||||
data: dict[str, Any] = {
|
||||
"agents": {
|
||||
"main_agent": {
|
||||
"config": {
|
||||
"provider": "anthropic",
|
||||
"model": "claude-3",
|
||||
"unsafe": True,
|
||||
"options": {"max_tokens": 1000},
|
||||
}
|
||||
}
|
||||
},
|
||||
"routes": [{"from": "main_agent", "to": "end"}],
|
||||
}
|
||||
context.extracted = ActorConfiguration._extract_v2_actor(data)
|
||||
|
||||
|
||||
@then("the extracted provider and model should be set")
|
||||
def step_assert_extracted(context: Context) -> None:
|
||||
provider, model, _graph, unsafe = context.extracted
|
||||
assert provider == "anthropic"
|
||||
assert model == "claude-3"
|
||||
assert unsafe is True
|
||||
|
||||
|
||||
@then("the graph descriptor should contain agent info")
|
||||
def step_assert_graph_descriptor(context: Context) -> None:
|
||||
_, _, graph, _ = context.extracted
|
||||
assert graph is not None
|
||||
assert "agent" in graph
|
||||
assert "routes" in graph
|
||||
|
||||
|
||||
@when("I extract v2 actor from a config without agents block")
|
||||
def step_extract_v2_no_agents(context: Context) -> None:
|
||||
context.extracted = ActorConfiguration._extract_v2_actor({"provider": "openai"})
|
||||
|
||||
|
||||
@then("all extracted values should be None")
|
||||
def step_assert_all_none(context: Context) -> None:
|
||||
provider, model, graph, unsafe = context.extracted
|
||||
assert provider is None
|
||||
assert model is None
|
||||
assert graph is None
|
||||
assert unsafe is False
|
||||
|
||||
|
||||
@when("I extract v2 options from a v2-style config")
|
||||
def step_extract_v2_options(context: Context) -> None:
|
||||
data: dict[str, Any] = {
|
||||
"agents": {
|
||||
"main_agent": {
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"options": {"temperature": 0.7, "max_tokens": 500},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
context.options = ActorConfiguration._extract_v2_options(data)
|
||||
|
||||
|
||||
@then("the extracted options should contain expected keys")
|
||||
def step_assert_extracted_options(context: Context) -> None:
|
||||
assert context.options is not None
|
||||
assert "temperature" in context.options
|
||||
|
||||
|
||||
@when("I create an ActorConfiguration from the file")
|
||||
def step_from_file(context: Context) -> None:
|
||||
context.actor_cfg = ActorConfiguration.from_file(
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -525,88 +525,6 @@ def step_validation_error_mapping(context: Any) -> None:
|
||||
assert "mapping" in msg, f"Error should mention 'mapping': {context.add_error}"
|
||||
|
||||
|
||||
@given("a legacy v3 YAML text with invalid schema")
|
||||
def step_legacy_v3_yaml_invalid_schema(context: Any) -> None:
|
||||
# is_v3_yaml detects version starting with "3" or non-null type field.
|
||||
# Provide a blob that triggers the v3 validation path but fails schema.
|
||||
context.legacy_v3_yaml_invalid = yaml.safe_dump(
|
||||
{
|
||||
"name": "local/bad-v3",
|
||||
"version": "3.0",
|
||||
"type": "invalid_type_value", # not llm/graph/tool — fails ActorConfigSchema
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@when("I call ActorRegistry.add with the legacy invalid v3 YAML")
|
||||
def step_call_registry_add_legacy_invalid_v3(context: Any) -> None:
|
||||
try:
|
||||
context.registry.add(context.legacy_v3_yaml_invalid)
|
||||
context.add_error = None
|
||||
except ValidationError as exc:
|
||||
context.add_error = exc
|
||||
|
||||
|
||||
@then("a ValidationError should be raised mentioning invalid v3 actor")
|
||||
def step_validation_error_invalid_v3(context: Any) -> None:
|
||||
assert context.add_error is not None, "Expected a ValidationError to be raised"
|
||||
msg = str(context.add_error).lower()
|
||||
assert "invalid" in msg or "v3" in msg or "validation" in msg, (
|
||||
f"Error should mention invalid v3 actor: {context.add_error}"
|
||||
)
|
||||
|
||||
|
||||
@given("a legacy non-v3 YAML text missing provider and model")
|
||||
def step_legacy_non_v3_yaml_missing_fields(context: Any) -> None:
|
||||
context.legacy_non_v3_yaml = yaml.safe_dump(
|
||||
{
|
||||
"name": "local/no-provider",
|
||||
# no type field → not v3, no provider/model → should fail
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@when("I call ActorRegistry.add with the legacy non-v3 YAML")
|
||||
def step_call_registry_add_legacy_non_v3(context: Any) -> None:
|
||||
try:
|
||||
context.registry.add(context.legacy_non_v3_yaml)
|
||||
context.add_error = None
|
||||
except ValidationError as exc:
|
||||
context.add_error = exc
|
||||
|
||||
|
||||
@then("a ValidationError should be raised mentioning provider and model")
|
||||
def step_validation_error_provider_model(context: Any) -> None:
|
||||
assert context.add_error is not None, "Expected a ValidationError to be raised"
|
||||
msg = str(context.add_error).lower()
|
||||
assert "provider" in msg or "model" in msg, (
|
||||
f"Error should mention 'provider' or 'model': {context.add_error}"
|
||||
)
|
||||
|
||||
|
||||
@given("a legacy actor YAML text marked unsafe")
|
||||
def step_legacy_actor_yaml_unsafe(context: Any) -> None:
|
||||
context.legacy_unsafe_yaml = yaml.safe_dump(
|
||||
{
|
||||
"name": "local/legacy-unsafe",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"unsafe": True,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@when("I call ActorRegistry.add with the legacy unsafe YAML")
|
||||
def step_call_registry_add_legacy_unsafe(context: Any) -> None:
|
||||
try:
|
||||
context.registry.add(context.legacy_unsafe_yaml)
|
||||
context.add_error = None
|
||||
except ValidationError as exc:
|
||||
context.add_error = exc
|
||||
|
||||
|
||||
@when("I call ActorRegistry.upsert_actor with an invalid v3 config_blob")
|
||||
def step_call_registry_upsert_invalid_v3(context: Any) -> None:
|
||||
# Provide a blob that triggers is_v3_yaml() but fails ActorConfigSchema.
|
||||
@@ -629,6 +547,15 @@ def step_call_registry_upsert_invalid_v3(context: Any) -> None:
|
||||
context.add_error = exc
|
||||
|
||||
|
||||
@then("a ValidationError should be raised mentioning invalid v3 actor")
|
||||
def step_validation_error_invalid_v3(context: Any) -> None:
|
||||
assert context.add_error is not None, "Expected a ValidationError to be raised"
|
||||
msg = str(context.add_error).lower()
|
||||
assert "invalid" in msg or "v3" in msg or "validation" in msg, (
|
||||
f"Error should mention invalid v3 actor: {context.add_error}"
|
||||
)
|
||||
|
||||
|
||||
# ── Coverage gap: config_parser.py ───────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -93,22 +93,6 @@ def step_v3_graph_blob(context: Any) -> None:
|
||||
}
|
||||
|
||||
|
||||
@given("a v2 actor blob with agents and routes")
|
||||
def step_v2_blob(context: Any) -> None:
|
||||
context.v2_blob = {
|
||||
"agents": {
|
||||
"main": {
|
||||
"type": "llm",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
},
|
||||
}
|
||||
},
|
||||
"routes": {},
|
||||
}
|
||||
|
||||
|
||||
@given("a mock actor service for v3 testing")
|
||||
def step_mock_actor_service(context: Any) -> None:
|
||||
mock_actor_service = MagicMock()
|
||||
@@ -247,13 +231,6 @@ def step_v3_graph_config_dict(context: Any) -> None:
|
||||
}
|
||||
|
||||
|
||||
@given("a v2 actor config dict with agents key")
|
||||
def step_v2_config_dict(context: Any) -> None:
|
||||
context.v2_config = {
|
||||
"agents": {"main": {"type": "llm", "config": {"provider": "openai"}}},
|
||||
}
|
||||
|
||||
|
||||
@given("an existing actor in the mock service")
|
||||
def step_existing_actor(context: Any) -> None:
|
||||
"""Set up mock to return an existing actor for duplicate checks."""
|
||||
@@ -274,14 +251,6 @@ def step_call_from_blob_v3(context: Any) -> None:
|
||||
)
|
||||
|
||||
|
||||
@when("I call ActorConfiguration.from_blob with the v2 blob")
|
||||
def step_call_from_blob_v2(context: Any) -> None:
|
||||
context.actor_config = ActorConfiguration.from_blob(
|
||||
blob=context.v2_blob,
|
||||
name="local/test",
|
||||
)
|
||||
|
||||
|
||||
@when("I call ActorRegistry.add with the v3 YAML")
|
||||
def step_call_registry_add_v3(context: Any) -> None:
|
||||
context.result_actor = context.registry.add(context.v3_yaml_text)
|
||||
@@ -329,11 +298,6 @@ def step_parse_v3_config(context: Any) -> None:
|
||||
context.reactive_config = parser._build(context.v3_config)
|
||||
|
||||
|
||||
@when("I check if the config is v3 format")
|
||||
def step_check_v3_format(context: Any) -> None:
|
||||
context.is_v3 = ReactiveConfigParser._is_v3_format(context.v2_config)
|
||||
|
||||
|
||||
# ── Then steps ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -470,11 +434,6 @@ def step_reactive_has_graph_route(context: Any) -> None:
|
||||
assert len(route.edges) >= 1, f"Expected at least 1 edge, got {len(route.edges)}"
|
||||
|
||||
|
||||
@then("it should not be detected as v3")
|
||||
def step_not_v3(context: Any) -> None:
|
||||
assert context.is_v3 is False, "Expected v2 data to NOT be detected as v3"
|
||||
|
||||
|
||||
@then("the graph route edges should use source and target keys")
|
||||
def step_graph_edges_use_source_target(context: Any) -> None:
|
||||
routes = context.reactive_config.routes
|
||||
|
||||
@@ -807,6 +807,22 @@ def step_pec_output_valid_json_list(context: Context) -> None:
|
||||
assert isinstance(data, list), "Expected JSON array"
|
||||
|
||||
|
||||
@then("pec the output should be valid json envelope")
|
||||
def step_pec_output_valid_json_envelope(context: Context) -> None:
|
||||
parsed = json.loads(context.pec_result.output.strip())
|
||||
assert isinstance(parsed, dict), f"Expected JSON object, got {type(parsed)}"
|
||||
assert _ENVELOPE_KEYS.issubset(parsed.keys()), (
|
||||
f"Expected envelope keys {_ENVELOPE_KEYS}, got {set(parsed.keys())}"
|
||||
)
|
||||
data = parsed["data"]
|
||||
assert isinstance(data, dict), (
|
||||
f"Expected envelope data to be a dict, got {type(data)}"
|
||||
)
|
||||
assert "plan_id" in data, (
|
||||
f"Expected 'plan_id' in envelope data, got {set(data.keys())}"
|
||||
)
|
||||
|
||||
|
||||
@then("pec the tree should exclude the superseded grandchild")
|
||||
def step_pec_tree_excludes_orphan(context: Context) -> None:
|
||||
# Tree should have exactly one root with one child and zero grandchildren.
|
||||
|
||||
@@ -405,6 +405,18 @@ def step_tree_json_valid(context: Context) -> None:
|
||||
assert isinstance(parsed, list), "Expected a JSON array"
|
||||
|
||||
|
||||
@then("the json tree output should be a valid json envelope")
|
||||
def step_tree_json_valid_envelope(context: Context) -> None:
|
||||
parsed = json.loads(context.pe_tree_json)
|
||||
assert isinstance(parsed, dict), (
|
||||
f"Expected a JSON object (envelope), got {type(parsed)}"
|
||||
)
|
||||
_envelope_keys = {"command", "status", "exit_code", "data", "timing", "messages"}
|
||||
assert _envelope_keys.issubset(parsed.keys()), (
|
||||
f"Expected envelope keys {_envelope_keys}, got {set(parsed.keys())}"
|
||||
)
|
||||
|
||||
|
||||
@then('the json tree output should contain "{text}"')
|
||||
def step_tree_json_contains(context: Context, text: str) -> None:
|
||||
assert text in context.pe_tree_json, f"Expected '{text}' in tree JSON"
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Step definitions for PyYAML security dependency verification (#9055).
|
||||
|
||||
Note: the shared steps for reading pyproject.toml and verifying importability
|
||||
are defined in ``tdd_a2a_sdk_dependency_steps.py`` to avoid AmbiguousStep
|
||||
conflicts. This file contains only the PyYAML-specific step definitions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
from behave import then, when
|
||||
from behave.runner import Context
|
||||
from packaging.version import parse as parse_version
|
||||
|
||||
|
||||
@then('the dependency list should include a package matching "{pattern}"')
|
||||
def step_dependency_matches_pattern(context: Context, pattern: str) -> None:
|
||||
"""Assert that a dependency in the project dependencies matches the given regex pattern."""
|
||||
deps: list[str] = context.project_dependencies
|
||||
found = any(re.search(pattern, dep) for dep in deps)
|
||||
assert found, (
|
||||
f"No dependency matching '{pattern}' found in "
|
||||
f"[project.dependencies]. Current deps: {deps}"
|
||||
)
|
||||
|
||||
|
||||
@when("I read the PyYAML dependency specification from pyproject.toml")
|
||||
def step_read_pyyaml_dependency(context: Context) -> None:
|
||||
"""Read and parse the PyYAML dependency from pyproject.toml."""
|
||||
content = context.pyproject_path.read_bytes()
|
||||
data: dict[str, Any] = _load_toml(content)
|
||||
deps: list[str] = data.get("project", {}).get("dependencies", [])
|
||||
|
||||
# Find the pyyaml dependency entry
|
||||
pyyaml_dep = None
|
||||
for dep in deps:
|
||||
if dep.strip().startswith("pyyaml"):
|
||||
pyyaml_dep = dep.strip()
|
||||
break
|
||||
|
||||
assert pyyaml_dep is not None, (
|
||||
f"PyYAML dependency not found in [project.dependencies]. Current deps: {deps}"
|
||||
)
|
||||
|
||||
# Parse the minimum version from the constraint
|
||||
# Examples: "pyyaml>=6.0.3", "pyyaml>=6.0", "pyyaml==6.0.2"
|
||||
match = re.search(r"(?:>=|==|~>)([\d]+(?:\.[\d]+)*(?:\.\w+)*)", pyyaml_dep)
|
||||
assert match is not None, (
|
||||
f"Could not parse version from PyYAML dependency: {pyyaml_dep}"
|
||||
)
|
||||
|
||||
context.pyyaml_version_spec = pyyaml_dep
|
||||
context.pyyaml_min_version = match.group(1)
|
||||
|
||||
|
||||
@then('the minimum version should be at least "{expected}"')
|
||||
def step_pyyaml_minimum_version(context: Context, expected: str) -> None:
|
||||
"""Assert that the PyYAML minimum version is >= expected version."""
|
||||
min_version = context.pyyaml_min_version
|
||||
assert parse_version(min_version) >= parse_version(expected), (
|
||||
f"PyYAML minimum version {min_version} is less than required {expected}. "
|
||||
f"Dependency spec: {context.pyyaml_version_spec}"
|
||||
)
|
||||
|
||||
|
||||
def _flatten_toml_dict(d: dict[Any, Any]) -> dict[str, Any]:
|
||||
"""Recursively convert a tomlkit dict to a plain Python dict.
|
||||
|
||||
tomlkit wraps every value in a proxy object that behaves like the
|
||||
underlying Python type but is not identical to it. This helper
|
||||
strips those wrappers so the result is a plain ``dict[str, Any]``
|
||||
compatible with ``tomllib`` output.
|
||||
"""
|
||||
result: dict[str, Any] = {}
|
||||
for key, value in d.items():
|
||||
if isinstance(value, dict):
|
||||
result[key] = _flatten_toml_dict(value)
|
||||
else:
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
|
||||
def _toml_to_python(data: object) -> dict[str, Any]:
|
||||
"""Convert a tomlkit document to a plain Python dict.
|
||||
|
||||
Delegates recursive flattening to the module-level
|
||||
``_flatten_toml_dict`` helper so the logic stays testable in
|
||||
isolation and ruff does not flag nested function definitions.
|
||||
"""
|
||||
if isinstance(data, dict):
|
||||
return _flatten_toml_dict(data)
|
||||
raise TypeError(f"Expected dict-like TOML table, got {type(data).__name__}")
|
||||
|
||||
|
||||
def _load_toml(content: bytes) -> dict[str, Any]:
|
||||
"""Load a TOML file from bytes. Uses tomllib if available, else fallback."""
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
return tomllib.loads(content.decode("utf-8"))
|
||||
except ImportError: # pragma: no cover — Python <3.11
|
||||
# Fallback for older Python versions without tomllib
|
||||
import tomlkit
|
||||
|
||||
parsed_doc = tomlkit.parse(content.decode("utf-8"))
|
||||
return _toml_to_python(parsed_doc)
|
||||
@@ -27,6 +27,7 @@ from unittest.mock import MagicMock, patch
|
||||
from behave import given, then, when
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.services.session_workflow import TellResult
|
||||
from cleveragents.cli.commands.session import app as session_app
|
||||
from cleveragents.core.exceptions import DatabaseError
|
||||
from cleveragents.domain.models.core.cost_budget import SessionCostBudget
|
||||
@@ -101,15 +102,13 @@ def _mock_service() -> MagicMock:
|
||||
|
||||
|
||||
def _patch_service(context, svc):
|
||||
"""Patch _get_session_service to return *svc* deterministically.
|
||||
"""Patch ``_get_session_service`` to return *svc* deterministically.
|
||||
|
||||
Patching the function directly (rather than the module-level ``_service``
|
||||
attribute) prevents a race condition in parallel Behave workers: if a
|
||||
concurrent worker's cleanup resets ``_service`` to ``None`` while this
|
||||
scenario's CLI command is executing, ``_get_session_service()`` would fall
|
||||
through to the real DI container and raise, producing a spurious exit
|
||||
code 1. Patching the function itself short-circuits the cache check
|
||||
entirely and is immune to cross-worker ``_service`` mutations.
|
||||
Patches the function rather than mutating the module-level ``_service``
|
||||
singleton to prevent a race condition in parallel Behave workers.
|
||||
Mutating ``_service`` could cause concurrent cleanup in one worker to
|
||||
reset it to ``None`` while another worker was still using it, leading
|
||||
to intermittent test failures.
|
||||
"""
|
||||
patcher = patch(
|
||||
"cleveragents.cli.commands.session._get_session_service",
|
||||
@@ -119,6 +118,33 @@ def _patch_service(context, svc):
|
||||
context._scvbst_cleanups.append(patcher.stop)
|
||||
|
||||
|
||||
def _patch_workflow(context, wf):
|
||||
"""Patch _build_session_workflow to return *wf* deterministically."""
|
||||
patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=wf,
|
||||
)
|
||||
patcher.start()
|
||||
context._scvbst_cleanups.append(patcher.stop)
|
||||
|
||||
|
||||
def _make_tell_result(
|
||||
session_id: str = _ULID1,
|
||||
prompt: str = "Hello world",
|
||||
response: str = "Acknowledged: Hello world",
|
||||
) -> TellResult:
|
||||
return TellResult(
|
||||
session_id=session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=response,
|
||||
input_tokens=5,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -601,7 +627,14 @@ def step_invoke_import_db_error(context):
|
||||
def step_tell_service(context):
|
||||
svc = _mock_service()
|
||||
svc.append_message.return_value = None
|
||||
svc.get_messages.return_value = []
|
||||
_patch_service(context, svc)
|
||||
# Patch workflow so tell uses a canned response without LLM/DB.
|
||||
wf = MagicMock()
|
||||
wf.tell.return_value = _make_tell_result()
|
||||
wf.tell_stream.return_value = iter(["Acknowledged: Hello stream"])
|
||||
_patch_workflow(context, wf)
|
||||
context._scvbst_mock_wf = wf
|
||||
|
||||
|
||||
@when("session coverage boost I invoke the tell command without stream")
|
||||
@@ -622,6 +655,11 @@ def step_invoke_tell_stream(context):
|
||||
|
||||
@when("session coverage boost I invoke the tell command with actor override")
|
||||
def step_invoke_tell_actor(context):
|
||||
# Override the mock workflow response to include actor name
|
||||
context._scvbst_mock_wf.tell.return_value = _make_tell_result(
|
||||
prompt="Plan a feature",
|
||||
response="[openai/gpt-4] Acknowledged: Plan a feature",
|
||||
)
|
||||
context.result = _runner.invoke(
|
||||
session_app,
|
||||
[
|
||||
@@ -647,6 +685,11 @@ def step_tell_not_found(context):
|
||||
svc = _mock_service()
|
||||
svc.append_message.side_effect = SessionNotFoundError("no session")
|
||||
_patch_service(context, svc)
|
||||
# Workflow raises SessionNotFoundError so the CLI handler catches it.
|
||||
wf = MagicMock()
|
||||
wf.tell.side_effect = SessionNotFoundError("no session")
|
||||
wf.tell_stream.side_effect = SessionNotFoundError("no session")
|
||||
_patch_workflow(context, wf)
|
||||
|
||||
|
||||
@given("session coverage boost a mock service that raises DatabaseError on append")
|
||||
@@ -654,6 +697,11 @@ def step_tell_db_error(context):
|
||||
svc = _mock_service()
|
||||
svc.append_message.side_effect = DatabaseError("tell db fail")
|
||||
_patch_service(context, svc)
|
||||
# Workflow raises DatabaseError so the CLI handler catches it.
|
||||
wf = MagicMock()
|
||||
wf.tell.side_effect = DatabaseError("tell db fail")
|
||||
wf.tell_stream.side_effect = DatabaseError("tell db fail")
|
||||
_patch_workflow(context, wf)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -8,14 +8,14 @@ import re
|
||||
import tempfile
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from typer.testing import CliRunner
|
||||
from ulid import ULID
|
||||
|
||||
from cleveragents.cli.commands import session as session_mod
|
||||
from cleveragents.application.services.session_workflow import TellResult
|
||||
from cleveragents.cli.commands.session import app as session_app
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
@@ -98,12 +98,43 @@ def step_session_cli_runner(context: Context) -> None:
|
||||
# Default: create returns a session
|
||||
default_session = _make_session(session_id=_SESSION_ID)
|
||||
context.mock_service.create.return_value = default_session
|
||||
# get_messages returns empty list by default (used by SessionWorkflow)
|
||||
context.mock_service.get_messages.return_value = []
|
||||
|
||||
# Patch the module-level service
|
||||
session_mod._service = context.mock_service
|
||||
# Patch _get_session_service so tests are safe for parallel workers
|
||||
# and do not mutate the module-level _service cache (M9).
|
||||
svc_patcher = patch(
|
||||
"cleveragents.cli.commands.session._get_session_service",
|
||||
return_value=context.mock_service,
|
||||
)
|
||||
svc_patcher.start()
|
||||
|
||||
# Patch _build_session_workflow to return a controllable mock workflow.
|
||||
# This lets tell-specific tests configure exact return values without
|
||||
# needing a real LLM or provider registry in the test environment.
|
||||
context.mock_workflow = MagicMock()
|
||||
_default_tell_result = TellResult(
|
||||
session_id=_SESSION_ID,
|
||||
user_message="",
|
||||
assistant_message="Acknowledged",
|
||||
input_tokens=10,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.mock_workflow.tell.return_value = _default_tell_result
|
||||
context.mock_workflow.tell_stream.return_value = iter(["Acknowledged"])
|
||||
|
||||
workflow_patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=context.mock_workflow,
|
||||
)
|
||||
workflow_patcher.start()
|
||||
|
||||
def cleanup() -> None:
|
||||
session_mod._service = None
|
||||
svc_patcher.stop()
|
||||
workflow_patcher.stop()
|
||||
_cleanup(context)
|
||||
|
||||
context.add_cleanup(cleanup)
|
||||
@@ -306,10 +337,17 @@ def step_capture_first_ulid(context: Context) -> None:
|
||||
def step_tell_with_stored_ulid(context: Context, prompt: str) -> None:
|
||||
"""Run session tell using the full ULID stored from session list."""
|
||||
session_id = context.full_session_id
|
||||
context.mock_service.append_message.side_effect = [
|
||||
_make_message(MessageRole.USER, prompt, 0),
|
||||
_make_message(MessageRole.ASSISTANT, f"Acknowledged: {prompt}", 1),
|
||||
]
|
||||
# Configure mock workflow for this prompt.
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Acknowledged: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", session_id, prompt],
|
||||
@@ -563,21 +601,37 @@ def step_import_succeeds(context: Context) -> None:
|
||||
|
||||
@given("there is a mocked session for tell")
|
||||
def step_session_for_tell(context: Context) -> None:
|
||||
session = _make_session(session_id=_SESSION_ID)
|
||||
# Session with actor so tell can proceed without --actor flag.
|
||||
session = _make_session(session_id=_SESSION_ID, actor_name="openai/gpt-4")
|
||||
context.mock_service.get.return_value = session
|
||||
context.mock_service.append_message.side_effect = [
|
||||
_make_message(MessageRole.USER, "Hello, world", 0),
|
||||
_make_message(MessageRole.ASSISTANT, "Acknowledged: Hello, world", 1),
|
||||
]
|
||||
context.mock_service.get_messages.return_value = []
|
||||
context.session_id = _SESSION_ID
|
||||
# Configure the mock workflow to return a canned TellResult.
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=_SESSION_ID,
|
||||
user_message="Hello, world",
|
||||
assistant_message="Acknowledged: Hello, world",
|
||||
input_tokens=10,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
|
||||
|
||||
@when('I run session CLI tell with a prompt "{prompt}"')
|
||||
def step_tell_prompt(context: Context, prompt: str) -> None:
|
||||
context.mock_service.append_message.side_effect = [
|
||||
_make_message(MessageRole.USER, prompt, 0),
|
||||
_make_message(MessageRole.ASSISTANT, f"Acknowledged: {prompt}", 1),
|
||||
]
|
||||
# Update the mock workflow result to reflect the current prompt.
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Acknowledged: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, prompt],
|
||||
@@ -586,14 +640,17 @@ def step_tell_prompt(context: Context, prompt: str) -> None:
|
||||
|
||||
@when('I run session CLI tell with --actor "{actor}" and prompt "{prompt}"')
|
||||
def step_tell_with_actor(context: Context, actor: str, prompt: str) -> None:
|
||||
context.mock_service.append_message.side_effect = [
|
||||
_make_message(MessageRole.USER, prompt, 0),
|
||||
_make_message(
|
||||
MessageRole.ASSISTANT,
|
||||
f"[{actor}] Acknowledged: {prompt}",
|
||||
1,
|
||||
),
|
||||
]
|
||||
# Update the mock workflow result to reflect actor and prompt.
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"[{actor}] Acknowledged: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "--actor", actor, prompt],
|
||||
@@ -602,9 +659,8 @@ def step_tell_with_actor(context: Context, actor: str, prompt: str) -> None:
|
||||
|
||||
@when("I run session CLI tell to a non-existent session")
|
||||
def step_tell_nonexistent(context: Context) -> None:
|
||||
context.mock_service.append_message.side_effect = SessionNotFoundError(
|
||||
"Session not found"
|
||||
)
|
||||
# Configure the workflow to raise SessionNotFoundError.
|
||||
context.mock_workflow.tell.side_effect = SessionNotFoundError("Session not found")
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", "NONEXISTENT", "Hello"],
|
||||
|
||||
@@ -9,6 +9,7 @@ from unittest.mock import MagicMock, patch
|
||||
from behave import given, then, when
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.services.session_workflow import TellResult
|
||||
from cleveragents.cli.commands.session import app as session_app
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
@@ -358,10 +359,28 @@ def step_export_output_contains(context, text):
|
||||
@given("session cli branch a mock session service for tell")
|
||||
def step_service_for_tell(context):
|
||||
svc = _mock_service()
|
||||
# append_message doesn't need to return anything meaningful for the
|
||||
# code path under test — the assistant content is computed inline.
|
||||
svc.append_message.return_value = None
|
||||
svc.get_messages.return_value = []
|
||||
_patch_service(context, svc)
|
||||
# Patch _build_session_workflow to return a controllable mock.
|
||||
mock_wf = MagicMock()
|
||||
mock_wf.tell_stream.return_value = iter(["Acknowledged: Hello world"])
|
||||
mock_wf.tell.return_value = TellResult(
|
||||
session_id=_ULID,
|
||||
user_message="Hello world",
|
||||
assistant_message="Acknowledged: Hello world",
|
||||
input_tokens=5,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
wf_patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=mock_wf,
|
||||
)
|
||||
wf_patcher.start()
|
||||
context._cleanup_handlers.append(wf_patcher.stop)
|
||||
|
||||
|
||||
@when("session cli branch I invoke the tell command with --stream")
|
||||
@@ -374,7 +393,8 @@ def step_invoke_tell_stream(context):
|
||||
|
||||
@then("session cli branch the streamed output contains the assistant response")
|
||||
def step_streamed_output(context):
|
||||
# The assistant content for no actor is: "Acknowledged: Hello world"
|
||||
# After the stub LLM replacement the assistant response comes from the
|
||||
# mock workflow's tell_stream. Check that ANY assistant output arrived.
|
||||
assert "Acknowledged" in context.result.output, (
|
||||
f"Expected 'Acknowledged' in output. Got:\n{context.result.output}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,565 @@
|
||||
"""Step definitions for session tell real LLM invocation tests (issue #5784).
|
||||
|
||||
Tests verify that ``agents session tell`` routes through
|
||||
:class:`~cleveragents.application.services.session_workflow.SessionWorkflow`
|
||||
with a real (mock) :class:`LLMCaller`, persists the response, and records
|
||||
token usage.
|
||||
|
||||
All LLM interactions are handled via a mock ``LLMCaller`` so no real API
|
||||
keys or network access is required.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.services.session_workflow import SessionWorkflow
|
||||
from cleveragents.cli.commands.session import app as session_app
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
Session,
|
||||
SessionMessage,
|
||||
SessionService,
|
||||
SessionTokenUsage,
|
||||
)
|
||||
|
||||
_ULID = "01KJ1N8YDN05N29P5RZV4SPDVZ"
|
||||
_STUB_RESPONSE = "Here is what I can help you with."
|
||||
|
||||
|
||||
class _StubResp:
|
||||
"""Minimal LLM response stub for _StubLLM.invoke()."""
|
||||
|
||||
def __init__(self, content: str) -> None:
|
||||
self.content = content
|
||||
self.tool_calls: list[Any] = []
|
||||
self.response_metadata: dict[str, Any] = {
|
||||
"usage": {"input_tokens": 10, "output_tokens": 5}
|
||||
}
|
||||
|
||||
|
||||
class _StubChunk:
|
||||
"""Minimal streaming chunk for _StubLLM.stream()."""
|
||||
|
||||
def __init__(self, content: str) -> None:
|
||||
self.content = content
|
||||
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
class _StubLLM:
|
||||
"""Minimal LangChain-compatible LLM stub for LangChainSessionCaller tests."""
|
||||
|
||||
def __init__(self, response: str = _STUB_RESPONSE) -> None:
|
||||
self._response = response
|
||||
self.invoked_messages: list[Any] = []
|
||||
|
||||
def invoke(self, messages: Any, **kwargs: Any) -> _StubResp:
|
||||
self.invoked_messages = list(messages)
|
||||
return _StubResp(self._response)
|
||||
|
||||
def stream(self, messages: Any, **kwargs: Any) -> Iterator[_StubChunk]:
|
||||
self.invoked_messages = list(messages)
|
||||
words = self._response.split()
|
||||
for i, word in enumerate(words):
|
||||
# Reconstruct spaces between words so the concatenated
|
||||
# output matches the original response string verbatim.
|
||||
token = word if i == 0 else " " + word
|
||||
yield _StubChunk(token)
|
||||
|
||||
|
||||
def _make_session(
|
||||
session_id: str = _ULID,
|
||||
actor_name: str | None = None,
|
||||
) -> Session:
|
||||
return Session(
|
||||
session_id=session_id,
|
||||
actor_name=actor_name,
|
||||
namespace="local",
|
||||
messages=[],
|
||||
token_usage=SessionTokenUsage(),
|
||||
)
|
||||
|
||||
|
||||
def _make_mock_service(session: Session) -> MagicMock:
|
||||
"""Build a MagicMock SessionService pre-configured for session tell."""
|
||||
svc = MagicMock(spec=SessionService)
|
||||
svc.get.return_value = session
|
||||
svc.get_messages.return_value = []
|
||||
svc.append_message.return_value = MagicMock(spec=SessionMessage)
|
||||
svc.update_token_usage.return_value = None
|
||||
return svc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a session tell LLM mock environment")
|
||||
def step_setup_llm_mock_env(context: Context) -> None:
|
||||
"""Set up runner, stub LLM, and cleanup handlers."""
|
||||
context.runner = runner
|
||||
context.stub_llm = _StubLLM()
|
||||
context.captured_actor_name: str | None = None
|
||||
context._cleanup_handlers: list[Any] = []
|
||||
|
||||
def cleanup() -> None:
|
||||
for stop in reversed(context._cleanup_handlers):
|
||||
with contextlib.suppress(Exception):
|
||||
stop()
|
||||
|
||||
context.add_cleanup(cleanup)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('a session with actor "{actor}" exists')
|
||||
def step_session_with_actor(context: Context, actor: str) -> None:
|
||||
session = _make_session(actor_name=actor)
|
||||
context.session = session
|
||||
context.svc = _make_mock_service(session)
|
||||
# Patch _get_session_service instead of module-level _service so tests
|
||||
# are resilient to changes in the internal caching strategy (m4).
|
||||
svc_patcher = patch(
|
||||
"cleveragents.cli.commands.session._get_session_service",
|
||||
return_value=context.svc,
|
||||
)
|
||||
svc_patcher.start()
|
||||
context._cleanup_handlers.append(svc_patcher.stop)
|
||||
|
||||
|
||||
@given("a session with no actor exists")
|
||||
def step_session_no_actor(context: Context) -> None:
|
||||
session = _make_session(actor_name=None)
|
||||
context.session = session
|
||||
context.svc = _make_mock_service(session)
|
||||
svc_patcher = patch(
|
||||
"cleveragents.cli.commands.session._get_session_service",
|
||||
return_value=context.svc,
|
||||
)
|
||||
svc_patcher.start()
|
||||
context._cleanup_handlers.append(svc_patcher.stop)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I invoke session tell with prompt "{prompt}"')
|
||||
def step_invoke_tell(context: Context, prompt: str) -> None:
|
||||
"""Invoke tell routing through _facade_dispatch.
|
||||
|
||||
The non-streaming CLI path calls :func:`_facade_dispatch`, which in
|
||||
production delegates to ``A2aLocalFacade``. The test intercepts
|
||||
``_facade_dispatch`` with a thin adapter that runs the real
|
||||
``SessionWorkflow.tell()`` (backed by the test's stub LLM) and
|
||||
returns a dict matching the facade's response envelope. This
|
||||
keeps the workflow assertions (append_message, token usage) working
|
||||
while also testing the CLI's facade-routing code path.
|
||||
"""
|
||||
context.prompt = prompt
|
||||
context.invoked_actor_name = None
|
||||
stub_llm = context.stub_llm
|
||||
svc = context.svc
|
||||
|
||||
def _llm_factory(actor: str) -> Any:
|
||||
context.invoked_actor_name = actor
|
||||
return stub_llm
|
||||
|
||||
wf = SessionWorkflow(session_service=svc, llm_factory=_llm_factory)
|
||||
|
||||
def _mock_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if operation == "message/send":
|
||||
result = wf.tell(
|
||||
session_id=params["session_id"],
|
||||
prompt=params["message"],
|
||||
actor_override=params.get("actor"),
|
||||
)
|
||||
return {
|
||||
"session_id": result.session_id,
|
||||
"assistant_message": result.assistant_message,
|
||||
"usage": result.usage_dict(),
|
||||
}
|
||||
raise RuntimeError(f"Unknown operation: {operation}")
|
||||
|
||||
dispatch_patcher = patch(
|
||||
"cleveragents.cli.commands.session._facade_dispatch",
|
||||
side_effect=_mock_dispatch,
|
||||
)
|
||||
dispatch_patcher.start()
|
||||
context._cleanup_handlers.append(dispatch_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I invoke session tell with --stream and prompt "{prompt}"')
|
||||
def step_invoke_tell_stream(context: Context, prompt: str) -> None:
|
||||
"""Invoke tell --stream with a real SessionWorkflow backed by _StubLLM."""
|
||||
context.prompt = prompt
|
||||
stub_llm = context.stub_llm
|
||||
svc = context.svc
|
||||
|
||||
def _workflow_factory() -> SessionWorkflow:
|
||||
return SessionWorkflow(
|
||||
session_service=svc,
|
||||
llm_factory=lambda actor: stub_llm,
|
||||
)
|
||||
|
||||
wf_patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
side_effect=_workflow_factory,
|
||||
)
|
||||
wf_patcher.start()
|
||||
context._cleanup_handlers.append(wf_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, "--stream", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I invoke session tell with --actor "{actor}" and prompt "{prompt}"')
|
||||
def step_invoke_tell_with_actor(context: Context, actor: str, prompt: str) -> None:
|
||||
"""Invoke tell with --actor override, routing through _facade_dispatch."""
|
||||
context.prompt = prompt
|
||||
context.override_actor = actor
|
||||
stub_llm = context.stub_llm
|
||||
svc = context.svc
|
||||
|
||||
def _llm_factory(actor_name: str) -> Any:
|
||||
context.invoked_actor_name = actor_name
|
||||
return stub_llm
|
||||
|
||||
wf = SessionWorkflow(session_service=svc, llm_factory=_llm_factory)
|
||||
|
||||
def _mock_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if operation == "message/send":
|
||||
result = wf.tell(
|
||||
session_id=params["session_id"],
|
||||
prompt=params["message"],
|
||||
actor_override=params.get("actor"),
|
||||
)
|
||||
return {
|
||||
"session_id": result.session_id,
|
||||
"assistant_message": result.assistant_message,
|
||||
"usage": result.usage_dict(),
|
||||
}
|
||||
raise RuntimeError(f"Unknown operation: {operation}")
|
||||
|
||||
dispatch_patcher = patch(
|
||||
"cleveragents.cli.commands.session._facade_dispatch",
|
||||
side_effect=_mock_dispatch,
|
||||
)
|
||||
dispatch_patcher.start()
|
||||
context._cleanup_handlers.append(dispatch_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, "--actor", actor, prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I invoke session tell with --format json and prompt "{prompt}"')
|
||||
def step_invoke_tell_json(context: Context, prompt: str) -> None:
|
||||
"""Invoke tell --format json, routing through _facade_dispatch."""
|
||||
context.prompt = prompt
|
||||
stub_llm = context.stub_llm
|
||||
svc = context.svc
|
||||
|
||||
wf = SessionWorkflow(
|
||||
session_service=svc,
|
||||
llm_factory=lambda actor: stub_llm,
|
||||
)
|
||||
|
||||
def _mock_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if operation == "message/send":
|
||||
result = wf.tell(
|
||||
session_id=params["session_id"],
|
||||
prompt=params["message"],
|
||||
actor_override=params.get("actor"),
|
||||
)
|
||||
return {
|
||||
"session_id": result.session_id,
|
||||
"assistant_message": result.assistant_message,
|
||||
"usage": result.usage_dict(),
|
||||
}
|
||||
raise RuntimeError(f"Unknown operation: {operation}")
|
||||
|
||||
dispatch_patcher = patch(
|
||||
"cleveragents.cli.commands.session._facade_dispatch",
|
||||
side_effect=_mock_dispatch,
|
||||
)
|
||||
dispatch_patcher.start()
|
||||
context._cleanup_handlers.append(dispatch_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, "--format", "json", prompt],
|
||||
)
|
||||
|
||||
def _mock_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
if operation == "message/send":
|
||||
result = wf.tell(
|
||||
session_id=params["session_id"],
|
||||
prompt=params["message"],
|
||||
actor_override=params.get("actor"),
|
||||
)
|
||||
return {
|
||||
"session_id": result.session_id,
|
||||
"assistant_message": result.assistant_message,
|
||||
"usage": result.usage_dict(),
|
||||
}
|
||||
raise RuntimeError(f"Unknown operation: {operation}")
|
||||
|
||||
dispatch_patcher = patch(
|
||||
"cleveragents.cli.commands.session._facade_dispatch",
|
||||
side_effect=_mock_dispatch,
|
||||
)
|
||||
dispatch_patcher.start()
|
||||
context._cleanup_handlers.append(dispatch_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, "--format", "json", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I invoke session tell with --stream --format json and prompt "{prompt}"')
|
||||
def step_invoke_tell_stream_json(context: Context, prompt: str) -> None:
|
||||
"""Invoke tell --stream --format json with a real SessionWorkflow backed by _StubLLM."""
|
||||
context.prompt = prompt
|
||||
stub_llm = context.stub_llm
|
||||
svc = context.svc
|
||||
|
||||
def _workflow_factory() -> SessionWorkflow:
|
||||
return SessionWorkflow(
|
||||
session_service=svc,
|
||||
llm_factory=lambda actor: stub_llm,
|
||||
)
|
||||
|
||||
wf_patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
side_effect=_workflow_factory,
|
||||
)
|
||||
wf_patcher.start()
|
||||
context._cleanup_handlers.append(wf_patcher.stop)
|
||||
|
||||
context.result = runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", _ULID, "--stream", "--format", "json", prompt],
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the tell command returns the LLM response")
|
||||
def step_tell_returns_llm_response(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}.\n"
|
||||
f"Output:\n{context.result.output}"
|
||||
)
|
||||
# The stub LLM response should appear in the output
|
||||
assert _STUB_RESPONSE in context.result.output, (
|
||||
f"LLM response not found in output:\n{context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the assistant message is persisted to the session")
|
||||
def step_assistant_message_persisted(context: Context) -> None:
|
||||
# append_message should be called exactly twice:
|
||||
# once for the user message and once for the assistant response.
|
||||
call_count = context.svc.append_message.call_count
|
||||
assert call_count == 2, (
|
||||
f"Expected append_message to be called exactly 2 times (user + assistant), "
|
||||
f"got {call_count}"
|
||||
)
|
||||
# The second call should be for the ASSISTANT role
|
||||
calls = context.svc.append_message.call_args_list
|
||||
assistant_calls = [
|
||||
c
|
||||
for c in calls
|
||||
if c.kwargs.get("role") == MessageRole.ASSISTANT
|
||||
or (len(c.args) >= 2 and c.args[1] == MessageRole.ASSISTANT)
|
||||
]
|
||||
assert len(assistant_calls) == 1, (
|
||||
f"Expected exactly 1 ASSISTANT persist, got {len(assistant_calls)}. "
|
||||
f"Calls: {calls}"
|
||||
)
|
||||
# The second call should be for the ASSISTANT role
|
||||
calls = context.svc.append_message.call_args_list
|
||||
assistant_calls = [
|
||||
c
|
||||
for c in calls
|
||||
if c.kwargs.get("role") == MessageRole.ASSISTANT
|
||||
or (len(c.args) >= 2 and c.args[1] == MessageRole.ASSISTANT)
|
||||
]
|
||||
assert len(assistant_calls) == 1, (
|
||||
f"Expected exactly 1 ASSISTANT persist, got {len(assistant_calls)}. "
|
||||
f"Calls: {calls}"
|
||||
)
|
||||
|
||||
|
||||
@then("token usage is recorded")
|
||||
def step_token_usage_recorded(context: Context) -> None:
|
||||
assert context.svc.update_token_usage.called, (
|
||||
"update_token_usage was not called — token usage was not recorded"
|
||||
)
|
||||
assert context.svc.update_token_usage.call_count == 1, (
|
||||
f"Expected update_token_usage to be called exactly once, "
|
||||
f"got {context.svc.update_token_usage.call_count}"
|
||||
)
|
||||
# Verify exact token counts match the metadata from the stub LLM (m5).
|
||||
call_kwargs = context.svc.update_token_usage.call_args.kwargs
|
||||
assert call_kwargs.get("input_tokens") == 10, (
|
||||
f"Expected input_tokens=10, got {call_kwargs.get('input_tokens')}"
|
||||
)
|
||||
assert call_kwargs.get("output_tokens") == 5, (
|
||||
f"Expected output_tokens=5, got {call_kwargs.get('output_tokens')}"
|
||||
)
|
||||
assert context.svc.update_token_usage.call_count == 1, (
|
||||
f"Expected update_token_usage to be called exactly once, "
|
||||
f"got {context.svc.update_token_usage.call_count}"
|
||||
)
|
||||
# Verify exact token counts match the metadata from the stub LLM (m5).
|
||||
call_kwargs = context.svc.update_token_usage.call_args.kwargs
|
||||
assert call_kwargs.get("input_tokens") == 10, (
|
||||
f"Expected input_tokens=10, got {call_kwargs.get('input_tokens')}"
|
||||
)
|
||||
assert call_kwargs.get("output_tokens") == 5, (
|
||||
f"Expected output_tokens=5, got {call_kwargs.get('output_tokens')}"
|
||||
)
|
||||
|
||||
|
||||
@then("the streamed output contains the LLM response tokens")
|
||||
def step_streamed_output_contains_tokens(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}.\n"
|
||||
f"Output:\n{context.result.output}"
|
||||
)
|
||||
# Strip ANSI escape sequences for text matching
|
||||
output_clean = re.sub(r"\x1b\[[0-9;]*m", "", context.result.output)
|
||||
# When streamed through the A2A facade (Option B: fallback to
|
||||
# non-streaming), verify the full response is present in the output
|
||||
# rather than checking token-by-token word positions (C5).
|
||||
assert _STUB_RESPONSE in output_clean, (
|
||||
f"Expected full response '{_STUB_RESPONSE}' in streamed output, "
|
||||
f"but it was not found.\nOutput:\n{context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the assistant message is persisted after streaming")
|
||||
def step_assistant_persisted_after_stream(context: Context) -> None:
|
||||
call_count = context.svc.append_message.call_count
|
||||
assert call_count == 2, (
|
||||
f"Expected append_message to be called exactly 2 times (user + assistant), "
|
||||
f"got {call_count}"
|
||||
)
|
||||
# Verify that the last persisted message is an ASSISTANT role.
|
||||
calls = context.svc.append_message.call_args_list
|
||||
assert calls[-1].kwargs.get("role") == MessageRole.ASSISTANT, (
|
||||
f"Last persisted message role is not ASSISTANT. Calls: {calls}"
|
||||
)
|
||||
# Verify that the last persisted message is an ASSISTANT role.
|
||||
calls = context.svc.append_message.call_args_list
|
||||
assert calls[-1].kwargs.get("role") == MessageRole.ASSISTANT, (
|
||||
f"Last persisted message role is not ASSISTANT. Calls: {calls}"
|
||||
)
|
||||
|
||||
|
||||
@then("the tell command exits with code 1")
|
||||
def step_tell_exits_code_1(context: Context) -> None:
|
||||
assert context.result.exit_code == 1, (
|
||||
f"Expected exit code 1, got {context.result.exit_code}.\n"
|
||||
f"Output:\n{context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the error output mentions actor configuration")
|
||||
def step_error_mentions_actor(context: Context) -> None:
|
||||
output = context.result.output.lower()
|
||||
assert "actor" in output, (
|
||||
f"Expected 'actor' in error output.\nGot:\n{context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the tell command uses the override actor "{actor}"')
|
||||
def step_tell_uses_override_actor(context: Context, actor: str) -> None:
|
||||
invoked = getattr(context, "invoked_actor_name", None)
|
||||
assert invoked is not None, "Actor name was never captured from _resolve_llm"
|
||||
assert invoked == actor, f"Expected actor '{actor}' to be used, but got '{invoked}'"
|
||||
|
||||
|
||||
@then("the tell output is valid JSON with a data envelope")
|
||||
def step_tell_output_is_valid_json(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}.\n"
|
||||
f"Output:\n{context.result.output}"
|
||||
)
|
||||
try:
|
||||
raw = json.loads(context.result.output.strip())
|
||||
except json.JSONDecodeError as exc:
|
||||
raise AssertionError(
|
||||
f"Output is not valid JSON: {exc}\nOutput:\n{context.result.output}"
|
||||
) from exc
|
||||
# format_output wraps data in a spec-required envelope
|
||||
context.json_output = raw
|
||||
context.json_data = raw.get("data", {})
|
||||
assert isinstance(context.json_data, dict), (
|
||||
f"Expected data envelope to be a dict, got {type(context.json_data)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the data section contains the session_id")
|
||||
def step_json_data_contains_session_id(context: Context) -> None:
|
||||
assert context.json_data.get("session_id") == _ULID, (
|
||||
f"Expected session_id={_ULID}, got {context.json_data.get('session_id')}"
|
||||
)
|
||||
|
||||
|
||||
@then("the data section contains a usage object with expected keys")
|
||||
def step_json_data_contains_usage_object(context: Context) -> None:
|
||||
usage = context.json_data.get("usage")
|
||||
assert isinstance(usage, dict), f"Expected usage to be a dict, got {type(usage)}"
|
||||
expected_keys = {
|
||||
"input_tokens",
|
||||
"output_tokens",
|
||||
"cost_usd",
|
||||
"duration_ms",
|
||||
"tool_calls",
|
||||
}
|
||||
missing = expected_keys - set(usage.keys())
|
||||
assert not missing, f"Usage object missing expected keys: {missing}\nUsage: {usage}"
|
||||
|
||||
|
||||
@then("the output contains a Usage panel")
|
||||
def step_output_contains_usage_panel(context: Context) -> None:
|
||||
output = context.result.output
|
||||
# Rich Usage panel is rendered with title="Usage"
|
||||
assert "Usage" in output, f"Expected Usage panel in output.\nOutput:\n{output}"
|
||||
assert "Input tokens" in output, (
|
||||
f"Expected input tokens in Usage panel.\nOutput:\n{output}"
|
||||
)
|
||||
assert "Output tokens" in output, (
|
||||
f"Expected output tokens in Usage panel.\nOutput:\n{output}"
|
||||
)
|
||||
@@ -0,0 +1,473 @@
|
||||
"""Step definitions for session_workflow.py coverage boost tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.session_caller import (
|
||||
LangChainSessionCaller,
|
||||
_MinimalStubLLM,
|
||||
estimate_cost,
|
||||
extract_content,
|
||||
extract_token_usage,
|
||||
history_to_langchain_messages,
|
||||
)
|
||||
from cleveragents.application.services.session_workflow import (
|
||||
SessionWorkflow,
|
||||
)
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
SessionMessage,
|
||||
)
|
||||
from cleveragents.tool.actor_runtime import LLMResponse
|
||||
|
||||
|
||||
@given("the session workflow coverage environment is set up")
|
||||
def step_sw_coverage_setup(context: Context) -> None:
|
||||
"""No special setup needed."""
|
||||
pass
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _extract_content
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given('coverage boost a mock response with content "hello world"')
|
||||
def step_cb_mock_content(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.content = "hello world"
|
||||
resp.text = "ignored"
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given('coverage boost a mock response with text "from text attr" and no content')
|
||||
def step_cb_text_fallback(context: Context) -> None:
|
||||
class _TextOnly:
|
||||
def __init__(self) -> None:
|
||||
self.text = "from text attr"
|
||||
|
||||
context.cb_mock_response = _TextOnly()
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a mock response with list content containing text dicts and plain strings"
|
||||
)
|
||||
def step_cb_list_content(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.content = [
|
||||
{"text": "hello"},
|
||||
"world",
|
||||
{"content": "test"},
|
||||
]
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given("coverage boost a mock response with no content or text attribute")
|
||||
def step_cb_no_attrs(context: Context) -> None:
|
||||
class _Unknown:
|
||||
pass
|
||||
|
||||
context.cb_mock_response = _Unknown()
|
||||
|
||||
|
||||
@when("coverage boost _extract_content is called")
|
||||
def step_cb_call_extract_content(context: Context) -> None:
|
||||
context.cb_extract_result = extract_content(context.cb_mock_response)
|
||||
|
||||
|
||||
@then('coverage boost the extracted result should be "{expected}"')
|
||||
def step_cb_extract_result_is(context: Context, expected: str) -> None:
|
||||
assert context.cb_extract_result == expected, (
|
||||
f"Expected {expected!r}, got {context.cb_extract_result!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("coverage boost the result should contain the concatenated texts")
|
||||
def step_cb_result_concat(context: Context) -> None:
|
||||
assert "hello" in context.cb_extract_result
|
||||
assert "world" in context.cb_extract_result
|
||||
assert "test" in context.cb_extract_result
|
||||
|
||||
|
||||
@then("coverage boost the result should be a string")
|
||||
def step_cb_result_str(context: Context) -> None:
|
||||
assert isinstance(context.cb_extract_result, str)
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _extract_token_usage
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a mock response with response_metadata usage input_tokens=100 output_tokens=50"
|
||||
)
|
||||
def step_cb_usage_metadata(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.response_metadata = {"usage": {"input_tokens": 100, "output_tokens": 50}}
|
||||
resp.usage_metadata = {}
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a mock response with response_metadata token_usage input_tokens=200 output_tokens=100"
|
||||
)
|
||||
def step_cb_token_usage_metadata(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.response_metadata = {
|
||||
"token_usage": {"input_tokens": 200, "output_tokens": 100}
|
||||
}
|
||||
resp.usage_metadata = {}
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a mock response with response_metadata usage prompt_tokens=300 completion_tokens=150"
|
||||
)
|
||||
def step_cb_prompt_completion(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.response_metadata = {"usage": {"prompt_tokens": 300, "completion_tokens": 150}}
|
||||
resp.usage_metadata = {}
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a mock response with usage_metadata input_tokens=400 output_tokens=200"
|
||||
)
|
||||
def step_cb_usage_meta_attr(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.response_metadata = {}
|
||||
resp.usage_metadata = {"input_tokens": 400, "output_tokens": 200}
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@given("coverage boost a mock response with no usage metadata")
|
||||
def step_cb_no_metadata(context: Context) -> None:
|
||||
resp = MagicMock()
|
||||
resp.response_metadata = {}
|
||||
resp.usage_metadata = {}
|
||||
context.cb_mock_response = resp
|
||||
|
||||
|
||||
@when("coverage boost _extract_token_usage is called")
|
||||
def step_cb_call_extract_token_usage(context: Context) -> None:
|
||||
context.cb_token_input, context.cb_token_output = extract_token_usage(
|
||||
context.cb_mock_response
|
||||
)
|
||||
|
||||
|
||||
@then(
|
||||
"coverage boost input tokens should be {input_tokens:d} and output tokens should be {output_tokens:d}"
|
||||
)
|
||||
def step_cb_assert_token_counts(
|
||||
context: Context, input_tokens: int, output_tokens: int
|
||||
) -> None:
|
||||
assert context.cb_token_input == input_tokens, (
|
||||
f"Expected {input_tokens} input tokens, got {context.cb_token_input}"
|
||||
)
|
||||
assert context.cb_token_output == output_tokens, (
|
||||
f"Expected {output_tokens} output tokens, got {context.cb_token_output}"
|
||||
)
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _estimate_cost
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost input tokens {input_tokens:d} and output tokens {output_tokens:d}"
|
||||
)
|
||||
def step_cb_given_tokens(
|
||||
context: Context, input_tokens: int, output_tokens: int
|
||||
) -> None:
|
||||
context.cb_input_tokens = input_tokens
|
||||
context.cb_output_tokens = output_tokens
|
||||
|
||||
|
||||
@when("coverage boost _estimate_cost is called")
|
||||
def step_cb_call_estimate_cost(context: Context) -> None:
|
||||
context.cb_estimated_cost = estimate_cost(
|
||||
context.cb_input_tokens, context.cb_output_tokens
|
||||
)
|
||||
|
||||
|
||||
@then("coverage boost the estimated cost should be positive")
|
||||
def step_cb_cost_positive(context: Context) -> None:
|
||||
assert context.cb_estimated_cost > 0.0, (
|
||||
f"Expected positive cost, got {context.cb_estimated_cost}"
|
||||
)
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _history_to_langchain_messages
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given("coverage boost session messages with roles SYSTEM, USER, ASSISTANT, and TOOL")
|
||||
def step_cb_messages_all_roles(context: Context) -> None:
|
||||
context.cb_history_messages = [
|
||||
SessionMessage(
|
||||
message_id="01KJ1N8YDN05N29P5RZV4SPDVZ",
|
||||
role=MessageRole.SYSTEM,
|
||||
content="You are helpful.",
|
||||
sequence=1,
|
||||
timestamp="2026-01-01T00:00:00Z",
|
||||
),
|
||||
SessionMessage(
|
||||
message_id="01KJ1N8YDN05N29P5RZV4SPDW0",
|
||||
role=MessageRole.USER,
|
||||
content="Hello",
|
||||
sequence=2,
|
||||
timestamp="2026-01-01T00:00:00Z",
|
||||
),
|
||||
SessionMessage(
|
||||
message_id="01KJ1N8YDN05N29P5RZV4SPDW1",
|
||||
role=MessageRole.ASSISTANT,
|
||||
content="Hi!",
|
||||
sequence=3,
|
||||
timestamp="2026-01-01T00:00:00Z",
|
||||
),
|
||||
SessionMessage(
|
||||
message_id="01KJ1N8YDN05N29P5RZV4SPDW2",
|
||||
role=MessageRole.TOOL,
|
||||
content="result",
|
||||
sequence=4,
|
||||
timestamp="2026-01-01T00:00:00Z",
|
||||
tool_call_id="tc1",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@when("coverage boost _history_to_langchain_messages is called")
|
||||
def step_cb_call_history_to_lc(context: Context) -> None:
|
||||
context.cb_lc_messages = history_to_langchain_messages(context.cb_history_messages)
|
||||
|
||||
|
||||
@then(
|
||||
"coverage boost the result should contain SystemMessage, HumanMessage, AIMessage, and ToolMessage"
|
||||
)
|
||||
def step_cb_assert_lc_types(context: Context) -> None:
|
||||
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
|
||||
from langchain_core.messages.tool import ToolMessage
|
||||
|
||||
types = [type(m) for m in context.cb_lc_messages]
|
||||
assert SystemMessage in types, f"Missing SystemMessage in {types}"
|
||||
assert HumanMessage in types, f"Missing HumanMessage in {types}"
|
||||
assert AIMessage in types, f"Missing AIMessage in {types}"
|
||||
assert ToolMessage in types, f"Missing ToolMessage in {types}"
|
||||
|
||||
|
||||
@given("coverage boost a session message with an unknown role")
|
||||
def step_cb_message_unknown_role(context: Context) -> None:
|
||||
msg = MagicMock(spec=SessionMessage)
|
||||
msg.role = "NOT_A_REAL_ROLE"
|
||||
msg.content = "test content"
|
||||
msg.tool_call_id = None
|
||||
context.cb_history_messages = [msg]
|
||||
|
||||
|
||||
@then("coverage boost the result should contain a HumanMessage")
|
||||
def step_cb_result_contains_human(context: Context) -> None:
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
assert len(context.cb_lc_messages) > 0, "Expected at least one message"
|
||||
assert isinstance(context.cb_lc_messages[0], HumanMessage), (
|
||||
f"Expected HumanMessage, got {type(context.cb_lc_messages[0])}"
|
||||
)
|
||||
|
||||
|
||||
@given("coverage boost an empty list of session messages")
|
||||
def step_cb_empty_history(context: Context) -> None:
|
||||
context.cb_history_messages = []
|
||||
|
||||
|
||||
@then("coverage boost the result should be an empty list")
|
||||
def step_cb_empty_list(context: Context) -> None:
|
||||
assert context.cb_lc_messages == [], (
|
||||
f"Expected empty list, got {context.cb_lc_messages}"
|
||||
)
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# LangChainSessionCaller.invoke() tool_results
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given("coverage boost a LangChainSessionCaller with a stub LLM and empty history")
|
||||
def step_cb_caller_with_stub(context: Context) -> None:
|
||||
stub_llm = MagicMock()
|
||||
stub_llm.invoke.return_value = MagicMock(
|
||||
content="response text",
|
||||
tool_calls=[],
|
||||
response_metadata={},
|
||||
usage_metadata={},
|
||||
)
|
||||
context.cb_caller = LangChainSessionCaller(llm=stub_llm, history=[])
|
||||
context.cb_stub_llm = stub_llm
|
||||
|
||||
|
||||
@when(
|
||||
"coverage boost invoke is called with tool_results containing one success and one failure"
|
||||
)
|
||||
def step_cb_invoke_tool_results(context: Context) -> None:
|
||||
context.cb_caller.invoke(prompt="test prompt", tool_schemas=[])
|
||||
context.cb_tool_result_response = context.cb_caller.invoke(
|
||||
prompt="test prompt",
|
||||
tool_schemas=[],
|
||||
tool_results=[
|
||||
{
|
||||
"success": True,
|
||||
"output": {"result": "ok"},
|
||||
"call_id": "c1",
|
||||
"tool_name": "tool1",
|
||||
},
|
||||
{
|
||||
"success": False,
|
||||
"error": "failed",
|
||||
"call_id": "c2",
|
||||
"tool_name": "tool2",
|
||||
},
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@then("coverage boost the accumulated messages should include tool result messages")
|
||||
def step_cb_accumulated_has_tool_results(context: Context) -> None:
|
||||
from langchain_core.messages.tool import ToolMessage
|
||||
|
||||
tool_msgs = [
|
||||
m for m in context.cb_caller._accumulated if isinstance(m, ToolMessage)
|
||||
]
|
||||
assert len(tool_msgs) >= 2, f"Expected >= 2 ToolMessages, got {len(tool_msgs)}"
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# LangChainSessionCaller.invoke() with tool_calls in response
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given(
|
||||
"coverage boost a LangChainSessionCaller with a stub LLM that returns tool calls"
|
||||
)
|
||||
def step_cb_caller_tool_calls(context: Context) -> None:
|
||||
stub_llm = MagicMock()
|
||||
resp = MagicMock(
|
||||
content="response with tool calls",
|
||||
tool_calls=[
|
||||
{"name": "test_tool", "args": {"arg1": "val1"}, "id": "call_123"},
|
||||
],
|
||||
response_metadata={},
|
||||
usage_metadata={},
|
||||
)
|
||||
stub_llm.invoke.return_value = resp
|
||||
context.cb_caller = LangChainSessionCaller(llm=stub_llm, history=[])
|
||||
|
||||
|
||||
@when("coverage boost invoke is called for the first time")
|
||||
def step_cb_invoke_first_time(context: Context) -> None:
|
||||
context.cb_first_invoke_result = context.cb_caller.invoke(
|
||||
prompt="test prompt", tool_schemas=[]
|
||||
)
|
||||
|
||||
|
||||
@then("coverage boost the LLMResponse should contain the extracted tool calls")
|
||||
def step_cb_llm_response_has_tool_calls(context: Context) -> None:
|
||||
assert isinstance(context.cb_first_invoke_result, LLMResponse)
|
||||
assert len(context.cb_first_invoke_result.tool_calls) >= 1, (
|
||||
f"Expected >= 1 tool calls, got {len(context.cb_first_invoke_result.tool_calls)}"
|
||||
)
|
||||
tc = context.cb_first_invoke_result.tool_calls[0]
|
||||
assert tc.name == "test_tool"
|
||||
assert tc.arguments == {"arg1": "val1"}
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _build_lc_messages_from_history (SessionWorkflow method)
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given("coverage boost a SessionWorkflow with a stub service and no registry")
|
||||
def step_cb_workflow_with_stub(context: Context) -> None:
|
||||
svc = MagicMock()
|
||||
svc.get.return_value = MagicMock(actor_name="openai/gpt-4")
|
||||
svc.get_messages.return_value = []
|
||||
svc.append_message.return_value = MagicMock(spec=SessionMessage)
|
||||
svc.update_token_usage.return_value = None
|
||||
context.cb_workflow = SessionWorkflow(session_service=svc, provider_registry=None)
|
||||
|
||||
|
||||
@given("coverage boost session history without a system message")
|
||||
def step_cb_history_no_system(context: Context) -> None:
|
||||
context.cb_no_system_history = [
|
||||
SessionMessage(
|
||||
message_id="01KJ1N8YDN05N29P5RZV4SPDW0",
|
||||
role=MessageRole.USER,
|
||||
content="Hello",
|
||||
sequence=1,
|
||||
timestamp="2026-01-01T00:00:00Z",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@when("coverage boost _build_lc_messages_from_history is called with a prompt")
|
||||
def step_cb_call_build_lc(context: Context) -> None:
|
||||
context.cb_built_messages = context.cb_workflow._build_lc_messages_from_history(
|
||||
context.cb_no_system_history, "test prompt"
|
||||
)
|
||||
|
||||
|
||||
@then(
|
||||
"coverage boost the first message should be a SystemMessage with the session system prompt"
|
||||
)
|
||||
def step_cb_first_is_system(context: Context) -> None:
|
||||
from langchain_core.messages import SystemMessage
|
||||
|
||||
assert len(context.cb_built_messages) > 0, "Expected at least one message"
|
||||
first = context.cb_built_messages[0]
|
||||
assert isinstance(first, SystemMessage), (
|
||||
f"Expected SystemMessage, got {type(first)}"
|
||||
)
|
||||
assert "CleverAgents orchestrator" in first.content, (
|
||||
f"Expected system prompt content, got {first.content}"
|
||||
)
|
||||
|
||||
|
||||
# ====================================================================
|
||||
# _MinimalStubLLM
|
||||
# ====================================================================
|
||||
|
||||
|
||||
@given("coverage boost a _MinimalStubLLM instance")
|
||||
def step_cb_minimal_stub(context: Context) -> None:
|
||||
context.cb_stub = _MinimalStubLLM()
|
||||
|
||||
|
||||
@when("coverage boost invoke on the stub is called")
|
||||
def step_cb_stub_invoke(context: Context) -> None:
|
||||
context.cb_stub_response = context.cb_stub.invoke([])
|
||||
|
||||
|
||||
@then('coverage boost the stub response content should be "(no LLM configured)"')
|
||||
def step_cb_stub_content(context: Context) -> None:
|
||||
assert context.cb_stub_response.content == "(no LLM configured)"
|
||||
|
||||
|
||||
@then("coverage boost the stub response should have empty tool_calls")
|
||||
def step_cb_stub_empty_tool_calls(context: Context) -> None:
|
||||
assert context.cb_stub_response.tool_calls == []
|
||||
|
||||
|
||||
@when("coverage boost stream on the stub is called")
|
||||
def step_cb_stub_stream(context: Context) -> None:
|
||||
context.cb_stub_stream_chunks = list(context.cb_stub.stream([]))
|
||||
|
||||
|
||||
@then('coverage boost it should yield a chunk with content "(no LLM configured)"')
|
||||
def step_cb_stub_chunk_content(context: Context) -> None:
|
||||
assert len(context.cb_stub_stream_chunks) >= 1, "Expected at least one chunk"
|
||||
assert context.cb_stub_stream_chunks[0].content == "(no LLM configured)"
|
||||
@@ -15,18 +15,21 @@ from unittest.mock import MagicMock, patch
|
||||
from behave import given, then, when
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.services.session_workflow import TellResult
|
||||
from cleveragents.cli.commands.session import app as session_app
|
||||
from cleveragents.domain.models.core.session import SessionService
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
_ULID = "01KJ1N8YDN05N29P5RZV4SPDVZ"
|
||||
_STUB_RESPONSE = "Acknowledged: Hello world"
|
||||
|
||||
|
||||
def _mock_service() -> MagicMock:
|
||||
"""Create a MagicMock that passes isinstance checks for SessionService."""
|
||||
svc = MagicMock(spec=SessionService)
|
||||
svc.append_message.return_value = None
|
||||
svc.get_messages.return_value = []
|
||||
return svc
|
||||
|
||||
|
||||
@@ -37,6 +40,33 @@ def _patch_service(context: object, svc: MagicMock) -> None:
|
||||
context._cleanup_handlers.append(patcher.stop) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def _mock_workflow(response: str = _STUB_RESPONSE) -> MagicMock:
|
||||
"""Create a mock SessionWorkflow with canned tell and tell_stream returns."""
|
||||
wf = MagicMock()
|
||||
wf.tell.return_value = TellResult(
|
||||
session_id=_ULID,
|
||||
user_message="",
|
||||
assistant_message=response,
|
||||
input_tokens=5,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
wf.tell_stream.return_value = iter([response])
|
||||
return wf
|
||||
|
||||
|
||||
def _patch_workflow(context: object, wf: MagicMock) -> None:
|
||||
"""Patch _build_session_workflow and register cleanup."""
|
||||
patcher = patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=wf,
|
||||
)
|
||||
patcher.start()
|
||||
context._cleanup_handlers.append(patcher.stop) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -45,8 +75,11 @@ def _patch_service(context: object, svc: MagicMock) -> None:
|
||||
@given("a session tell stream redaction mock service")
|
||||
def step_setup_mock_service(context: object) -> None:
|
||||
svc = _mock_service()
|
||||
wf = _mock_workflow()
|
||||
_patch_service(context, svc)
|
||||
_patch_workflow(context, wf)
|
||||
context.mock_svc = svc # type: ignore[attr-defined]
|
||||
context.mock_wf = wf # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -117,10 +150,15 @@ def step_stdout_write_not_called(context: object) -> None:
|
||||
return original_write(s)
|
||||
|
||||
svc = _mock_service()
|
||||
wf = _mock_workflow()
|
||||
sensitive_prompt = "my-api-key=sk-secret1234567890abcdef"
|
||||
|
||||
with (
|
||||
mock_patch("cleveragents.cli.commands.session._service", svc),
|
||||
mock_patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=wf,
|
||||
),
|
||||
mock_patch("sys.stdout.write", side_effect=tracking_write),
|
||||
):
|
||||
runner.invoke(
|
||||
@@ -159,8 +197,8 @@ def step_output_via_console(context: object) -> None:
|
||||
that it was NOT split into individual characters in the captured output.
|
||||
"""
|
||||
result = context.result # type: ignore[attr-defined]
|
||||
# The assistant content for "Hello world" is "Acknowledged: Hello world"
|
||||
expected = "Acknowledged: Hello world"
|
||||
# After real LLM wiring, the stub workflow returns _STUB_RESPONSE.
|
||||
expected = _STUB_RESPONSE
|
||||
assert expected in result.output, (
|
||||
f"Expected '{expected}' in streaming output.\nGot:\n{result.output}"
|
||||
)
|
||||
|
||||
@@ -47,6 +47,7 @@ dependencies = [
|
||||
"tomlkit>=0.13.0", # TOML writing with comment preservation for config CLI
|
||||
"tenacity>=8.2.0", # Retry framework for service layer resilience
|
||||
"aiohttp>=3.13.4", # CVE-2026-34515 mitigation: open redirect vulnerability
|
||||
"pyyaml>=6.0.3", # Security: address known YAML parsing vulnerabilities
|
||||
"a2a-sdk>=0.3.0,<1.0.0", # A2A Python SDK — required transport for local (stdio) and server (HTTP) modes (ADR-047); pinned <1.0.0 (removed legacy A2AClient)
|
||||
]
|
||||
|
||||
|
||||
@@ -4,40 +4,6 @@ Library Collections
|
||||
Library Process
|
||||
|
||||
*** Test Cases ***
|
||||
V2 Actor Config Produces Provider And Graph Descriptor
|
||||
${config}= Set Variable ${OUTPUT DIR}/v2_actor.yaml
|
||||
${content}= Catenate SEPARATOR=\n
|
||||
... cleveragents:
|
||||
... ${SPACE*2}default_router: main_router
|
||||
... agents:
|
||||
... ${SPACE*2}paper_writer:
|
||||
... ${SPACE*4}type: llm
|
||||
... ${SPACE*4}config:
|
||||
... ${SPACE*6}provider: openai
|
||||
... ${SPACE*6}model: gpt-4
|
||||
... ${SPACE*6}unsafe: true
|
||||
... ${SPACE*6}options:
|
||||
... ${SPACE*8}temperature: 0.5
|
||||
... routes:
|
||||
... ${SPACE*2}main_router:
|
||||
... ${SPACE*4}type: stream
|
||||
... ${SPACE*4}operators:
|
||||
... ${SPACE*6}- type: map
|
||||
... ${SPACE*8}params:
|
||||
... ${SPACE*10}agent: paper_writer
|
||||
... ${SPACE*4}publications:
|
||||
... ${SPACE*6}- __output__
|
||||
Create File ${config} ${content}
|
||||
${result}= Run Process ${PYTHON} robot/helper_actor_config.py ${config}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${payload}= Evaluate __import__('json').loads('''${result.stdout.strip()}''')
|
||||
Should Be Equal ${payload['provider']} openai
|
||||
Should Be Equal ${payload['model']} gpt-4
|
||||
Should Be True ${payload['unsafe']}
|
||||
List Should Contain Value ${payload['graph_keys']} agents
|
||||
List Should Contain Value ${payload['graph_keys']} routes
|
||||
Should Be Equal As Numbers ${payload['options']['temperature']} 0.5
|
||||
|
||||
Missing Config File Exits With Non-Zero Code And Stderr Message
|
||||
[Tags] tdd_issue tdd_issue_4202 tdd_expected_fail
|
||||
${missing}= Set Variable ${OUTPUT DIR}/does_not_exist_actor.yaml
|
||||
|
||||
@@ -386,11 +386,24 @@ M6 E2E Hierarchical Decomposition Via Plan Tree
|
||||
# P0-4: Hard assertion — tree command must succeed.
|
||||
Should Be Equal As Integers ${tree.rc} 0 msg=plan tree failed (rc=${tree.rc}): ${tree.stderr}
|
||||
Should Not Be Empty ${tree.stdout} Plan tree output should not be empty
|
||||
# P0-4: Hard assertion — at least one decision node must exist after execution.
|
||||
${decision_count}= Evaluate $tree.stdout.count('"decision_id"')
|
||||
Log Decision tree contains ${decision_count} decision node(s)
|
||||
Should Be True ${decision_count} >= 1
|
||||
... Plan tree should contain at least one decision node after execution (found ${decision_count})
|
||||
# P0-4: Hard assertion — tree output must contain the spec-required envelope.
|
||||
# The new envelope format wraps the tree in a command envelope with
|
||||
# "command", "status", "exit_code", "data", "timing", "messages" keys.
|
||||
# The "data" field contains "plan_id", "tree", "summary", "child_plans",
|
||||
# and "decision_ids" (a mapping of human-readable keys to decision ULIDs).
|
||||
${has_command_key}= Evaluate '"command"' in $tree.stdout
|
||||
Should Be True ${has_command_key}
|
||||
... Plan tree JSON output should contain spec-required envelope "command" key
|
||||
${has_data_key}= Evaluate '"data"' in $tree.stdout
|
||||
Should Be True ${has_data_key}
|
||||
... Plan tree JSON output should contain spec-required envelope "data" key
|
||||
${has_plan_id_key}= Evaluate '"plan_id"' in $tree.stdout
|
||||
Should Be True ${has_plan_id_key}
|
||||
... Plan tree JSON output should contain "plan_id" in envelope data
|
||||
# Check for decision_ids mapping (proves at least one decision was recorded)
|
||||
${has_decision_ids}= Evaluate '"decision_ids"' in $tree.stdout
|
||||
Should Be True ${has_decision_ids}
|
||||
... Plan tree should contain decision_ids mapping after execution
|
||||
# Check for hierarchical children (decomposition infrastructure indicator)
|
||||
${has_children_key}= Evaluate '"children"' in $tree.stdout
|
||||
IF ${has_children_key}
|
||||
|
||||
@@ -97,7 +97,7 @@ def list_operations() -> None:
|
||||
facade = A2aLocalFacade()
|
||||
ops = facade.list_operations()
|
||||
expected = {"session.create", "plan.create", "plan.execute", "context.get"}
|
||||
if expected.issubset(set(ops)) and len(ops) == 42:
|
||||
if expected.issubset(set(ops)) and len(ops) == 44:
|
||||
print("a2a-list-operations-ok")
|
||||
else:
|
||||
print(f"FAIL: ops={ops}", file=sys.stderr)
|
||||
|
||||
@@ -10,7 +10,7 @@ import sys
|
||||
import tempfile
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
# Ensure local source tree is importable
|
||||
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||
@@ -20,6 +20,7 @@ if _SRC not in sys.path:
|
||||
from typer.testing import CliRunner # noqa: E402
|
||||
from ulid import ULID # noqa: E402
|
||||
|
||||
from cleveragents.application.services.session_workflow import TellResult # noqa: E402
|
||||
from cleveragents.cli.commands import session as session_mod # noqa: E402
|
||||
from cleveragents.cli.commands.session import app as session_app # noqa: E402
|
||||
from cleveragents.domain.models.core.session import ( # noqa: E402
|
||||
@@ -284,13 +285,27 @@ def import_rich_panels() -> None:
|
||||
def tell_message() -> None:
|
||||
sid = str(ULID())
|
||||
svc = _setup_service()
|
||||
svc.get.return_value = _mock_session(session_id=sid)
|
||||
svc.append_message.side_effect = [
|
||||
_mock_message(MessageRole.USER, "Hello", 0),
|
||||
_mock_message(MessageRole.ASSISTANT, "Acknowledged: Hello", 1),
|
||||
]
|
||||
# Session must have actor_name so tell can proceed without --actor flag.
|
||||
svc.get.return_value = _mock_session(session_id=sid, actor_name="openai/gpt-4")
|
||||
svc.get_messages.return_value = []
|
||||
# Build a mock workflow that returns a canned TellResult.
|
||||
mock_wf = MagicMock()
|
||||
mock_wf.tell.return_value = TellResult(
|
||||
session_id=sid,
|
||||
user_message="Hello",
|
||||
assistant_message="Acknowledged: Hello",
|
||||
input_tokens=5,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
try:
|
||||
result = runner.invoke(session_app, ["tell", "--session", sid, "Hello"])
|
||||
with patch(
|
||||
"cleveragents.cli.commands.session._build_session_workflow",
|
||||
return_value=mock_wf,
|
||||
):
|
||||
result = runner.invoke(session_app, ["tell", "--session", sid, "Hello"])
|
||||
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
|
||||
assert "Acknowledged" in result.output
|
||||
print("session-cli-tell-message-ok")
|
||||
|
||||
@@ -0,0 +1,213 @@
|
||||
"""Helper script for session_tell_llm.robot integration tests.
|
||||
|
||||
Each subcommand is a self-contained check that prints a sentinel on success.
|
||||
Tests verify SessionWorkflow.tell() / tell_stream() behaviour with a stub LLM.
|
||||
|
||||
No real API keys or network access required — all LLM calls are intercepted
|
||||
by a monkey-patched ``_resolve_llm`` that returns a minimal stub.
|
||||
"""
|
||||
|
||||
# ruff: noqa: E402, I001
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# Ensure local source tree is importable
|
||||
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||
if _SRC not in sys.path:
|
||||
sys.path.insert(0, _SRC)
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from cleveragents.application.services.session_service import (
|
||||
PersistentSessionService,
|
||||
)
|
||||
from cleveragents.application.services.session_workflow import (
|
||||
SessionWorkflow,
|
||||
)
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
SessionActorNotConfiguredError,
|
||||
)
|
||||
from cleveragents.infrastructure.database.models import Base
|
||||
from cleveragents.infrastructure.database.repositories import (
|
||||
SessionMessageRepository,
|
||||
SessionRepository,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stub LLM — returns deterministic responses without LLM API calls
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_STUB_TEXT = "This is a real response from the stub LLM actor."
|
||||
|
||||
|
||||
class _StubLLM:
|
||||
"""Minimal LLM stub that behaves like a LangChain chat model."""
|
||||
|
||||
def invoke(self, messages: Any, **kwargs: Any) -> Any:
|
||||
class _Resp:
|
||||
content = _STUB_TEXT
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.tool_calls: list[Any] = []
|
||||
self.response_metadata: dict[str, Any] = {
|
||||
"usage": {"input_tokens": 10, "output_tokens": 20}
|
||||
}
|
||||
|
||||
return _Resp()
|
||||
|
||||
def stream(self, messages: Any, **kwargs: Any) -> Iterator[Any]:
|
||||
for word in _STUB_TEXT.split():
|
||||
|
||||
class _Chunk:
|
||||
def __init__(self, t: str) -> None:
|
||||
self.content = t + " "
|
||||
|
||||
yield _Chunk(word)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixture helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_db_session_factory() -> Any:
|
||||
"""Create an in-memory SQLite engine and return a SQLAlchemy session factory."""
|
||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
Base.metadata.create_all(engine)
|
||||
return sessionmaker(bind=engine, expire_on_commit=False)
|
||||
|
||||
|
||||
def _make_workflow(
|
||||
actor_name: str | None = "openai/gpt-4",
|
||||
llm_factory: Any = None,
|
||||
) -> tuple[SessionWorkflow, Any]:
|
||||
"""Build a real SessionWorkflow backed by an in-memory SQLite database."""
|
||||
db_factory = _make_db_session_factory()
|
||||
db_session = db_factory()
|
||||
|
||||
def get_db() -> Any:
|
||||
return db_session
|
||||
|
||||
session_repo = SessionRepository(get_db)
|
||||
message_repo = SessionMessageRepository(get_db)
|
||||
svc = PersistentSessionService(session_repo=session_repo, message_repo=message_repo)
|
||||
|
||||
# Create a real session in the DB
|
||||
session = svc.create(actor_name=actor_name)
|
||||
|
||||
factory = llm_factory if llm_factory is not None else lambda _actor: _StubLLM()
|
||||
wf = SessionWorkflow(session_service=svc, llm_factory=factory)
|
||||
# No monkey-patching needed — llm_factory provides clean DI (addresses M8)
|
||||
|
||||
return wf, session
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Subcommands
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def tell_persists() -> None:
|
||||
"""Verify tell() returns real LLM response and persists messages."""
|
||||
wf, session = _make_workflow(actor_name="openai/gpt-4")
|
||||
result = wf.tell(
|
||||
session_id=session.session_id,
|
||||
prompt="What can you help me with?",
|
||||
)
|
||||
assert result.assistant_message == _STUB_TEXT, (
|
||||
f"Expected stub response, got: {result.assistant_message!r}"
|
||||
)
|
||||
assert result.input_tokens > 0, "input_tokens should be > 0"
|
||||
assert result.output_tokens > 0, "output_tokens should be > 0"
|
||||
# Verify messages were persisted via the service
|
||||
messages = wf._session_service.get_messages(session.session_id)
|
||||
roles = [m.role for m in messages]
|
||||
assert MessageRole.USER in roles, "User message not persisted"
|
||||
assert MessageRole.ASSISTANT in roles, "Assistant message not persisted"
|
||||
print("session-tell-persists-ok")
|
||||
|
||||
|
||||
def tell_no_actor() -> None:
|
||||
"""Verify tell() raises SessionActorNotConfiguredError when no actor set."""
|
||||
wf, session = _make_workflow(actor_name=None)
|
||||
try:
|
||||
wf.tell(session_id=session.session_id, prompt="Hello")
|
||||
print(
|
||||
"FAIL: expected SessionActorNotConfiguredError not raised",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
except SessionActorNotConfiguredError as exc:
|
||||
assert "actor" in str(exc).lower(), (
|
||||
f"Error message should mention 'actor', got: {exc}"
|
||||
)
|
||||
print("session-tell-no-actor-ok")
|
||||
|
||||
|
||||
def tell_stream() -> None:
|
||||
"""Verify tell_stream() yields tokens from the stub LLM."""
|
||||
wf, session = _make_workflow(actor_name="openai/gpt-4")
|
||||
tokens: list[str] = []
|
||||
for token in wf.tell_stream(
|
||||
session_id=session.session_id,
|
||||
prompt="Hello",
|
||||
):
|
||||
tokens.append(token)
|
||||
assert len(tokens) > 0, "Expected tokens to be yielded, got none"
|
||||
full_response = "".join(tokens).strip()
|
||||
assert len(full_response) > 0, f"Expected non-empty stream, got: {full_response!r}"
|
||||
print("session-tell-stream-ok")
|
||||
|
||||
|
||||
def tell_actor_override() -> None:
|
||||
"""Verify tell() uses actor_override instead of session actor."""
|
||||
captured_actor: list[str] = []
|
||||
|
||||
def _tracking_resolve(actor_name: str) -> Any:
|
||||
captured_actor.append(actor_name)
|
||||
return _StubLLM()
|
||||
|
||||
wf, session = _make_workflow(
|
||||
actor_name="anthropic/claude-3-haiku",
|
||||
llm_factory=_tracking_resolve,
|
||||
)
|
||||
|
||||
result = wf.tell(
|
||||
session_id=session.session_id,
|
||||
prompt="Hello",
|
||||
actor_override="openai/gpt-4",
|
||||
)
|
||||
assert captured_actor, "Actor resolver was never called"
|
||||
assert captured_actor[0] == "openai/gpt-4", (
|
||||
f"Expected 'openai/gpt-4' actor, got {captured_actor[0]!r}"
|
||||
)
|
||||
assert result.assistant_message is not None
|
||||
print("session-tell-actor-override-ok")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
COMMANDS = {
|
||||
"tell-persists": tell_persists,
|
||||
"tell-no-actor": tell_no_actor,
|
||||
"tell-stream": tell_stream,
|
||||
"tell-actor-override": tell_actor_override,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in COMMANDS:
|
||||
print(
|
||||
f"Usage: {sys.argv[0]} <command>\nCommands: {', '.join(COMMANDS)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
COMMANDS[sys.argv[1]]()
|
||||
@@ -0,0 +1,56 @@
|
||||
*** Settings ***
|
||||
Documentation Integration tests for session tell real LLM actor invocation.
|
||||
...
|
||||
... Verifies that ``agents session tell`` calls ``SessionWorkflow.tell()``
|
||||
... (not the stub), persists the response, and handles the no-actor
|
||||
... error case. Uses a stub LLM actor so no real API keys are
|
||||
... required.
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment With Database Isolation
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Variables ***
|
||||
${HELPER} ${CURDIR}/helper_session_tell_llm.py
|
||||
|
||||
*** Test Cases ***
|
||||
Session Tell Workflow Returns Real Response
|
||||
[Documentation] Verify SessionWorkflow.tell() returns a real (stub) response
|
||||
... and persists user + assistant messages.
|
||||
[Tags] session_tell_llm tdd_issue tdd_issue_5784
|
||||
${result}= Run Process ${PYTHON} ${HELPER} tell-persists
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} session-tell-persists-ok
|
||||
|
||||
Session Tell No Actor Raises Clear Error
|
||||
[Documentation] Verify SessionWorkflow.tell() raises SessionActorNotConfiguredError
|
||||
... when no actor is configured and no --actor override is supplied.
|
||||
[Tags] session_tell_llm tdd_issue tdd_issue_5784
|
||||
${result}= Run Process ${PYTHON} ${HELPER} tell-no-actor
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} session-tell-no-actor-ok
|
||||
|
||||
Session Tell Stream Yields Tokens
|
||||
[Documentation] Verify SessionWorkflow.tell_stream() yields tokens from the LLM.
|
||||
[Tags] session_tell_llm tdd_issue tdd_issue_5784
|
||||
${result}= Run Process ${PYTHON} ${HELPER} tell-stream
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} session-tell-stream-ok
|
||||
|
||||
Session Tell Actor Override Works
|
||||
[Documentation] Verify that the actor_override parameter overrides the session actor.
|
||||
[Tags] session_tell_llm tdd_issue tdd_issue_5784
|
||||
${result}= Run Process ${PYTHON} ${HELPER} tell-actor-override
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} session-tell-actor-override-ok
|
||||
+115
-18
@@ -21,7 +21,7 @@ response so the facade never crashes due to missing wiring.
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import Any, cast
|
||||
|
||||
import structlog
|
||||
from ulid import ULID
|
||||
@@ -30,22 +30,27 @@ from cleveragents.a2a.errors import (
|
||||
A2aOperationNotFoundError,
|
||||
map_domain_error,
|
||||
)
|
||||
from cleveragents.a2a.events import A2aEventQueue
|
||||
from cleveragents.a2a.models import (
|
||||
A2aErrorDetail,
|
||||
A2aRequest,
|
||||
A2aResponse,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.a2a.events import A2aEventQueue
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.application.services.resource_registry_service import (
|
||||
ResourceRegistryService,
|
||||
)
|
||||
from cleveragents.domain.models.core.session import SessionService
|
||||
from cleveragents.tool.registry import ToolRegistry
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.application.services.resource_registry_service import (
|
||||
ResourceRegistryService,
|
||||
)
|
||||
from cleveragents.application.services.session_workflow import SessionWorkflow
|
||||
from cleveragents.core.exceptions import DatabaseError
|
||||
from cleveragents.domain.models.core.session import (
|
||||
SessionActorNotConfiguredError,
|
||||
SessionNotFoundError,
|
||||
SessionService,
|
||||
)
|
||||
from cleveragents.providers.registry import ProviderRegistry
|
||||
from cleveragents.tool.registry import ToolRegistry
|
||||
|
||||
logger: structlog.stdlib.BoundLogger = structlog.get_logger(__name__)
|
||||
|
||||
@@ -53,6 +58,12 @@ logger: structlog.stdlib.BoundLogger = structlog.get_logger(__name__)
|
||||
# Supported operations
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Standard A2A protocol operations (message/send and message/stream).
|
||||
_STANDARD_A2A_OPERATIONS: list[str] = [
|
||||
"message/send",
|
||||
"message/stream",
|
||||
]
|
||||
|
||||
# Spec-aligned _cleveragents/ extension method names (ADR-047).
|
||||
_EXTENSION_OPERATIONS: list[str] = [
|
||||
# Plan lifecycle
|
||||
@@ -109,7 +120,9 @@ _LEGACY_OPERATIONS: list[str] = [
|
||||
"event.subscribe",
|
||||
]
|
||||
|
||||
_SUPPORTED_OPERATIONS: list[str] = _EXTENSION_OPERATIONS + _LEGACY_OPERATIONS
|
||||
_SUPPORTED_OPERATIONS: list[str] = (
|
||||
_STANDARD_A2A_OPERATIONS + _EXTENSION_OPERATIONS + _LEGACY_OPERATIONS
|
||||
)
|
||||
|
||||
|
||||
class A2aLocalFacade:
|
||||
@@ -134,25 +147,33 @@ class A2aLocalFacade:
|
||||
|
||||
@property
|
||||
def _session_service(self) -> SessionService | None:
|
||||
return self._services.get("session_service") # type: ignore[return-value]
|
||||
return cast(SessionService | None, self._services.get("session_service"))
|
||||
|
||||
@property
|
||||
def _provider_registry(self) -> ProviderRegistry | None:
|
||||
return cast(ProviderRegistry | None, self._services.get("provider_registry"))
|
||||
|
||||
@property
|
||||
def _session_workflow(self) -> SessionWorkflow | None:
|
||||
return cast(SessionWorkflow | None, self._services.get("session_workflow"))
|
||||
|
||||
@property
|
||||
def _plan_lifecycle_service(self) -> PlanLifecycleService | None:
|
||||
svc: object | None = self._services.get("plan_lifecycle_service")
|
||||
return svc # type: ignore[return-value]
|
||||
return cast(PlanLifecycleService | None, svc)
|
||||
|
||||
@property
|
||||
def _tool_registry(self) -> ToolRegistry | None:
|
||||
return self._services.get("tool_registry") # type: ignore[return-value]
|
||||
return cast(ToolRegistry | None, self._services.get("tool_registry"))
|
||||
|
||||
@property
|
||||
def _resource_registry_service(self) -> ResourceRegistryService | None:
|
||||
svc: object | None = self._services.get("resource_registry_service")
|
||||
return svc # type: ignore[return-value]
|
||||
return cast(ResourceRegistryService | None, svc)
|
||||
|
||||
@property
|
||||
def _event_queue(self) -> A2aEventQueue | None:
|
||||
return self._services.get("event_queue") # type: ignore[return-value]
|
||||
return cast(A2aEventQueue | None, self._services.get("event_queue"))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public API
|
||||
@@ -164,6 +185,10 @@ class A2aLocalFacade:
|
||||
Returns an :class:`A2aResponse` with ``result`` set on success or
|
||||
``error`` set when the operation fails. Domain exceptions
|
||||
are mapped to A2A error codes via :func:`map_domain_error`.
|
||||
|
||||
Handlers may re-raise domain exceptions (e.g.
|
||||
``SessionNotFoundError``) to signal that the caller should receive
|
||||
the original exception rather than an A2A error envelope.
|
||||
"""
|
||||
if not isinstance(request, A2aRequest):
|
||||
raise TypeError("request must be an A2aRequest instance")
|
||||
@@ -184,6 +209,11 @@ class A2aLocalFacade:
|
||||
)
|
||||
except A2aOperationNotFoundError:
|
||||
raise
|
||||
except (SessionNotFoundError, SessionActorNotConfiguredError, DatabaseError):
|
||||
# Let domain exceptions propagate — they are re-raised by
|
||||
# handlers that want the caller (e.g. CLI) to receive the
|
||||
# original exception type for proper error handling.
|
||||
raise
|
||||
except Exception as exc:
|
||||
elapsed = (time.monotonic() - start) * 1000.0
|
||||
code, message = map_domain_error(exc)
|
||||
@@ -242,6 +272,9 @@ class A2aLocalFacade:
|
||||
"""
|
||||
if self._handler_map is None:
|
||||
self._handler_map = {
|
||||
# --- Standard A2A operations ---
|
||||
"message/send": self._handle_message_send,
|
||||
"message/stream": self._handle_message_stream,
|
||||
# --- _cleveragents/ extension methods (spec-aligned) ---
|
||||
# Plan lifecycle
|
||||
"_cleveragents/plan/use": self._handle_plan_create,
|
||||
@@ -336,6 +369,70 @@ class A2aLocalFacade:
|
||||
self._cleanup_session_devcontainers(session_id)
|
||||
return {"status": "closed"}
|
||||
|
||||
def _get_or_build_session_workflow(self) -> SessionWorkflow:
|
||||
"""Return the registered ``SessionWorkflow`` or build a fresh one."""
|
||||
workflow = self._session_workflow
|
||||
if workflow is not None:
|
||||
return workflow
|
||||
|
||||
# Lazy-build using registered services.
|
||||
svc = self._session_service
|
||||
if svc is None:
|
||||
raise RuntimeError("Session service not available — create a session first")
|
||||
registry = self._provider_registry
|
||||
return SessionWorkflow(
|
||||
session_service=svc,
|
||||
provider_registry=registry,
|
||||
)
|
||||
|
||||
def _handle_message_send(self, params: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Handle A2A ``message/send`` — invoke orchestrator actor (non-streaming).
|
||||
|
||||
Required params:
|
||||
``session_id``: Target session ULID.
|
||||
``message``: User message text.
|
||||
|
||||
Optional params:
|
||||
``actor``: Actor override (``namespace/name``).
|
||||
"""
|
||||
session_id = params.get("session_id", "")
|
||||
message = params.get("message", "")
|
||||
actor_override: str | None = params.get("actor") or None
|
||||
|
||||
if not session_id:
|
||||
raise ValueError("session_id is required")
|
||||
if not message:
|
||||
raise ValueError("message is required")
|
||||
|
||||
workflow = self._get_or_build_session_workflow()
|
||||
result = workflow.tell(
|
||||
session_id=session_id,
|
||||
prompt=message,
|
||||
actor_override=actor_override,
|
||||
)
|
||||
|
||||
return {
|
||||
"session_id": result.session_id,
|
||||
"assistant_message": result.assistant_message,
|
||||
"usage": result.usage_dict(),
|
||||
}
|
||||
|
||||
def _handle_message_stream(self, params: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Handle A2A ``message/stream`` — invoke orchestrator actor (streaming).
|
||||
|
||||
In local-mode A2A, streaming is not natively supported via the
|
||||
synchronous :meth:`dispatch` response model. This handler falls
|
||||
back to non-streaming (``message/send``) and returns the complete
|
||||
response with a ``"streamed": false`` flag so callers can detect
|
||||
the degradation (addresses M4).
|
||||
|
||||
Callers that require real token-by-token streaming should use
|
||||
:meth:`~SessionWorkflow.tell_stream` directly.
|
||||
"""
|
||||
result = self._handle_message_send(params)
|
||||
result["streamed"] = False
|
||||
return result
|
||||
|
||||
def _cleanup_session_devcontainers(self, session_id: str) -> None:
|
||||
"""Best-effort stop of devcontainers associated with a session.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, cast
|
||||
from typing import Any, cast
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -42,7 +42,7 @@ class ActorConfiguration(BaseModel):
|
||||
|
||||
@staticmethod
|
||||
def load_blob_from_file(config_path: Path) -> dict[str, Any]:
|
||||
"""Load a configuration blob from a JSON or YAML file (v2 parity)."""
|
||||
"""Load a configuration blob from a JSON or YAML file."""
|
||||
path = config_path.expanduser()
|
||||
if not path.exists():
|
||||
raise ValueError(f"Config file not found: {path}")
|
||||
@@ -88,36 +88,25 @@ class ActorConfiguration(BaseModel):
|
||||
) -> ActorConfiguration:
|
||||
"""Coerce an incoming config blob plus CLI overrides into a validated model.
|
||||
|
||||
Supports both the legacy v2 ``agents/routes`` YAML layout and the v3
|
||||
``ActorConfigSchema`` format (identified by a top-level ``type`` key
|
||||
whose value is one of ``llm``, ``graph``, or ``tool``).
|
||||
Supports the v3 ``ActorConfigSchema`` format (identified by a top-level
|
||||
``type`` key whose value is one of ``llm``, ``graph``, or ``tool``).
|
||||
Provider and model may be at the top level OR nested inside
|
||||
``actors.<actor-name>.config``.
|
||||
"""
|
||||
|
||||
data: dict[str, Any] = dict(blob) if isinstance(blob, dict) else {}
|
||||
|
||||
# Try v3 extraction first (presence of 'type' with a v3 value).
|
||||
v3_provider, v3_model, v3_graph, v3_unsafe = cls._extract_v3_actor(data)
|
||||
|
||||
# Fall back to v2 extraction when v3 didn't match.
|
||||
v2_provider, v2_model, v2_graph, v2_unsafe = cls._extract_v2_actor(data)
|
||||
v2_options = cls._extract_v2_options(data)
|
||||
|
||||
resolved_provider = (
|
||||
provider
|
||||
or data.get("provider")
|
||||
or data.get("provider_type")
|
||||
or v3_provider
|
||||
or v2_provider
|
||||
)
|
||||
resolved_model = (
|
||||
model or data.get("model") or data.get("model_id") or v3_model or v2_model
|
||||
provider or data.get("provider") or data.get("provider_type") or v3_provider
|
||||
)
|
||||
resolved_model = model or data.get("model") or data.get("model_id") or v3_model
|
||||
resolved_graph = (
|
||||
graph_descriptor
|
||||
or data.get("graph_descriptor")
|
||||
or data.get("graph")
|
||||
or v3_graph
|
||||
or v2_graph
|
||||
)
|
||||
|
||||
options_raw = data.get("options")
|
||||
@@ -127,8 +116,6 @@ class ActorConfiguration(BaseModel):
|
||||
merged_options: dict[str, Any] = {}
|
||||
if default_options:
|
||||
merged_options.update(default_options)
|
||||
if v2_options:
|
||||
merged_options.update(v2_options)
|
||||
if options_value:
|
||||
merged_options.update(options_value)
|
||||
if option_overrides:
|
||||
@@ -136,10 +123,7 @@ class ActorConfiguration(BaseModel):
|
||||
|
||||
top_unsafe_raw = data.get("unsafe", False)
|
||||
resolved_unsafe = (
|
||||
(top_unsafe_raw is True or top_unsafe_raw == 1)
|
||||
or v3_unsafe
|
||||
or v2_unsafe
|
||||
or unsafe
|
||||
(top_unsafe_raw is True or top_unsafe_raw == 1) or v3_unsafe or unsafe
|
||||
)
|
||||
|
||||
if not resolved_provider:
|
||||
@@ -167,27 +151,6 @@ class ActorConfiguration(BaseModel):
|
||||
unsafe=resolved_unsafe,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@staticmethod
|
||||
def _resolve_actors_map(
|
||||
data: dict[str, Any],
|
||||
) -> tuple[Any, Literal["actors", "agents"]]:
|
||||
"""Resolve the actors/agents map from a parsed YAML blob.
|
||||
|
||||
The spec allows either ``actors:`` or ``agents:`` as the top-level
|
||||
map key (oneOf). ``actors:`` is preferred (spec-canonical); the
|
||||
``agents:`` key is only consulted when ``actors:`` is absent or
|
||||
explicitly ``null``.
|
||||
|
||||
Returns ``(map_obj, map_key)`` where *map_obj* is the raw value
|
||||
(may be ``None``, a non-dict, or a dict) and *map_key* is the
|
||||
string ``"actors"`` or ``"agents"``.
|
||||
"""
|
||||
actors_val = data.get("actors")
|
||||
if actors_val is not None:
|
||||
return actors_val, "actors"
|
||||
return data.get("agents"), "agents"
|
||||
|
||||
@staticmethod
|
||||
def _extract_v3_actor(
|
||||
data: dict[str, Any],
|
||||
@@ -195,13 +158,13 @@ class ActorConfiguration(BaseModel):
|
||||
"""Derive provider/model/graph from the v3 Actor YAML schema format.
|
||||
|
||||
The v3 schema is identified by a top-level ``type`` key whose value
|
||||
is one of ``llm``, ``graph``, or ``tool``. The model is taken from
|
||||
the top-level ``model`` key. Because v3 YAML does not carry a
|
||||
``provider`` field the provider is inferred from the model string:
|
||||
is one of ``llm``, ``graph``, or ``tool`` — OR by a top-level
|
||||
``actors:`` map containing at least one entry whose ``config:`` block
|
||||
carries a ``type`` field with one of those values.
|
||||
|
||||
* If the model contains ``/`` the prefix is used as provider
|
||||
(e.g. ``openai/gpt-4`` → provider ``openai``).
|
||||
* Otherwise ``"custom"`` is used as a fallback.
|
||||
Provider and model may appear at the top level OR nested inside
|
||||
``actors.<actor-name>.config``. When found in the nested config,
|
||||
they take precedence over top-level values.
|
||||
|
||||
For ``type: graph`` actors the ``route`` block is wrapped into a
|
||||
graph descriptor.
|
||||
@@ -211,28 +174,65 @@ class ActorConfiguration(BaseModel):
|
||||
may be ``None`` if the data does not look like v3.
|
||||
"""
|
||||
actor_type = data.get("type")
|
||||
if not isinstance(actor_type, str):
|
||||
return None, None, None, False
|
||||
if actor_type.lower() not in _V3_ACTOR_TYPES:
|
||||
return None, None, None, False
|
||||
if not isinstance(actor_type, str) or actor_type.lower() not in _V3_ACTOR_TYPES:
|
||||
actors_val = data.get("actors")
|
||||
if isinstance(actors_val, dict) and actors_val:
|
||||
for _, first_entry in actors_val.items():
|
||||
if isinstance(first_entry, dict):
|
||||
config_block = first_entry.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
nested_type = config_block.get("type")
|
||||
is_v3_type = (
|
||||
isinstance(nested_type, str)
|
||||
and nested_type.lower() in _V3_ACTOR_TYPES
|
||||
)
|
||||
if is_v3_type:
|
||||
actor_type = nested_type
|
||||
break
|
||||
break
|
||||
if not isinstance(actor_type, str):
|
||||
return None, None, None, False
|
||||
|
||||
model_value = data.get("model")
|
||||
# C2: model is optional for TOOL actors — only reject when model
|
||||
# is absent for types that require it (LLM, GRAPH).
|
||||
if not model_value or not isinstance(model_value, str):
|
||||
if actor_type.lower() != "tool":
|
||||
return None, None, None, False
|
||||
# TOOL actors may omit model; use empty string as sentinel.
|
||||
model_value = ""
|
||||
|
||||
# Infer provider from model string or default to "custom".
|
||||
provider_value = (
|
||||
infer_provider_from_model(model_value) if model_value else "custom"
|
||||
)
|
||||
|
||||
provider_value = data.get("provider")
|
||||
unsafe_flag = bool(data.get("unsafe", False))
|
||||
|
||||
# Build a graph descriptor for graph actors from the route block.
|
||||
if not model_value or not isinstance(model_value, str):
|
||||
if actor_type.lower() != "tool":
|
||||
actors_val = data.get("actors")
|
||||
if isinstance(actors_val, dict) and actors_val:
|
||||
for _, first_entry in actors_val.items():
|
||||
if isinstance(first_entry, dict):
|
||||
config_block = first_entry.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
mv = config_block.get("model")
|
||||
if isinstance(mv, str) and mv:
|
||||
model_value = mv
|
||||
break
|
||||
break
|
||||
if not model_value:
|
||||
if actor_type.lower() != "tool":
|
||||
return None, None, None, False
|
||||
model_value = ""
|
||||
|
||||
if not provider_value or not isinstance(provider_value, str):
|
||||
actors_val = data.get("actors")
|
||||
if isinstance(actors_val, dict) and actors_val:
|
||||
for _, first_entry in actors_val.items():
|
||||
if isinstance(first_entry, dict):
|
||||
config_block = first_entry.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
pv = config_block.get("provider")
|
||||
if isinstance(pv, str) and pv:
|
||||
provider_value = pv
|
||||
break
|
||||
break
|
||||
|
||||
if not provider_value:
|
||||
provider_value = (
|
||||
infer_provider_from_model(model_value) if model_value else "custom"
|
||||
)
|
||||
|
||||
graph_descriptor: dict[str, Any] | None = None
|
||||
if actor_type.lower() == "graph":
|
||||
route_raw = data.get("route")
|
||||
@@ -244,127 +244,3 @@ class ActorConfiguration(BaseModel):
|
||||
}
|
||||
|
||||
return provider_value, model_value, graph_descriptor, unsafe_flag
|
||||
|
||||
@staticmethod
|
||||
def _extract_v2_actor(
|
||||
data: dict[str, Any],
|
||||
) -> tuple[str | None, str | None, dict[str, Any] | None, bool]:
|
||||
"""Derive provider/model/graph from the v2/spec YAML actor config format.
|
||||
|
||||
Supports both the legacy ``agents:`` map key and the spec-compliant
|
||||
``actors:`` map key. Within each actor's ``config:`` block, the
|
||||
combined ``actor`` field (e.g. ``"openai/gpt-4"``) is also accepted
|
||||
as an alternative to separate ``provider`` + ``model`` fields.
|
||||
"""
|
||||
|
||||
map_obj, map_key = ActorConfiguration._resolve_actors_map(data)
|
||||
|
||||
# NOTE: Any non-None value for ``actors:`` — including falsy values
|
||||
# such as ``actors: {}``, ``actors: false``, ``actors: 0``, or
|
||||
# ``actors: ""`` — blocks the ``agents:`` fallback. This is
|
||||
# intentional: a present ``actors:`` key explicitly declares the
|
||||
# actors map (even if empty or invalid), whereas ``actors: null``
|
||||
# (or absent) defers to ``agents:``.
|
||||
#
|
||||
# NOTE: Only the **first** entry in the actors/agents map is inspected
|
||||
# for provider, model, unsafe, and graph descriptor. The ``add()``
|
||||
# method rejects multi-actor YAML (>1 entry) with a ``ValidationError``
|
||||
# to prevent silent data loss. The multi-actor case is handled by the
|
||||
# v3 schema validation path.
|
||||
if isinstance(map_obj, dict) and map_obj:
|
||||
map_dict = cast(dict[str, Any], map_obj)
|
||||
# Only the first actor entry is used for provider/model extraction.
|
||||
for actor_key, first_entry in map_dict.items():
|
||||
if isinstance(first_entry, dict):
|
||||
entry_dict = cast(dict[str, Any], first_entry)
|
||||
config_block = entry_dict.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
config_dict = cast(dict[str, Any], config_block)
|
||||
|
||||
# Support the combined ``actor`` field format
|
||||
# (e.g. ``"openai/gpt-4"`` → provider + model).
|
||||
# NOTE: ``provider_type`` and ``model_id`` are
|
||||
# accepted as aliases for backward compatibility,
|
||||
# even though the spec's JSON schema for the
|
||||
# ``config:`` block only lists ``provider``,
|
||||
# ``model``, and ``actor``.
|
||||
# Use ``is not None`` (not ``or``) so that falsy
|
||||
# values like ``""`` or ``0`` are preserved rather
|
||||
# than falling through to the alias.
|
||||
provider_value = config_dict.get("provider")
|
||||
if provider_value is None:
|
||||
provider_value = config_dict.get("provider_type")
|
||||
model_value = config_dict.get("model")
|
||||
if model_value is None:
|
||||
model_value = config_dict.get("model_id")
|
||||
|
||||
combined_actor = config_dict.get("actor")
|
||||
if (
|
||||
combined_actor
|
||||
and isinstance(combined_actor, str)
|
||||
and "/" in combined_actor
|
||||
):
|
||||
parts = combined_actor.split("/", 1)
|
||||
if not provider_value:
|
||||
provider_value = parts[0]
|
||||
if not model_value:
|
||||
model_value = parts[1]
|
||||
|
||||
unsafe_raw = config_dict.get("unsafe", False)
|
||||
# Accept only boolean ``True`` or integer ``1`` as
|
||||
# unsafe. Note: ``unsafe_raw == 1`` also matches
|
||||
# ``1.0`` (float) due to Python's numeric equality
|
||||
# rules — this is fail-safe (treats 1.0 as unsafe).
|
||||
unsafe_flag = unsafe_raw is True or unsafe_raw == 1
|
||||
# The graph descriptor is always built and returned
|
||||
# when a valid ``config:`` block is found, regardless
|
||||
# of whether ``provider``/``model`` were extracted.
|
||||
# This allows the caller to preserve the graph
|
||||
# structure even when provider/model come from
|
||||
# elsewhere (e.g. top-level fields).
|
||||
descriptor: dict[str, Any] = {
|
||||
"agent": actor_key,
|
||||
map_key: dict(map_dict),
|
||||
}
|
||||
for key in (
|
||||
"routes",
|
||||
"merges",
|
||||
"templates",
|
||||
"cleveragents",
|
||||
):
|
||||
if key in data and data[key] is not None:
|
||||
descriptor[key] = data[key]
|
||||
return (
|
||||
str(provider_value) if provider_value else None,
|
||||
str(model_value) if model_value else None,
|
||||
descriptor,
|
||||
unsafe_flag,
|
||||
)
|
||||
break # first entry only
|
||||
return None, None, None, False
|
||||
|
||||
@staticmethod
|
||||
def _extract_v2_options(data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Extract option defaults from the v2/spec actor config format.
|
||||
|
||||
Supports both ``actors:`` (spec-canonical) and ``agents:`` (legacy)
|
||||
map keys.
|
||||
"""
|
||||
|
||||
map_obj, _ = ActorConfiguration._resolve_actors_map(data)
|
||||
if isinstance(map_obj, dict) and map_obj:
|
||||
map_dict = cast(dict[str, Any], map_obj)
|
||||
for _, first_entry in map_dict.items():
|
||||
if isinstance(first_entry, dict):
|
||||
entry_dict = cast(dict[str, Any], first_entry)
|
||||
config_block = entry_dict.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
config_dict = cast(dict[str, Any], config_block)
|
||||
options = config_dict.get("options")
|
||||
if isinstance(options, dict):
|
||||
# Return a shallow copy so downstream mutations
|
||||
# (e.g. ``config_blob["options"]`` updates) do
|
||||
# not propagate back into the original blob.
|
||||
return dict(cast(dict[str, Any], options))
|
||||
break # first entry only
|
||||
return None
|
||||
|
||||
@@ -1,171 +0,0 @@
|
||||
"""Legacy (v2) actor registration logic extracted from ``ActorRegistry``.
|
||||
|
||||
This module handles the v2 blob registration path, including provider/model
|
||||
extraction, unsafe flag enforcement, and nested option merging. It is used
|
||||
by :class:`ActorRegistry` to keep the main registry module under the 500-line
|
||||
limit.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pydantic
|
||||
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
from cleveragents.actor.schema import ActorConfigSchema, is_v3_yaml
|
||||
from cleveragents.application.services.actor_service import ActorService
|
||||
from cleveragents.core.exceptions import NotFoundError, ValidationError
|
||||
from cleveragents.domain.models.core import Actor
|
||||
|
||||
|
||||
def add_legacy(
|
||||
blob: dict[str, Any],
|
||||
*,
|
||||
yaml_text: str,
|
||||
update: bool,
|
||||
schema_version: str,
|
||||
compiled_metadata: dict[str, Any] | None,
|
||||
unsafe: bool = False,
|
||||
allow_unsafe: bool = False,
|
||||
actor_service: ActorService,
|
||||
ensure_namespaced: callable, # type: ignore[type-arg]
|
||||
) -> Actor:
|
||||
"""Register an actor from a legacy (v2) blob.
|
||||
|
||||
Args:
|
||||
blob: Parsed YAML blob.
|
||||
yaml_text: Original YAML source text.
|
||||
update: When ``True`` allow overwriting an existing actor.
|
||||
schema_version: Schema version string.
|
||||
compiled_metadata: Optional compiler-produced metadata dict.
|
||||
unsafe: Asserts that the actor IS unsafe.
|
||||
allow_unsafe: Permits registration of an already-unsafe actor.
|
||||
actor_service: The actor persistence service.
|
||||
ensure_namespaced: Function to namespace actor names.
|
||||
|
||||
Returns:
|
||||
The persisted ``Actor`` domain object.
|
||||
"""
|
||||
name_raw: str = blob.get("name", "")
|
||||
if not name_raw:
|
||||
raise ValidationError("Actor YAML must include a 'name' field.")
|
||||
|
||||
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
|
||||
# This ensures v3 actors are validated against the full schema, including
|
||||
# cycle detection, required fields, and enum validation.
|
||||
blob_is_v3 = is_v3_yaml(blob)
|
||||
if blob_is_v3:
|
||||
try:
|
||||
ActorConfigSchema.model_validate(blob)
|
||||
except pydantic.ValidationError as exc:
|
||||
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
|
||||
|
||||
name = ensure_namespaced(name_raw)
|
||||
provider_raw = blob.get("provider") or blob.get("provider_type", "")
|
||||
model_raw = blob.get("model") or blob.get("model_id", "")
|
||||
|
||||
# ── Guard: reject multi-actor YAML ─────────────────────────────────
|
||||
# ``add()`` is designed for single-actor YAML files. Provider, model,
|
||||
# unsafe, and graph extraction all operate on the *first* entry only,
|
||||
# so a multi-actor map would silently discard later entries (including
|
||||
# their ``unsafe`` flags — a spec-compliance and security concern).
|
||||
# Multi-actor configurations must be split and registered individually.
|
||||
# Uses ``_resolve_actors_map()`` for consistent ``actors:``/``agents:``
|
||||
# resolution across all call sites.
|
||||
actors_map, _ = ActorConfiguration._resolve_actors_map(blob)
|
||||
if isinstance(actors_map, dict) and len(actors_map) > 1:
|
||||
raise ValidationError(
|
||||
"registry.add() only supports single-actor YAML files. "
|
||||
"Multi-actor configurations must be registered individually."
|
||||
)
|
||||
|
||||
# Always extract from the nested ``actors:``/``agents:`` map so that
|
||||
# ``unsafe`` and ``graph_descriptor`` are captured even when top-level
|
||||
# ``provider``/``model`` are present. This matches the behaviour of
|
||||
# ``from_blob()`` which unconditionally calls ``_extract_v2_actor()``.
|
||||
nested_provider, nested_model, nested_graph, nested_unsafe = (
|
||||
ActorConfiguration._extract_v2_actor(blob)
|
||||
)
|
||||
if not provider_raw and nested_provider:
|
||||
provider_raw = nested_provider
|
||||
if not model_raw and nested_model:
|
||||
model_raw = nested_model
|
||||
|
||||
if not blob_is_v3 and (not provider_raw or not model_raw):
|
||||
raise ValidationError("Actor YAML must include 'provider' and 'model' fields.")
|
||||
|
||||
# For v3 actors without provider/model (e.g. TOOL actors), provider_raw
|
||||
# and model_raw may be empty strings here. This is expected — the legacy
|
||||
# canonicalization path in upsert_actor() / from_blob() will reject
|
||||
# provider-less TOOL actors. See follow-up issue #9971.
|
||||
provider = str(provider_raw) if provider_raw else ""
|
||||
model = str(model_raw) if model_raw else ""
|
||||
|
||||
top_unsafe_raw = blob.get("unsafe", False)
|
||||
# Accept only boolean ``True`` or integer ``1`` as unsafe. Note:
|
||||
# ``top_unsafe_raw == 1`` also matches ``1.0`` (float) due to
|
||||
# Python's numeric equality rules — this is fail-safe (treats 1.0
|
||||
# as unsafe).
|
||||
#
|
||||
# ``unsafe`` (the parameter) is an *assertion* — "this actor IS
|
||||
# unsafe" (maps to the spec's "CLI --unsafe flag").
|
||||
# ``allow_unsafe`` is a *permission* — "I PERMIT an already-unsafe
|
||||
# actor to be registered" but does NOT itself make the actor unsafe.
|
||||
# Therefore only ``unsafe`` participates in the stored value; the
|
||||
# spec rule is: "the result is true if *any* of: top-level unsafe,
|
||||
# nested unsafe, or the CLI --unsafe flag is true."
|
||||
effective_unsafe = (
|
||||
(top_unsafe_raw is True or top_unsafe_raw == 1) or nested_unsafe or unsafe
|
||||
)
|
||||
if effective_unsafe and not (unsafe or allow_unsafe):
|
||||
raise ValidationError(
|
||||
"Actor configuration is marked unsafe; pass unsafe=True "
|
||||
"or allow_unsafe=True to confirm."
|
||||
)
|
||||
if not update:
|
||||
try:
|
||||
actor_service.get_actor(name)
|
||||
except NotFoundError:
|
||||
pass # Actor does not exist yet — proceed with add
|
||||
else:
|
||||
raise ValidationError(
|
||||
f"Actor '{name}' already exists. Pass update=True to overwrite."
|
||||
)
|
||||
|
||||
# ── Extract nested options so they are not silently discarded ─────
|
||||
# ``from_blob()`` calls ``_extract_v2_options()`` internally, but
|
||||
# ``add()`` bypasses ``from_blob()``. Without this call, options
|
||||
# nested inside ``actors.<name>.config.options`` would be lost.
|
||||
v2_options = ActorConfiguration._extract_v2_options(blob)
|
||||
|
||||
config_blob: dict[str, Any] = dict(blob)
|
||||
config_blob.setdefault("source", "yaml")
|
||||
if v2_options is not None:
|
||||
existing_options = config_blob.get("options")
|
||||
if existing_options is None or not isinstance(existing_options, dict):
|
||||
# No valid top-level options dict — use nested options directly.
|
||||
config_blob["options"] = v2_options
|
||||
else:
|
||||
# Both top-level and nested options exist. Match ``from_blob()``
|
||||
# merge semantics: nested options as base, top-level overrides.
|
||||
merged = dict(v2_options)
|
||||
merged.update(existing_options)
|
||||
config_blob["options"] = merged
|
||||
|
||||
top_graph_raw = blob.get("graph_descriptor")
|
||||
top_graph = top_graph_raw if top_graph_raw is not None else blob.get("graph")
|
||||
resolved_graph = top_graph if top_graph is not None else nested_graph
|
||||
|
||||
return actor_service.upsert_actor(
|
||||
name=name,
|
||||
provider=provider,
|
||||
model=model,
|
||||
config_blob=config_blob,
|
||||
graph_descriptor=resolved_graph,
|
||||
unsafe=effective_unsafe,
|
||||
set_default=False,
|
||||
yaml_text=yaml_text,
|
||||
schema_version=schema_version,
|
||||
compiled_metadata=compiled_metadata,
|
||||
)
|
||||
@@ -26,7 +26,6 @@ import pydantic
|
||||
import yaml
|
||||
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
from cleveragents.actor.legacy_registry import add_legacy
|
||||
from cleveragents.actor.schema import ActorConfigSchema, is_v3_yaml
|
||||
from cleveragents.actor.v3_registry import add_v3, is_v3_blob
|
||||
from cleveragents.application.services.actor_service import ActorService
|
||||
@@ -326,6 +325,7 @@ class ActorRegistry:
|
||||
# v3 path: validate against ActorConfigSchema when detected
|
||||
# ----------------------------------------------------------
|
||||
if is_v3_blob(blob):
|
||||
effective_allow_unsafe = allow_unsafe or unsafe
|
||||
return add_v3(
|
||||
blob,
|
||||
yaml_text=yaml_text,
|
||||
@@ -333,22 +333,12 @@ class ActorRegistry:
|
||||
schema_version=version,
|
||||
compiled_metadata=compiled_metadata,
|
||||
actor_service=self._actor_service,
|
||||
allow_unsafe=allow_unsafe,
|
||||
allow_unsafe=effective_allow_unsafe,
|
||||
)
|
||||
|
||||
# ----------------------------------------------------------
|
||||
# Legacy (v2) path: flat provider/model extraction
|
||||
# ----------------------------------------------------------
|
||||
return add_legacy(
|
||||
blob,
|
||||
yaml_text=yaml_text,
|
||||
update=update,
|
||||
schema_version=version,
|
||||
compiled_metadata=compiled_metadata,
|
||||
unsafe=unsafe,
|
||||
allow_unsafe=allow_unsafe,
|
||||
actor_service=self._actor_service,
|
||||
ensure_namespaced=self._ensure_namespaced,
|
||||
raise ValidationError(
|
||||
"Invalid actor configuration: no valid type field found. "
|
||||
"Expected a v3 YAML with type: llm, type: graph, or type: tool."
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -26,9 +26,111 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def is_v3_blob(blob: dict[str, Any]) -> bool:
|
||||
"""Return ``True`` when *blob* looks like a v3 ``ActorConfigSchema``."""
|
||||
"""Return ``True`` when *blob* looks like a v3 ``ActorConfigSchema``.
|
||||
|
||||
A blob is considered v3 if it has:
|
||||
1. A top-level ``type`` field with a v3 value (llm/graph/tool), OR
|
||||
2. An ``actors:`` map containing at least one entry whose ``type`` field
|
||||
has a v3 value (nested actor format per spec line 21417).
|
||||
|
||||
The nested format allows configurations like:
|
||||
|
||||
name: local/my-actor
|
||||
actors:
|
||||
my_actor:
|
||||
type: llm
|
||||
config:
|
||||
provider: anthropic
|
||||
model: claude-3.5-sonnet
|
||||
|
||||
Returns:
|
||||
True if the blob appears to be v3 format.
|
||||
"""
|
||||
actor_type = blob.get("type")
|
||||
return isinstance(actor_type, str) and actor_type.lower() in _V3_ACTOR_TYPES
|
||||
if isinstance(actor_type, str) and actor_type.lower() in _V3_ACTOR_TYPES:
|
||||
return True
|
||||
|
||||
actors_val = blob.get("actors")
|
||||
if isinstance(actors_val, dict) and actors_val:
|
||||
for _, entry in actors_val.items():
|
||||
if isinstance(entry, dict):
|
||||
nested_type = entry.get("type")
|
||||
if (
|
||||
isinstance(nested_type, str)
|
||||
and nested_type.lower() in _V3_ACTOR_TYPES
|
||||
):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def _extract_nested_v3_config(data: dict[str, Any]) -> None:
|
||||
"""Extract provider/model/type/description/name from nested actors.<name> block.
|
||||
|
||||
Per spec line 21417, v3 YAML may have provider/model at the top level
|
||||
OR nested inside ``actors.<actor-name>.config``. When found in the nested
|
||||
config, values are promoted to the top level.
|
||||
|
||||
Similarly, ``type``, ``description``, and ``name`` may be at the top level
|
||||
OR nested inside ``actors.<actor-name>`` (for type/description) or
|
||||
``actors.<actor-name>.config`` (for name). This function promotes all
|
||||
of these from their nested locations when not present at the top level.
|
||||
|
||||
Args:
|
||||
data: The actor blob (modified in place).
|
||||
"""
|
||||
actors_val = data.get("actors")
|
||||
if not isinstance(actors_val, dict) or not actors_val:
|
||||
return
|
||||
|
||||
for _, entry in actors_val.items():
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
|
||||
if "type" not in data:
|
||||
type_val = entry.get("type")
|
||||
if isinstance(type_val, str) and type_val:
|
||||
data["type"] = type_val
|
||||
|
||||
if "description" not in data:
|
||||
desc_val = entry.get("description")
|
||||
if isinstance(desc_val, str) and desc_val:
|
||||
data["description"] = desc_val
|
||||
|
||||
if "name" not in data:
|
||||
name_val = entry.get("name")
|
||||
if isinstance(name_val, str) and name_val:
|
||||
data["name"] = name_val
|
||||
|
||||
config_block = entry.get("config")
|
||||
if isinstance(config_block, dict):
|
||||
if "model" not in data:
|
||||
mv = config_block.get("model")
|
||||
if isinstance(mv, str) and mv:
|
||||
data["model"] = mv
|
||||
|
||||
if "provider" not in data:
|
||||
pv = config_block.get("provider")
|
||||
if isinstance(pv, str) and pv:
|
||||
data["provider"] = pv
|
||||
|
||||
if "name" not in data:
|
||||
name_val = config_block.get("name")
|
||||
if isinstance(name_val, str) and name_val:
|
||||
data["name"] = name_val
|
||||
|
||||
if "description" not in data:
|
||||
desc_val = config_block.get("description")
|
||||
if isinstance(desc_val, str) and desc_val:
|
||||
data["description"] = desc_val
|
||||
|
||||
if (
|
||||
"type" in data
|
||||
and "description" in data
|
||||
and "model" in data
|
||||
and "provider" in data
|
||||
):
|
||||
break
|
||||
|
||||
|
||||
def add_v3(
|
||||
@@ -73,6 +175,10 @@ def add_v3(
|
||||
if isinstance(name_raw, str) and name_raw and "/" not in name_raw:
|
||||
data["name"] = f"local/{name_raw}"
|
||||
|
||||
# Extract provider and model from nested actors.<name>.config block if present.
|
||||
# This handles the v3 nested format per spec line 21417.
|
||||
_extract_nested_v3_config(data)
|
||||
|
||||
# Infer provider before schema validation — the schema now requires
|
||||
# provider for LLM and GRAPH actors (added by #5869).
|
||||
if not data.get("provider"):
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
"""Session caller — LangChain-based LLM invocation for session conversations.
|
||||
|
||||
Implements the :class:`LangChainSessionCaller` that wraps a LangChain chat
|
||||
model with full session history context, plus token usage extraction and
|
||||
cost estimation helpers. Used by :class:`SessionWorkflow` in
|
||||
:mod:`cleveragents.application.services.session_workflow`.
|
||||
|
||||
Forgejo: #5784
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
|
||||
from langchain_core.messages.tool import ToolMessage
|
||||
|
||||
from cleveragents.domain.models.core.session import MessageRole
|
||||
from cleveragents.tool.actor_runtime import LLMResponse, LLMToolCall
|
||||
|
||||
# Token cost estimation constants (USD per 1K tokens).
|
||||
# These are rough defaults; real cost comes from the LLM response metadata.
|
||||
_COST_PER_1K_INPUT = 0.001
|
||||
_COST_PER_1K_OUTPUT = 0.002
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Token extraction and cost estimation helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def extract_content(response: Any) -> str:
|
||||
"""Extract plain text content from a LangChain response object."""
|
||||
raw_content = getattr(response, "content", None)
|
||||
if raw_content is None:
|
||||
raw_content = getattr(response, "text", None)
|
||||
if raw_content is None:
|
||||
raw_content = str(response)
|
||||
if isinstance(raw_content, list):
|
||||
parts: list[str] = []
|
||||
for chunk in raw_content:
|
||||
if isinstance(chunk, dict):
|
||||
parts.append(str(chunk.get("text") or chunk.get("content", "")))
|
||||
else:
|
||||
parts.append(str(chunk))
|
||||
return " ".join(parts)
|
||||
return str(raw_content)
|
||||
|
||||
|
||||
def extract_token_usage(response: Any) -> tuple[int, int]:
|
||||
"""Extract (input_tokens, output_tokens) from a LangChain response.
|
||||
|
||||
Attempts ``response_metadata`` and ``usage_metadata`` attributes.
|
||||
Returns (0, 0) when no usage information is present.
|
||||
"""
|
||||
# LangChain ≥ 0.2: response_metadata may carry provider-specific usage
|
||||
meta = getattr(response, "response_metadata", None) or {}
|
||||
usage = meta.get("usage") or meta.get("token_usage") or {}
|
||||
if usage:
|
||||
try:
|
||||
inp = int(usage.get("input_tokens") or usage.get("prompt_tokens") or 0)
|
||||
out = int(usage.get("output_tokens") or usage.get("completion_tokens") or 0)
|
||||
return inp, out
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
|
||||
# LangChain ≥ 0.3: usage_metadata attribute
|
||||
usage_meta = getattr(response, "usage_metadata", None) or {}
|
||||
if usage_meta:
|
||||
try:
|
||||
inp = int(usage_meta.get("input_tokens") or 0)
|
||||
out = int(usage_meta.get("output_tokens") or 0)
|
||||
return inp, out
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
|
||||
return 0, 0
|
||||
|
||||
|
||||
def estimate_cost(input_tokens: int, output_tokens: int) -> float:
|
||||
"""Estimate LLM cost in USD from token counts."""
|
||||
raw = input_tokens * _COST_PER_1K_INPUT + output_tokens * _COST_PER_1K_OUTPUT
|
||||
return raw / 1000.0
|
||||
|
||||
|
||||
def history_to_langchain_messages(
|
||||
messages: list[Any],
|
||||
) -> list[Any]:
|
||||
"""Convert session message history to LangChain message objects.
|
||||
|
||||
Maps:
|
||||
- ``MessageRole.SYSTEM`` → ``SystemMessage``
|
||||
- ``MessageRole.USER`` → ``HumanMessage``
|
||||
- ``MessageRole.ASSISTANT`` → ``AIMessage``
|
||||
- ``MessageRole.TOOL`` → ``ToolMessage``
|
||||
|
||||
Returns an ordered list of LangChain message objects, empty if
|
||||
no history is available.
|
||||
"""
|
||||
lc_messages: list[Any] = []
|
||||
for msg in messages:
|
||||
role = msg.role
|
||||
content = msg.content
|
||||
if role == MessageRole.SYSTEM:
|
||||
lc_messages.append(SystemMessage(content=content))
|
||||
elif role == MessageRole.USER:
|
||||
lc_messages.append(HumanMessage(content=content))
|
||||
elif role == MessageRole.ASSISTANT:
|
||||
lc_messages.append(AIMessage(content=content))
|
||||
elif role == MessageRole.TOOL:
|
||||
tool_id = msg.tool_call_id or "unknown"
|
||||
lc_messages.append(ToolMessage(content=content, tool_call_id=tool_id))
|
||||
else:
|
||||
# Unknown role — default to human message to preserve context
|
||||
lc_messages.append(HumanMessage(content=content))
|
||||
return lc_messages
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# LangChainSessionCaller
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class LangChainSessionCaller:
|
||||
"""LLMCaller implementation for session conversations.
|
||||
|
||||
Wraps a LangChain chat model with full session history context.
|
||||
Implements the :class:`~cleveragents.tool.actor_runtime.LLMCaller`
|
||||
protocol so it can be used with :class:`ToolCallingRuntime`.
|
||||
|
||||
The caller is stateful within a single ``tell()`` invocation: it
|
||||
accumulates the LangChain message thread across tool-call loop
|
||||
iterations. Create a fresh instance for each ``tell()`` call.
|
||||
|
||||
Args:
|
||||
llm: A LangChain ``BaseLanguageModel`` (chat model).
|
||||
history: Pre-converted list of LangChain message objects
|
||||
representing the session history *before* the current
|
||||
user message.
|
||||
session_system_prompt: Optional system prompt prepended to
|
||||
the message list when no system message exists in history.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
llm: Any,
|
||||
history: list[Any],
|
||||
session_system_prompt: str | None = (
|
||||
"You are the CleverAgents orchestrator. You interpret the user's "
|
||||
"natural-language requests and help them manage their software "
|
||||
"development workflows. Be concise, accurate, and helpful."
|
||||
),
|
||||
) -> None:
|
||||
self._llm = llm
|
||||
self._last_response: Any = None
|
||||
# Build initial accumulated messages: optional system + history
|
||||
self._accumulated: list[Any] = []
|
||||
|
||||
# Prepend system prompt only when history doesn't already have one
|
||||
has_system = any(isinstance(m, SystemMessage) for m in history)
|
||||
if session_system_prompt and not has_system:
|
||||
self._accumulated.append(SystemMessage(content=session_system_prompt))
|
||||
self._accumulated.extend(history)
|
||||
self._first_call = True
|
||||
# Accumulated real token usage across all invoke() calls (M1).
|
||||
self._total_input_tokens = 0
|
||||
self._total_output_tokens = 0
|
||||
|
||||
def invoke(
|
||||
self,
|
||||
prompt: str,
|
||||
tool_schemas: list[dict[str, Any]],
|
||||
tool_results: list[dict[str, Any]] | None = None,
|
||||
actor_config: dict[str, Any] | None = None,
|
||||
) -> Any:
|
||||
"""Invoke the LLM with the accumulated conversation context.
|
||||
|
||||
On the first call (``tool_results is None``):
|
||||
Appends the user message and invokes the LLM.
|
||||
|
||||
On subsequent calls (``tool_results`` present):
|
||||
Appends the previous AI response (if available) and tool
|
||||
results, then invokes the LLM again.
|
||||
|
||||
Args:
|
||||
prompt: The original user prompt (used only on the first call).
|
||||
tool_schemas: Provider tool schemas (unused — no tool binding).
|
||||
tool_results: Tool execution results from the previous iteration.
|
||||
actor_config: Actor configuration dict (currently unused).
|
||||
|
||||
Returns:
|
||||
:class:`~cleveragents.tool.actor_runtime.LLMResponse` with the
|
||||
LLM's text response.
|
||||
"""
|
||||
if self._first_call:
|
||||
# First iteration: append the user's prompt and invoke
|
||||
self._accumulated.append(HumanMessage(content=prompt))
|
||||
self._first_call = False
|
||||
elif tool_results is not None:
|
||||
# Subsequent iteration: add previous AI response + tool results
|
||||
if self._last_response is not None:
|
||||
self._accumulated.append(self._last_response)
|
||||
for tr in tool_results:
|
||||
tool_result_content = (
|
||||
str(tr.get("output", {}))
|
||||
if tr.get("success")
|
||||
else tr.get("error", "error")
|
||||
)
|
||||
self._accumulated.append(
|
||||
ToolMessage(
|
||||
content=tool_result_content,
|
||||
tool_call_id=(
|
||||
tr.get("call_id") or tr.get("tool_name", "unknown")
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
# Invoke the LLM (no tool binding — session tell is conversational)
|
||||
response = self._llm.invoke(self._accumulated)
|
||||
self._last_response = response
|
||||
|
||||
# Extract real token usage from the LLM response metadata (M1).
|
||||
inp, out = extract_token_usage(response)
|
||||
self._total_input_tokens += inp
|
||||
self._total_output_tokens += out
|
||||
|
||||
content = extract_content(response)
|
||||
|
||||
# Extract any tool calls from the response
|
||||
tool_calls: list[LLMToolCall] = []
|
||||
raw_tool_calls = getattr(response, "tool_calls", None) or []
|
||||
for tc in raw_tool_calls:
|
||||
if isinstance(tc, dict):
|
||||
name = tc.get("name") or tc.get("id", "")
|
||||
args = tc.get("args") or tc.get("arguments") or {}
|
||||
call_id = tc.get("id", "")
|
||||
if name:
|
||||
tool_calls.append(
|
||||
LLMToolCall(name=name, arguments=args, call_id=call_id)
|
||||
)
|
||||
|
||||
return LLMResponse(content=content, tool_calls=tool_calls)
|
||||
|
||||
def get_token_usage(self) -> tuple[int, int]:
|
||||
"""Return accumulated ``(input_tokens, output_tokens)`` across all calls.
|
||||
|
||||
Tokens are extracted from real LLM response metadata via
|
||||
:func:`extract_token_usage`. Returns ``(0, 0)`` when the LLM
|
||||
provides no usage metadata (stub / minimal mode).
|
||||
"""
|
||||
return self._total_input_tokens, self._total_output_tokens
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Minimal stub LLM response helpers (used by SessionWorkflow._make_stub_llm)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def estimate_tokens(prompt: str, response: str) -> tuple[int, int]:
|
||||
"""Estimate token counts using a simple word-based heuristic.
|
||||
|
||||
Returns ``(input_tokens, output_tokens)`` based on the
|
||||
conventional approximation of ~4 characters per token.
|
||||
"""
|
||||
input_tokens = max(1, len(prompt) // 4)
|
||||
output_tokens = max(1, len(response) // 4)
|
||||
return input_tokens, output_tokens
|
||||
|
||||
|
||||
class _StubResp:
|
||||
"""Minimal LLM response object for stub mode."""
|
||||
|
||||
content = "(no LLM configured)"
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.tool_calls: list[Any] = []
|
||||
self.response_metadata: dict[str, Any] = {}
|
||||
|
||||
|
||||
class _StubChunk:
|
||||
"""Minimal streaming chunk for stub mode."""
|
||||
|
||||
content = "(no LLM configured)"
|
||||
|
||||
|
||||
class _MinimalStubLLM:
|
||||
"""Minimal LLM stub for environments without langchain_community."""
|
||||
|
||||
def invoke(self, messages: Any, **kwargs: Any) -> _StubResp: # pragma: no cover
|
||||
"""Return a stub response."""
|
||||
return _StubResp()
|
||||
|
||||
def stream(
|
||||
self, messages: Any, **kwargs: Any
|
||||
) -> Iterator[_StubChunk]: # pragma: no cover
|
||||
"""Return a single stub chunk."""
|
||||
yield _StubChunk()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"LangChainSessionCaller",
|
||||
"estimate_cost",
|
||||
"extract_content",
|
||||
"extract_token_usage",
|
||||
"history_to_langchain_messages",
|
||||
]
|
||||
@@ -328,6 +328,23 @@ class PersistentSessionService(SessionService):
|
||||
except (KeyError, ValueError, TypeError) as exc:
|
||||
raise SessionImportError(f"Invalid import data: {exc}") from exc
|
||||
|
||||
def get_messages(self, session_id: str) -> list[SessionMessage]:
|
||||
"""Retrieve all messages for a session in order.
|
||||
|
||||
Args:
|
||||
session_id: The ULID of the session.
|
||||
|
||||
Returns:
|
||||
Ordered list of SessionMessage objects (by sequence number).
|
||||
|
||||
Raises:
|
||||
SessionNotFoundError: If no session with the given ID exists.
|
||||
"""
|
||||
session = self._session_repo.get_by_id(session_id)
|
||||
if session is None:
|
||||
raise SessionNotFoundError(f"Session '{session_id}' not found")
|
||||
return self._message_repo.get_for_session(session_id)
|
||||
|
||||
def update_token_usage(
|
||||
self,
|
||||
session_id: str,
|
||||
|
||||
@@ -0,0 +1,467 @@
|
||||
"""Session workflow — orchestrator LLM invocation for ``agents session tell``.
|
||||
|
||||
Replaces the M3 stub with real LLM actor invocation. The user prompt is
|
||||
sent to the session's bound orchestrator actor via the A2A ``message/send`` /
|
||||
``message/stream`` operations routed through :class:`A2aLocalFacade`.
|
||||
|
||||
Call chain in local mode::
|
||||
|
||||
tell CLI ──▸ A2aLocalFacade.dispatch("message/send")
|
||||
──▸ SessionWorkflow.tell()
|
||||
──▸ LangChainSessionCaller.invoke()
|
||||
──▸ LangChain chat model (real LLM)
|
||||
|
||||
Forgejo: #5784
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from collections.abc import Callable, Iterator
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import structlog
|
||||
from langchain_core.messages import HumanMessage, SystemMessage
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
try:
|
||||
from langchain_community.llms import FakeListLLM
|
||||
except ImportError: # pragma: no cover — langchain_community optional
|
||||
FakeListLLM = None
|
||||
|
||||
from cleveragents.application.services.prompt_sanitizer import PromptSanitizer
|
||||
from cleveragents.application.services.session_caller import (
|
||||
LangChainSessionCaller,
|
||||
_MinimalStubLLM,
|
||||
estimate_cost,
|
||||
estimate_tokens,
|
||||
extract_content,
|
||||
history_to_langchain_messages,
|
||||
)
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
SessionActorNotConfiguredError,
|
||||
SessionMessage,
|
||||
)
|
||||
from cleveragents.tool.actor_runtime import (
|
||||
LLMCaller,
|
||||
ToolCallingRuntime,
|
||||
ToolCallRunResult,
|
||||
)
|
||||
from cleveragents.tool.registry import ToolRegistry
|
||||
from cleveragents.tool.runner import ToolRunner
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.domain.models.core.session import SessionService
|
||||
from cleveragents.providers.registry import ProviderRegistry
|
||||
|
||||
logger = structlog.get_logger(__name__)
|
||||
_sanitizer = PromptSanitizer()
|
||||
|
||||
# System prompt for the orchestrator actor in session context.
|
||||
_SESSION_SYSTEM_PROMPT = (
|
||||
"You are the CleverAgents orchestrator. You interpret the user's "
|
||||
"natural-language requests and help them manage their software "
|
||||
"development workflows. Be concise, accurate, and helpful."
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TellResult
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TellResult(BaseModel):
|
||||
"""Result of a :meth:`SessionWorkflow.tell` invocation.
|
||||
|
||||
Attributes:
|
||||
session_id: Session ULID.
|
||||
user_message: Original user prompt.
|
||||
assistant_message: Orchestrator response text.
|
||||
input_tokens: Input token count.
|
||||
output_tokens: Output token count.
|
||||
cost: Estimated cost in USD.
|
||||
duration_ms: Wall-clock LLM invocation duration (ms).
|
||||
tool_calls_count: Number of tool calls made.
|
||||
"""
|
||||
|
||||
session_id: str = Field(..., description="ULID of the session")
|
||||
user_message: str = Field(..., description="Original user prompt")
|
||||
assistant_message: str = Field(..., description="Orchestrator response text")
|
||||
input_tokens: int = Field(0, ge=0, description="Input token count")
|
||||
output_tokens: int = Field(0, ge=0, description="Output token count")
|
||||
cost: float = Field(0.0, ge=0.0, description="Estimated cost in USD")
|
||||
duration_ms: float = Field(0.0, ge=0.0, description="LLM invocation duration (ms)")
|
||||
tool_calls_count: int = Field(0, ge=0, description="Number of tool calls made")
|
||||
|
||||
model_config = ConfigDict(
|
||||
str_strip_whitespace=True,
|
||||
validate_assignment=True,
|
||||
)
|
||||
|
||||
def usage_dict(self) -> dict[str, Any]:
|
||||
"""Return a JSON-serializable usage summary dict."""
|
||||
return {
|
||||
"input_tokens": self.input_tokens,
|
||||
"output_tokens": self.output_tokens,
|
||||
"cost_usd": round(self.cost, 6),
|
||||
"duration_ms": round(self.duration_ms, 1),
|
||||
"tool_calls": self.tool_calls_count,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SessionWorkflow
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SessionWorkflow:
|
||||
"""Orchestrates ``agents session tell`` — real LLM actor invocation.
|
||||
|
||||
Resolves the session's actor (or ``--actor`` override), invokes the LLM
|
||||
via :class:`ToolCallingRuntime`, persists the response, and records
|
||||
token usage via :class:`SessionService`.
|
||||
|
||||
Args:
|
||||
session_service: Session persistence service.
|
||||
provider_registry: Optional provider registry for LLM resolution.
|
||||
When ``None``, falls back to a stub response.
|
||||
tool_registry: Optional ``ToolRegistry`` (defaults to empty).
|
||||
llm_factory: Optional ``(actor_name: str) -> Any`` test injection point.
|
||||
Tests inject stub LLMs through this factory.
|
||||
max_iterations: Max tool-call loop iterations (default 25).
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
session_service: SessionService,
|
||||
provider_registry: ProviderRegistry | None = None,
|
||||
tool_registry: ToolRegistry | None = None,
|
||||
llm_factory: Callable[[str], Any] | None = None,
|
||||
max_iterations: int = 25,
|
||||
) -> None:
|
||||
self._session_service = session_service
|
||||
self._provider_registry = provider_registry
|
||||
self._tool_registry = tool_registry or ToolRegistry()
|
||||
self._llm_factory = llm_factory
|
||||
self._max_iterations = max_iterations
|
||||
self._logger = logger.bind(component="session_workflow")
|
||||
# Populated by tell_stream() so callers can read real usage metrics
|
||||
# after the stream is exhausted (addresses M3, M4).
|
||||
self.last_stream_usage: dict[str, Any] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def tell(
|
||||
self,
|
||||
session_id: str,
|
||||
prompt: str,
|
||||
actor_override: str | None = None,
|
||||
) -> TellResult:
|
||||
"""Send a user prompt to the session's orchestrator actor (non-streaming).
|
||||
|
||||
Appends the user message, resolves the actor, invokes the LLM,
|
||||
persists the assistant response, and records token usage.
|
||||
|
||||
Raises:
|
||||
SessionNotFoundError: If the session does not exist.
|
||||
SessionActorNotConfiguredError: If no actor is configured.
|
||||
ValueError: If prompt is empty.
|
||||
"""
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("prompt must not be empty")
|
||||
|
||||
session = self._session_service.get(session_id)
|
||||
actor_name = actor_override or session.actor_name
|
||||
if not actor_name:
|
||||
raise SessionActorNotConfiguredError(
|
||||
"Session has no actor configured. "
|
||||
"Create a session with --actor <actor> or pass --actor."
|
||||
)
|
||||
|
||||
# Load history before appending the user message so the slice in
|
||||
# _invoke_actor reads a stable snapshot (addresses m11).
|
||||
# TODO(#5784-M9): If _invoke_actor() raises below, the session retains
|
||||
# an orphaned user message with no paired assistant response. Implement
|
||||
# compensation logic that removes the orphaned message on failure, or
|
||||
# persist the user message transactionally with the assistant response.
|
||||
# Sanitize the prompt before passing to the LLM (addresses M7).
|
||||
sanitized_prompt = _sanitizer.sanitize_user_input(prompt).sanitized
|
||||
|
||||
self._session_service.append_message(
|
||||
session_id=session_id,
|
||||
role=MessageRole.USER,
|
||||
content=prompt,
|
||||
)
|
||||
|
||||
start = time.monotonic()
|
||||
run_result, real_input_tokens, real_output_tokens = self._invoke_actor(
|
||||
session_id=session_id,
|
||||
prompt=sanitized_prompt,
|
||||
actor_name=actor_name,
|
||||
)
|
||||
duration_ms = (time.monotonic() - start) * 1000.0
|
||||
|
||||
assistant_content = run_result.content or ""
|
||||
|
||||
# Persist assistant response
|
||||
self._session_service.append_message(
|
||||
session_id=session_id,
|
||||
role=MessageRole.ASSISTANT,
|
||||
content=assistant_content if assistant_content else "(no response)",
|
||||
)
|
||||
|
||||
# Use real token counts from LLM response metadata when available,
|
||||
# falling back to the heuristic estimate otherwise (M1).
|
||||
if real_input_tokens > 0 or real_output_tokens > 0:
|
||||
input_tokens = real_input_tokens
|
||||
output_tokens = real_output_tokens
|
||||
else:
|
||||
input_tokens, output_tokens = estimate_tokens(
|
||||
sanitized_prompt, assistant_content
|
||||
)
|
||||
cost = estimate_cost(input_tokens, output_tokens)
|
||||
self._session_service.update_token_usage(
|
||||
session_id=session_id,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
cost=cost,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"session.tell.complete",
|
||||
session_id=session_id,
|
||||
actor=actor_name,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
tool_calls=len(run_result.tool_call_history),
|
||||
duration_ms=round(duration_ms, 1),
|
||||
)
|
||||
|
||||
return TellResult(
|
||||
session_id=session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=assistant_content,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
cost=cost,
|
||||
duration_ms=round(duration_ms, 1),
|
||||
tool_calls_count=len(run_result.tool_call_history),
|
||||
)
|
||||
|
||||
def tell_stream(
|
||||
self,
|
||||
session_id: str,
|
||||
prompt: str,
|
||||
actor_override: str | None = None,
|
||||
) -> Iterator[str]:
|
||||
"""Stream the assistant's response token-by-token.
|
||||
|
||||
Loads session history before appending the user message (addresses C1).
|
||||
After the stream is exhausted, the full response is persisted, token
|
||||
usage is recorded, and ``last_stream_usage`` is populated (M3, M4).
|
||||
|
||||
Yields:
|
||||
Successive text tokens from the LLM.
|
||||
|
||||
Raises:
|
||||
SessionNotFoundError: If the session does not exist.
|
||||
SessionActorNotConfiguredError: If no actor is configured.
|
||||
ValueError: If prompt is empty.
|
||||
"""
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("prompt must not be empty")
|
||||
|
||||
# Reset stale usage data before any operation that could raise (M4).
|
||||
self.last_stream_usage = {}
|
||||
|
||||
session = self._session_service.get(session_id)
|
||||
actor_name = actor_override or session.actor_name
|
||||
if not actor_name:
|
||||
raise SessionActorNotConfiguredError(
|
||||
"Session has no actor configured. "
|
||||
"Create a session with --actor <actor> or pass --actor."
|
||||
)
|
||||
|
||||
# Load history BEFORE appending the user message (addresses C1).
|
||||
# Build the LC message list from existing history + the new prompt
|
||||
# so the LLM does not receive a duplicate HumanMessage.
|
||||
existing_history = self._session_service.get_messages(session_id)
|
||||
|
||||
# Sanitize the prompt before passing to the LLM (addresses M7).
|
||||
sanitized_prompt = _sanitizer.sanitize_user_input(prompt).sanitized
|
||||
lc_messages = self._build_lc_messages_from_history(
|
||||
existing_history, sanitized_prompt
|
||||
)
|
||||
|
||||
# Append user message to the session (persisted separately from
|
||||
# the LC message list used for the LLM call).
|
||||
self._session_service.append_message(
|
||||
session_id=session_id,
|
||||
role=MessageRole.USER,
|
||||
content=prompt,
|
||||
)
|
||||
|
||||
llm = self._resolve_llm(actor_name)
|
||||
|
||||
start = time.monotonic()
|
||||
collected_chunks: list[str] = []
|
||||
|
||||
# try/finally guarantees persistence even when the consumer
|
||||
# breaks early. The stream is explicitly closed in the finally
|
||||
# block to release resources (M8).
|
||||
stream = llm.stream(lc_messages)
|
||||
try:
|
||||
for chunk in stream:
|
||||
token = extract_content(chunk)
|
||||
if token:
|
||||
collected_chunks.append(token)
|
||||
yield token
|
||||
finally:
|
||||
if hasattr(stream, "close"):
|
||||
stream.close()
|
||||
duration_ms = (time.monotonic() - start) * 1000.0
|
||||
full_content = "".join(collected_chunks)
|
||||
|
||||
# Persist and record usage
|
||||
self._session_service.append_message(
|
||||
session_id=session_id,
|
||||
role=MessageRole.ASSISTANT,
|
||||
content=full_content if full_content else "(no response)",
|
||||
)
|
||||
input_tokens, output_tokens = estimate_tokens(
|
||||
sanitized_prompt, full_content
|
||||
)
|
||||
cost = estimate_cost(input_tokens, output_tokens)
|
||||
self._session_service.update_token_usage(
|
||||
session_id=session_id,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
cost=cost,
|
||||
)
|
||||
|
||||
# Populate last_stream_usage so CLI can read real metrics (M3, M4)
|
||||
self.last_stream_usage = {
|
||||
"input_tokens": input_tokens,
|
||||
"output_tokens": output_tokens,
|
||||
"cost_usd": cost,
|
||||
"duration_ms": round(duration_ms, 1),
|
||||
"tool_calls": 0,
|
||||
}
|
||||
|
||||
self._logger.info(
|
||||
"session.tell_stream.complete",
|
||||
session_id=session_id,
|
||||
actor=actor_name,
|
||||
duration_ms=round(duration_ms, 1),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Private helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _resolve_llm(self, actor_name: str) -> Any:
|
||||
"""Create a LangChain LLM for the given actor name.
|
||||
|
||||
Delegates to ``llm_factory`` (test injection), ``ProviderRegistry``,
|
||||
or falls back to a stub when neither is available.
|
||||
"""
|
||||
# Test injection point — skips provider registry when llm_factory is set.
|
||||
# Must be checked before _provider_registry fallback so tests can inject
|
||||
# stubs even when the registry is None (the default).
|
||||
if self._llm_factory is not None:
|
||||
result = self._llm_factory(actor_name)
|
||||
if result is not None:
|
||||
return result
|
||||
# Factory returned None; fall through to provider registry
|
||||
|
||||
if self._provider_registry is None:
|
||||
return self._make_stub_llm()
|
||||
|
||||
from cleveragents.application.services.strategy_resolution import (
|
||||
_parse_actor_name,
|
||||
)
|
||||
|
||||
provider_type, model_id = _parse_actor_name(actor_name)
|
||||
return self._provider_registry.create_llm(
|
||||
provider_type=provider_type,
|
||||
model_id=model_id,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _make_stub_llm() -> Any:
|
||||
"""Return a deterministic stub LLM for environments without API keys."""
|
||||
if FakeListLLM is not None:
|
||||
return FakeListLLM(responses=["(no LLM configured)"] * 100)
|
||||
return _MinimalStubLLM()
|
||||
|
||||
def _build_lc_messages_from_history(
|
||||
self,
|
||||
history_messages: list[SessionMessage],
|
||||
prompt: str,
|
||||
) -> list[Any]:
|
||||
"""Build LangChain messages from pre-loaded history + new prompt.
|
||||
|
||||
Used by ``tell_stream()`` to avoid the duplicate-user-message bug (C1).
|
||||
The caller controls when history is sampled.
|
||||
"""
|
||||
lc_history = history_to_langchain_messages(history_messages)
|
||||
|
||||
messages: list[Any] = []
|
||||
if not any(isinstance(m, SystemMessage) for m in lc_history):
|
||||
messages.append(SystemMessage(content=_SESSION_SYSTEM_PROMPT))
|
||||
messages.extend(lc_history)
|
||||
messages.append(HumanMessage(content=prompt))
|
||||
return messages
|
||||
|
||||
def _invoke_actor(
|
||||
self,
|
||||
session_id: str,
|
||||
prompt: str,
|
||||
actor_name: str,
|
||||
) -> tuple[ToolCallRunResult, int, int]:
|
||||
"""Run the tool-calling loop against the resolved LLM.
|
||||
|
||||
Returns ``(run_result, input_tokens, output_tokens)`` with real token
|
||||
counts from LLM response metadata. Callers fall back to
|
||||
``estimate_tokens()`` when both counts are zero.
|
||||
"""
|
||||
llm = self._resolve_llm(actor_name)
|
||||
|
||||
# Load history — only SessionNotFoundError is handled;
|
||||
# infrastructure errors propagate (addresses M5).
|
||||
# TODO(#5784-m7): Session message history (assistant/tool roles) is fed to
|
||||
# the LLM without sanitization. Apply PromptSanitizer to historical
|
||||
# messages to prevent prompt injection from prior assistant responses.
|
||||
history_msgs = self._session_service.get_messages(session_id)
|
||||
|
||||
# Exclude the user message we just appended (last message)
|
||||
# so it is not duplicated when LangChainSessionCaller appends it
|
||||
lc_history = history_to_langchain_messages(history_msgs[:-1])
|
||||
|
||||
caller: LLMCaller = LangChainSessionCaller(
|
||||
llm=llm,
|
||||
history=lc_history,
|
||||
)
|
||||
|
||||
runner = ToolRunner(registry=self._tool_registry)
|
||||
runtime = ToolCallingRuntime(
|
||||
registry=self._tool_registry,
|
||||
runner=runner,
|
||||
llm_caller=caller,
|
||||
max_iterations=self._max_iterations,
|
||||
)
|
||||
|
||||
run_result = runtime.run_tool_loop(prompt=prompt)
|
||||
# Read real token usage accumulated by LangChainSessionCaller (M1).
|
||||
real_input, real_output = caller.get_token_usage()
|
||||
return run_result, real_input, real_output
|
||||
|
||||
|
||||
__all__ = [
|
||||
"LangChainSessionCaller",
|
||||
"SessionWorkflow",
|
||||
"TellResult",
|
||||
]
|
||||
@@ -27,7 +27,7 @@ import re
|
||||
import shutil
|
||||
import time
|
||||
from contextlib import suppress
|
||||
from datetime import datetime
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Annotated, Any, Literal, cast
|
||||
|
||||
@@ -4117,6 +4117,131 @@ def _get_decision_label(decision_type: str, per_type_ordinal: int = 0) -> str:
|
||||
return base_label
|
||||
|
||||
|
||||
def _build_tree_data(
|
||||
plan_id: str,
|
||||
tree_data: list[dict[str, object]],
|
||||
decisions: list[Decision],
|
||||
show_superseded: bool = False,
|
||||
started_at: datetime | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""Build the data payload for ``agents plan tree --format json/yaml``.
|
||||
|
||||
Returns the ``data`` dict that will be wrapped in the spec-required
|
||||
command envelope by ``format_output``.
|
||||
"""
|
||||
filtered = (
|
||||
decisions if show_superseded else [d for d in decisions if not d.is_superseded]
|
||||
)
|
||||
|
||||
def count_nodes(nodes: list[dict[str, object]]) -> int:
|
||||
count = 0
|
||||
for node in nodes:
|
||||
count += 1
|
||||
children = node.get("children", [])
|
||||
if isinstance(children, list):
|
||||
count += count_nodes(children)
|
||||
return count
|
||||
|
||||
def compute_depth(nodes: list[dict[str, object]]) -> int:
|
||||
if not nodes:
|
||||
return 0
|
||||
max_depth = 0
|
||||
for node in nodes:
|
||||
children = node.get("children", [])
|
||||
if isinstance(children, list) and children:
|
||||
max_depth = max(max_depth, 1 + compute_depth(children))
|
||||
return max_depth
|
||||
|
||||
nodes_count = count_nodes(tree_data)
|
||||
tree_depth = compute_depth(tree_data)
|
||||
|
||||
child_plan_ids: set[str] = set()
|
||||
for d in filtered:
|
||||
if d.decision_type in ("subplan_spawn", "subplan_parallel_spawn") and d.plan_id:
|
||||
child_plan_ids.add(d.plan_id)
|
||||
|
||||
child_plans_count = len(child_plan_ids)
|
||||
child_plans_str = f"{child_plans_count}+" if child_plans_count > 0 else "0"
|
||||
|
||||
invariants_count = sum(
|
||||
1 for d in filtered if d.decision_type == "invariant_enforced"
|
||||
)
|
||||
|
||||
superseded_count = sum(1 for d in decisions if d.is_superseded)
|
||||
|
||||
summary = {
|
||||
"nodes": nodes_count,
|
||||
"depth": tree_depth,
|
||||
"child_plans": child_plans_str,
|
||||
"invariants": invariants_count,
|
||||
"superseded": superseded_count,
|
||||
}
|
||||
|
||||
type_counts: dict[str, int] = {}
|
||||
decision_ids: dict[str, str] = {}
|
||||
|
||||
for d in filtered:
|
||||
type_counts[d.decision_type] = type_counts.get(d.decision_type, 0) + 1
|
||||
ordinal = type_counts[d.decision_type]
|
||||
|
||||
if d.decision_type == "prompt_definition":
|
||||
key = "root"
|
||||
elif d.decision_type == "invariant_enforced":
|
||||
key = f"invariant_{ordinal}"
|
||||
elif d.decision_type == "strategy_choice":
|
||||
key = "strategy"
|
||||
elif d.decision_type == "implementation_choice":
|
||||
key = f"implementation_{ordinal}"
|
||||
elif d.decision_type == "subplan_spawn":
|
||||
key = f"spawn_{ordinal}"
|
||||
elif d.decision_type == "subplan_parallel_spawn":
|
||||
key = f"parallel_{ordinal}"
|
||||
else:
|
||||
key = f"{d.decision_type}_{ordinal}"
|
||||
|
||||
decision_ids[key] = d.decision_id
|
||||
|
||||
child_plans_list: list[dict[str, object]] = []
|
||||
for d in filtered:
|
||||
if d.decision_type in ("subplan_spawn", "subplan_parallel_spawn") and d.plan_id:
|
||||
child_plans_list.append(
|
||||
{
|
||||
"id": d.plan_id,
|
||||
"phase": "execute",
|
||||
"state": "queued",
|
||||
}
|
||||
)
|
||||
|
||||
def convert_tree_node(node: dict[str, object]) -> dict[str, object]:
|
||||
"""Convert internal tree node format to spec format."""
|
||||
spec_node: dict[str, object] = {
|
||||
"type": node.get("type"),
|
||||
"description": node.get("question") or node.get("description"),
|
||||
}
|
||||
|
||||
if node.get("confidence") is not None:
|
||||
spec_node["confidence"] = node.get("confidence")
|
||||
|
||||
if node.get("type") in ("subplan_spawn", "subplan_parallel_spawn"):
|
||||
spec_node["plan_id"] = node.get("plan_id", "")
|
||||
|
||||
children = node.get("children", [])
|
||||
if isinstance(children, list) and children:
|
||||
spec_node["children"] = [convert_tree_node(child) for child in children]
|
||||
|
||||
return spec_node
|
||||
|
||||
spec_tree = convert_tree_node(tree_data[0]) if tree_data else None
|
||||
|
||||
return {
|
||||
"plan_id": plan_id,
|
||||
"tree": spec_tree,
|
||||
"summary": summary,
|
||||
"child_plans": child_plans_list,
|
||||
"decision_ids": decision_ids,
|
||||
}
|
||||
|
||||
|
||||
@app.command("tree")
|
||||
def tree_decisions_cmd(
|
||||
plan_id: Annotated[
|
||||
@@ -4139,6 +4264,7 @@ def tree_decisions_cmd(
|
||||
"""Display the decision tree for a plan."""
|
||||
from cleveragents.application.container import get_container
|
||||
|
||||
_tree_cmd_start = datetime.now(UTC)
|
||||
container = get_container()
|
||||
svc = container.decision_service()
|
||||
decisions = svc.list_decisions(plan_id)
|
||||
@@ -4153,7 +4279,17 @@ def tree_decisions_cmd(
|
||||
)
|
||||
|
||||
if fmt in (OutputFormat.JSON, OutputFormat.YAML):
|
||||
console.print(format_output(tree_data, fmt))
|
||||
tree_data_dict = _build_tree_data(
|
||||
plan_id, tree_data, decisions, show_superseded, started_at=_tree_cmd_start
|
||||
)
|
||||
console.print(
|
||||
format_output(
|
||||
tree_data_dict,
|
||||
fmt,
|
||||
command="plan tree",
|
||||
messages=[{"level": "ok", "text": "Decision tree rendered"}],
|
||||
)
|
||||
)
|
||||
elif fmt == OutputFormat.TABLE:
|
||||
# Flatten for table view
|
||||
filtered = (
|
||||
|
||||
@@ -16,7 +16,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import re
|
||||
import threading
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
@@ -28,17 +28,20 @@ from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
|
||||
from cleveragents.a2a.models import A2aRequest
|
||||
from cleveragents.application.services.session_workflow import SessionWorkflow
|
||||
from cleveragents.cli.formatting import OutputFormat, format_output
|
||||
from cleveragents.core.exceptions import DatabaseError
|
||||
from cleveragents.domain.models.core.session import (
|
||||
MessageRole,
|
||||
Session,
|
||||
SessionActorNotConfiguredError,
|
||||
SessionExportError,
|
||||
SessionImportError,
|
||||
SessionMessage,
|
||||
SessionNotFoundError,
|
||||
SessionService,
|
||||
)
|
||||
from cleveragents.providers.registry import ProviderRegistry
|
||||
|
||||
# Create sub-app for session commands
|
||||
app = typer.Typer(help="Manage interactive sessions.")
|
||||
@@ -58,7 +61,7 @@ _MCP_LOGGER_NAME = "cleveragents.mcp"
|
||||
_mcp_logger_lock = threading.Lock()
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level service accessor (patchable in tests)
|
||||
# Module-level service and workflow accessors (patchable in tests)
|
||||
# ---------------------------------------------------------------------------
|
||||
_service: SessionService | None = None
|
||||
|
||||
@@ -87,11 +90,43 @@ def _reset_session_service() -> None:
|
||||
_service = None
|
||||
|
||||
|
||||
def _build_session_workflow() -> SessionWorkflow:
|
||||
"""Build a :class:`SessionWorkflow` wired to the CLI session service.
|
||||
|
||||
Returns a fully configured workflow using the same ``SessionService``
|
||||
that the CLI commands use. Tests can replace this function or its
|
||||
dependencies by patching ``_service`` (for the service layer) and
|
||||
``_get_provider_registry`` (for the LLM layer).
|
||||
"""
|
||||
service = _get_session_service()
|
||||
provider_registry = _get_provider_registry()
|
||||
return SessionWorkflow(
|
||||
session_service=service,
|
||||
provider_registry=provider_registry,
|
||||
)
|
||||
|
||||
|
||||
def _get_provider_registry() -> ProviderRegistry | None:
|
||||
"""Attempt to load the provider registry; return ``None`` on failure.
|
||||
|
||||
Isolated in its own function so tests can patch it independently
|
||||
to avoid network / credential errors in CI.
|
||||
"""
|
||||
try:
|
||||
from cleveragents.providers.registry import get_provider_registry
|
||||
|
||||
return get_provider_registry()
|
||||
except (ImportError, RuntimeError, OSError) as exc:
|
||||
_log.warning("Provider registry unavailable: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _facade_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Route an operation through the A2A local facade.
|
||||
|
||||
Returns the response data dict on success. Raises on error
|
||||
so that CLI commands can render the appropriate error message.
|
||||
Returns the response data dict on success. Domain errors are mapped
|
||||
back to their original exception types so CLI error handlers work
|
||||
correctly. Unknown / infrastructure errors raise ``RuntimeError``.
|
||||
|
||||
stdout/stderr are redirected during facade construction to prevent
|
||||
structlog or Rich output from polluting CLI output captured by
|
||||
@@ -107,9 +142,25 @@ def _facade_dispatch(operation: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
from cleveragents.a2a.cli_bootstrap import get_facade
|
||||
|
||||
facade = get_facade()
|
||||
# Wire the CLI's session service and workflow into the facade
|
||||
# so that message/send and message/stream handlers use the same
|
||||
# SessionService and SessionWorkflow as the rest of the CLI.
|
||||
# Tests that patch _get_session_service() and
|
||||
# _build_session_workflow() therefore also control the facade.
|
||||
with contextlib.suppress(Exception):
|
||||
facade.register_service("session_service", _get_session_service())
|
||||
with contextlib.suppress(Exception):
|
||||
facade.register_service("session_workflow", _build_session_workflow())
|
||||
request = A2aRequest(method=operation, params=params)
|
||||
response = facade.dispatch(request)
|
||||
if response.error is not None:
|
||||
# Map A2A error codes back to domain exceptions so CLI error
|
||||
# handlers work correctly when routing through the facade.
|
||||
# Domain exceptions that handlers explicitly re-raise
|
||||
# (SessionNotFoundError, SessionActorNotConfiguredError,
|
||||
# DatabaseError) propagate directly through the facade
|
||||
# dispatch, so they never reach this point. Any error here
|
||||
# is an unknown/infrastructure failure.
|
||||
raise RuntimeError(response.error.message)
|
||||
return dict(response.result or {})
|
||||
|
||||
@@ -855,8 +906,9 @@ def tell(
|
||||
) -> None:
|
||||
"""Send a message to a session.
|
||||
|
||||
Appends a user message and generates an assistant response. For M3, the
|
||||
actor execution is stubbed — the assistant echoes an acknowledgement.
|
||||
Appends a user message and invokes the session's bound orchestrator actor
|
||||
(or ``--actor`` override) via the A2A ``message/send`` operation.
|
||||
The actor's real response is persisted and token usage is tracked.
|
||||
|
||||
Examples:
|
||||
agents session tell --session 01HXYZ... "Hello, world"
|
||||
@@ -865,49 +917,16 @@ def tell(
|
||||
agents session tell --session 01HXYZ... --format json "Hello"
|
||||
"""
|
||||
try:
|
||||
service = _get_session_service()
|
||||
|
||||
# Append user message
|
||||
service.append_message(
|
||||
_do_session_tell(
|
||||
session_id=session_id,
|
||||
role=MessageRole.USER,
|
||||
content=prompt,
|
||||
prompt=prompt,
|
||||
actor_override=actor,
|
||||
stream=stream,
|
||||
fmt=fmt,
|
||||
)
|
||||
|
||||
# Stub actor execution: generate simple assistant response
|
||||
assistant_content = (
|
||||
f"Acknowledged: {prompt[:100]}"
|
||||
if not actor
|
||||
else f"[{actor}] Acknowledged: {prompt[:100]}"
|
||||
)
|
||||
service.append_message(
|
||||
session_id=session_id,
|
||||
role=MessageRole.ASSISTANT,
|
||||
content=assistant_content,
|
||||
)
|
||||
|
||||
data = {
|
||||
"session_id": session_id,
|
||||
"user_message": prompt,
|
||||
"assistant_message": assistant_content,
|
||||
}
|
||||
|
||||
if fmt not in (OutputFormat.RICH.value, OutputFormat.COLOR.value):
|
||||
typer.echo(format_output(data, fmt))
|
||||
return
|
||||
|
||||
if stream:
|
||||
# Simulate streaming by printing character by character
|
||||
for char in assistant_content:
|
||||
sys.stdout.write(char)
|
||||
sys.stdout.flush()
|
||||
sys.stdout.write("\n")
|
||||
else:
|
||||
from rich.markup import escape
|
||||
|
||||
console.print(f"[dim]user:[/dim] {escape(prompt)}")
|
||||
console.print(f"[cyan]assistant:[/cyan] {escape(assistant_content)}")
|
||||
|
||||
except SessionActorNotConfiguredError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1) from exc
|
||||
except SessionNotFoundError as exc:
|
||||
console.print(f"[red]Session not found:[/red] {session_id}")
|
||||
raise typer.Exit(1) from exc
|
||||
@@ -918,3 +937,95 @@ def tell(
|
||||
"Hint: run 'agents init' to initialise the database."
|
||||
)
|
||||
raise typer.Exit(1) from exc
|
||||
except Exception as exc:
|
||||
if not isinstance(exc, typer.Exit):
|
||||
_log.exception("session tell failed with unexpected error")
|
||||
console.print(
|
||||
"[red]Error:[/red] An unexpected error occurred. "
|
||||
"Check the logs for details."
|
||||
)
|
||||
raise typer.Exit(1) from exc
|
||||
raise
|
||||
|
||||
|
||||
def _do_session_tell(
|
||||
session_id: str,
|
||||
prompt: str,
|
||||
actor_override: str | None,
|
||||
stream: bool,
|
||||
fmt: OutputFormat,
|
||||
) -> None:
|
||||
"""Core implementation of ``session tell``, extracted for testability.
|
||||
|
||||
Both streaming and non-streaming paths route through
|
||||
:class:`~cleveragents.a2a.facade.A2aLocalFacade` per the spec's
|
||||
A2A protocol mapping (message/send, message/stream; addresses C3, C5).
|
||||
"""
|
||||
# Validate --actor format before passing to the workflow (m8).
|
||||
if actor_override is not None and not re.match(
|
||||
r"^[a-z0-9][a-z0-9_-]*/[a-z0-9][a-z0-9_-]*$",
|
||||
actor_override,
|
||||
):
|
||||
console.print(
|
||||
f"[red]Error:[/red] Invalid actor name {actor_override!r}. "
|
||||
"Expected format: 'namespace/name' "
|
||||
"(e.g. 'openai/gpt-4')."
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Route through A2aLocalFacade per the spec's A2A protocol mapping.
|
||||
# message/send → SessionWorkflow.tell() (non-streaming)
|
||||
# message/stream → falls back to non-streaming with streamed=false
|
||||
# (true SSE-based streaming is deferred; acceptable per the spec).
|
||||
operation = "message/stream" if stream else "message/send"
|
||||
result_dict = _facade_dispatch(
|
||||
operation,
|
||||
{
|
||||
"session_id": session_id,
|
||||
"message": prompt,
|
||||
"actor": actor_override,
|
||||
},
|
||||
)
|
||||
|
||||
assistant_content = result_dict.get("assistant_message", "")
|
||||
usage = result_dict.get("usage", {})
|
||||
|
||||
if fmt not in (OutputFormat.RICH.value, OutputFormat.COLOR.value):
|
||||
data: dict[str, object] = {
|
||||
"session_id": session_id,
|
||||
"user_message": prompt,
|
||||
"assistant_message": assistant_content,
|
||||
"usage": usage,
|
||||
}
|
||||
typer.echo(format_output(data, fmt))
|
||||
return
|
||||
|
||||
from rich.markup import escape
|
||||
|
||||
console.print(f"[dim]user:[/dim] {escape(prompt)}")
|
||||
console.print(f"[cyan]assistant:[/cyan] {escape(assistant_content)}")
|
||||
_print_usage_panel(
|
||||
input_tokens=int(usage.get("input_tokens", 0)),
|
||||
output_tokens=int(usage.get("output_tokens", 0)),
|
||||
cost=float(usage.get("cost_usd", 0.0)),
|
||||
duration_ms=float(usage.get("duration_ms", 0.0)),
|
||||
tool_calls=int(usage.get("tool_calls", 0)),
|
||||
)
|
||||
|
||||
|
||||
def _print_usage_panel(
|
||||
input_tokens: int,
|
||||
output_tokens: int,
|
||||
cost: float,
|
||||
duration_ms: float,
|
||||
tool_calls: int,
|
||||
) -> None:
|
||||
"""Render a Rich Usage panel summarising token and cost metrics."""
|
||||
lines = [
|
||||
f"[bold]Input tokens:[/bold] {input_tokens}",
|
||||
f"[bold]Output tokens:[/bold] {output_tokens}",
|
||||
f"[bold]Est. cost:[/bold] ${cost:.6f}",
|
||||
f"[bold]Duration:[/bold] {duration_ms / 1000:.1f}s",
|
||||
f"[bold]Tool calls:[/bold] {tool_calls}",
|
||||
]
|
||||
console.print(Panel("\n".join(lines), title="Usage", expand=False))
|
||||
|
||||
@@ -280,6 +280,7 @@ from cleveragents.domain.models.core.session import (
|
||||
LinkedPlan,
|
||||
MessageRole,
|
||||
Session,
|
||||
SessionActorNotConfiguredError,
|
||||
SessionExportError,
|
||||
SessionImportError,
|
||||
SessionMessage,
|
||||
@@ -506,6 +507,7 @@ __all__ = [
|
||||
"ServiceRetryPolicy",
|
||||
"ServiceRetryPolicyRegistry",
|
||||
"Session",
|
||||
"SessionActorNotConfiguredError",
|
||||
"SessionCostBudget",
|
||||
"SessionExportError",
|
||||
"SessionImportError",
|
||||
|
||||
@@ -595,6 +595,14 @@ class SessionImportError(SessionServiceError):
|
||||
"""Raised when session import fails (schema mismatch, corrupt data)."""
|
||||
|
||||
|
||||
class SessionActorNotConfiguredError(SessionServiceError):
|
||||
"""Raised when session tell is invoked with no actor configured.
|
||||
|
||||
Occurs when neither the session has a bound actor nor an ``--actor``
|
||||
override was supplied to the ``agents session tell`` command.
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Service Contract (Abstract)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -716,6 +724,20 @@ class SessionService(ABC):
|
||||
SessionImportError: If import fails (bad schema, corrupt data).
|
||||
"""
|
||||
|
||||
@abstractmethod
|
||||
def get_messages(self, session_id: str) -> list[SessionMessage]:
|
||||
"""Retrieve all messages for a session in order.
|
||||
|
||||
Args:
|
||||
session_id: The ULID of the session.
|
||||
|
||||
Returns:
|
||||
Ordered list of SessionMessage objects (by sequence number).
|
||||
|
||||
Raises:
|
||||
SessionNotFoundError: If no session with the given ID exists.
|
||||
"""
|
||||
|
||||
@abstractmethod
|
||||
def update_token_usage(
|
||||
self,
|
||||
|
||||
@@ -140,6 +140,7 @@ if TYPE_CHECKING:
|
||||
CorrectionAttemptRecord,
|
||||
)
|
||||
from cleveragents.domain.models.core.decision import Decision
|
||||
from cleveragents.domain.models.core.session import SessionMessage
|
||||
|
||||
from cleveragents.domain.models.core.correction import (
|
||||
CORRECTION_ATTEMPT_TERMINAL_STATES,
|
||||
@@ -4374,7 +4375,7 @@ class SessionMessageRepository:
|
||||
session_id: str,
|
||||
limit: int | None = None,
|
||||
offset: int = 0,
|
||||
) -> list[Any]:
|
||||
) -> list[SessionMessage]:
|
||||
"""Retrieve messages for a session with optional pagination.
|
||||
|
||||
Args:
|
||||
|
||||
@@ -77,6 +77,12 @@ class LLMToolCall(BaseModel):
|
||||
arguments: dict[str, Any] = Field(
|
||||
default_factory=dict, description="Arguments for the tool"
|
||||
)
|
||||
call_id: str = Field(
|
||||
"",
|
||||
min_length=0,
|
||||
max_length=256,
|
||||
description="Provider-assigned unique call ID for this tool call",
|
||||
)
|
||||
|
||||
model_config = ConfigDict(
|
||||
str_strip_whitespace=True,
|
||||
@@ -401,6 +407,7 @@ class ToolCallingRuntime:
|
||||
# Build result for LLM feedback
|
||||
result_dict: dict[str, Any] = {
|
||||
"tool_name": tool_call.name,
|
||||
"call_id": tool_call.call_id,
|
||||
"success": success,
|
||||
"output": output,
|
||||
}
|
||||
|
||||
@@ -446,6 +446,7 @@ dependencies = [
|
||||
{ name = "pydantic" },
|
||||
{ name = "pydantic-settings" },
|
||||
{ name = "python-ulid" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "restrictedpython" },
|
||||
{ name = "rx" },
|
||||
{ name = "structlog" },
|
||||
@@ -528,6 +529,7 @@ requires-dist = [
|
||||
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" },
|
||||
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.23.0" },
|
||||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1.0" },
|
||||
{ name = "pyyaml", specifier = ">=6.0.3" },
|
||||
{ name = "python-ulid", specifier = ">=2.7.0" },
|
||||
{ name = "radon", marker = "extra == 'dev'", specifier = ">=6.0.1" },
|
||||
{ name = "restrictedpython", specifier = ">=7.0" },
|
||||
|
||||
Reference in New Issue
Block a user