Files
cleveragents-core/robot/helper_session_cli.py
T
hurui200320 87a7ce35d7
CI / benchmark-regression (push) Has been skipped
CI / helm (push) Successful in 38s
CI / lint (push) Successful in 1m14s
CI / build (push) Successful in 1m10s
CI / push-validation (push) Successful in 49s
CI / quality (push) Successful in 1m35s
CI / typecheck (push) Successful in 1m39s
CI / security (push) Successful in 1m45s
CI / integration_tests (push) Successful in 3m44s
CI / e2e_tests (push) Successful in 4m29s
CI / unit_tests (push) Successful in 5m10s
CI / docker (push) Successful in 1m55s
CI / coverage (push) Successful in 11m8s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 1h20m21s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Failing after 1m35s
CI / docker (pull_request) Successful in 1m26s
CI / unit_tests (pull_request) Successful in 6m40s
CI / push-validation (pull_request) Successful in 1m24s
CI / quality (pull_request) Successful in 4m4s
CI / integration_tests (pull_request) Successful in 5m44s
CI / e2e_tests (pull_request) Failing after 6m5s
CI / helm (pull_request) Successful in 1m10s
CI / build (pull_request) Successful in 2m39s
CI / lint (pull_request) Successful in 2m58s
CI / typecheck (pull_request) Successful in 3m56s
CI / security (pull_request) Successful in 3m56s
CI / coverage (pull_request) Successful in 10m52s
CI / status-check (pull_request) Failing after 3s
feat(session): implement real LLM actor invocation in session tell
Replace the M3 stub in `agents session tell` with real orchestrator actor
invocation via `SessionWorkflow.tell()`. The stub echoed a canned
`Acknowledged: ...` response without calling any LLM or actor; this
commit wires up the full pipeline:

Architecture:
- New `SessionWorkflow` (Application layer) orchestrates LLM invocation
  for session tell. Accepts a `ProviderRegistry` and `ToolRegistry`;
  falls back to `FakeListLLM` when no provider is configured.
- `LangChainSessionCaller` implements the `LLMCaller` protocol, building
  history-aware LangChain message lists from the session conversation and
  invoking the LLM via `ToolCallingRuntime.run_tool_loop()`.
- `TellResult` (Pydantic BaseModel) carries the assistant response plus
  token usage (input_tokens, output_tokens, cost_usd, duration_ms).

Domain / service layer:
- `SessionActorNotConfiguredError` added to the session domain model;
  raised when tell is invoked with no actor on the session and no
  `--actor` override. Clear message; CLI exits with code 1.
- `SessionService.get_messages()` abstract method added (+ implementation
  in `PersistentSessionService`) to load ordered message history.

A2A facade:
- `A2aLocalFacade` gains `message/send` and `message/stream` standard
  A2A operation handlers that route to `SessionWorkflow.tell()`.
  Total supported operations count: 42 → 44.

CLI:
- `session tell` command delegates to `_build_session_workflow()`
  (patchable factory) instead of directly calling `SessionService`.
- Non-streaming: `SessionWorkflow.tell()` returns `TellResult`; output
  includes a Usage panel (Rich/Plain) or a `usage` object (JSON/YAML).
- Streaming: `SessionWorkflow.tell_stream()` yields tokens; CLI prints
  them via `console.print` (not raw `sys.stdout.write`).
- Token usage recorded via `SessionService.update_token_usage()` in
  both paths.

Tests:
- New Behave feature `session_tell_llm.feature` (4 scenarios): real LLM
  response persisted; streaming yields tokens; no-actor exits code 1;
  `--actor` override resolves correctly.
- New Robot suite `session_tell_llm.robot` (4 tests): end-to-end with
  stub LLM injected via monkey-patching `_resolve_llm`.
- Updated all existing `session tell` test steps to patch
  `_build_session_workflow` returning a mock `TellResult` so tests
  remain isolated from the LLM layer.
- Updated operation-count assertions (42 → 44) in
  `a2a_cli_facade_integration`, `consolidated_misc`,
  `m6_autonomy_acceptance` feature files and steps.

Cycle 7 (Review ID 8088) fixes:
- Blocker 4: Split `session_workflow.py` (was 801 lines) into two files:
  `session_workflow.py` (467 lines) and `session_caller.py` (318 lines).
  Extracted: `LangChainSessionCaller`, `extract_content`,
  `extract_token_usage`, `estimate_cost`, `estimate_tokens`,
  `history_to_langchain_messages`, and stub classes.
- Blocker 5: Route CLI streaming path through
  `_facade_dispatch("message/stream", ...)` instead of directly
  calling `workflow.tell_stream()`. The facade's
  `_handle_message_stream` falls back to non-streaming with
  `streamed: false` (acceptable for this milestone per the spec).
- Updated streaming test assertion to validate full response presence
  (no longer checks token-by-token word positions, since the facade
  fallback returns a complete message).

ISSUES CLOSED: #5784
2026-05-11 04:39:29 +00:00

405 lines
13 KiB
Python

"""Helper script for session_cli.robot smoke tests.
Each subcommand is a self-contained check that prints a sentinel on success.
"""
from __future__ import annotations
import os
import sys
import tempfile
from datetime import datetime
from pathlib import Path
from unittest.mock import MagicMock, patch
# Ensure local source tree is importable
_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from typer.testing import CliRunner # noqa: E402
from ulid import ULID # noqa: E402
from cleveragents.application.services.session_workflow import TellResult # noqa: E402
from cleveragents.cli.commands import session as session_mod # noqa: E402
from cleveragents.cli.commands.session import app as session_app # noqa: E402
from cleveragents.domain.models.core.session import ( # noqa: E402
MessageRole,
Session,
SessionMessage,
SessionNotFoundError,
SessionTokenUsage,
)
runner = CliRunner()
def _mock_session(
session_id: str | None = None,
actor_name: str | None = None,
messages: list[SessionMessage] | None = None,
) -> Session:
"""Create a test Session instance."""
return Session(
session_id=session_id or str(ULID()),
actor_name=actor_name,
namespace="local",
messages=messages or [],
token_usage=SessionTokenUsage(
input_tokens=100,
output_tokens=50,
estimated_cost=0.005,
),
created_at=datetime.now(),
updated_at=datetime.now(),
)
def _mock_message(
role: MessageRole = MessageRole.USER,
content: str = "Hello",
sequence: int = 0,
) -> SessionMessage:
return SessionMessage(
message_id=str(ULID()),
role=role,
content=content,
sequence=sequence,
timestamp=datetime.now(),
)
def _setup_service() -> MagicMock:
"""Create and install a mock service."""
svc = MagicMock()
session_mod._service = svc
return svc
def _teardown() -> None:
session_mod._service = None
# ---------------------------------------------------------------------------
# Subcommands
# ---------------------------------------------------------------------------
def create_default() -> None:
svc = _setup_service()
svc.create.return_value = _mock_session()
try:
result = runner.invoke(session_app, ["create"])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
print("session-cli-create-default-ok")
finally:
_teardown()
def create_actor() -> None:
svc = _setup_service()
svc.create.return_value = _mock_session(actor_name="openai/gpt-4")
try:
result = runner.invoke(session_app, ["create", "--actor", "openai/gpt-4"])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "openai/gpt-4" in result.output
print("session-cli-create-actor-ok")
finally:
_teardown()
def list_empty() -> None:
svc = _setup_service()
svc.list.return_value = []
try:
result = runner.invoke(session_app, ["list"])
assert result.exit_code == 0
assert "No sessions found" in result.output
print("session-cli-list-empty-ok")
finally:
_teardown()
def list_populated() -> None:
svc = _setup_service()
svc.list.return_value = [
_mock_session(actor_name="openai/gpt-4"),
_mock_session(),
]
try:
result = runner.invoke(session_app, ["list"])
assert result.exit_code == 0
assert "Sessions" in result.output
print("session-cli-list-populated-ok")
finally:
_teardown()
def show_valid() -> None:
sid = str(ULID())
svc = _setup_service()
session = _mock_session(
session_id=sid,
messages=[
_mock_message(MessageRole.USER, "Hello", 0),
_mock_message(MessageRole.ASSISTANT, "Hi there", 1),
],
)
svc.get.return_value = session
try:
result = runner.invoke(session_app, ["show", sid])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "Session Summary" in result.output
print("session-cli-show-valid-ok")
finally:
_teardown()
def show_not_found() -> None:
svc = _setup_service()
svc.get.side_effect = SessionNotFoundError("not found")
try:
result = runner.invoke(session_app, ["show", "INVALID"])
assert result.exit_code != 0
assert "Session not found" in result.output
print("session-cli-show-not-found-ok")
finally:
_teardown()
def delete_yes() -> None:
sid = str(ULID())
svc = _setup_service()
svc.get.return_value = _mock_session(session_id=sid)
svc.delete.return_value = None
try:
result = runner.invoke(session_app, ["delete", sid, "--yes"])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "deleted" in result.output
print("session-cli-delete-yes-ok")
finally:
_teardown()
def export_import_roundtrip() -> None:
"""Test export to file then import back, verifying Rich panels are rendered."""
sid = str(ULID())
svc = _setup_service()
session = _mock_session(session_id=sid, actor_name="openai/gpt-4")
svc.get.return_value = session
export_data = session.as_export_dict()
svc.export_session.return_value = export_data
fd, path = tempfile.mkstemp(suffix=".json")
os.close(fd)
os.unlink(path)
try:
# Export — verify Rich panels are rendered
result = runner.invoke(session_app, ["export", sid, "--output", path])
assert result.exit_code == 0, f"export exit={result.exit_code}: {result.output}"
assert os.path.exists(path), "Export file was not created"
assert "Session Export" in result.output, (
f"Missing 'Session Export' panel in output:\n{result.output}"
)
assert "Contents" in result.output, (
f"Missing 'Contents' panel in output:\n{result.output}"
)
assert "Integrity" in result.output, (
f"Missing 'Integrity' panel in output:\n{result.output}"
)
assert "Export completed" in result.output, (
f"Missing 'Export completed' in output:\n{result.output}"
)
# Set up import mock
imported = _mock_session(actor_name="openai/gpt-4")
svc.import_session.return_value = imported
# Import
result = runner.invoke(session_app, ["import", "--input", path])
assert result.exit_code == 0, f"import exit={result.exit_code}: {result.output}"
assert "Session Import" in result.output
assert "Validation" in result.output
assert "Merge" in result.output
assert "Import completed" in result.output
print("session-cli-export-import-roundtrip-ok")
finally:
if os.path.exists(path):
os.unlink(path)
_teardown()
def import_rich_panels() -> None:
"""Test that session import renders Session Import, Validation, and Merge panels."""
import json as _json
svc = _setup_service()
session = _mock_session(actor_name="openai/gpt-4")
imported = _mock_session(actor_name="openai/gpt-4")
svc.import_session.return_value = imported
fd, path = tempfile.mkstemp(suffix=".json")
with os.fdopen(fd, "w") as fh:
_json.dump(session.as_export_dict(), fh, default=str)
try:
result = runner.invoke(session_app, ["import", "--input", path])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "Session Import" in result.output, (
f"Missing 'Session Import' panel: {result.output}"
)
assert "Validation" in result.output, (
f"Missing 'Validation' panel: {result.output}"
)
assert "Merge" in result.output, f"Missing 'Merge' panel: {result.output}"
assert "Import completed" in result.output, (
f"Missing 'Import completed': {result.output}"
)
assert "Input:" in result.output, f"Missing 'Input:' field: {result.output}"
assert "Session ID:" in result.output, (
f"Missing 'Session ID:' field: {result.output}"
)
assert "Checksum:" in result.output, (
f"Missing 'Checksum:' field: {result.output}"
)
assert "Actor Ref:" in result.output, (
f"Missing 'Actor Ref:' field: {result.output}"
)
assert "Existing:" in result.output, (
f"Missing 'Existing:' field: {result.output}"
)
assert "Strategy:" in result.output, (
f"Missing 'Strategy:' field: {result.output}"
)
print("session-cli-import-rich-panels-ok")
finally:
if os.path.exists(path):
os.unlink(path)
_teardown()
def tell_message() -> None:
sid = str(ULID())
svc = _setup_service()
# Session must have actor_name so tell can proceed without --actor flag.
svc.get.return_value = _mock_session(session_id=sid, actor_name="openai/gpt-4")
svc.get_messages.return_value = []
# Build a mock workflow that returns a canned TellResult.
mock_wf = MagicMock()
mock_wf.tell.return_value = TellResult(
session_id=sid,
user_message="Hello",
assistant_message="Acknowledged: Hello",
input_tokens=5,
output_tokens=5,
cost=0.0,
duration_ms=0.0,
tool_calls_count=0,
)
try:
with patch(
"cleveragents.cli.commands.session._build_session_workflow",
return_value=mock_wf,
):
result = runner.invoke(session_app, ["tell", "--session", sid, "Hello"])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "Acknowledged" in result.output
print("session-cli-tell-message-ok")
finally:
_teardown()
def export_rich_panels() -> None:
"""Test that export renders all three spec-required Rich panels."""
sid = str(ULID())
svc = _setup_service()
session = _mock_session(session_id=sid, actor_name="openai/gpt-4")
svc.get.return_value = session
export_data = session.as_export_dict()
svc.export_session.return_value = export_data
fd, path = tempfile.mkstemp(suffix=".json")
os.close(fd)
os.unlink(path)
try:
result = runner.invoke(session_app, ["export", sid, "--output", path])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "Session Export" in result.output, (
f"Missing 'Session Export' panel:\n{result.output}"
)
assert "Contents" in result.output, (
f"Missing 'Contents' panel:\n{result.output}"
)
assert "Integrity" in result.output, (
f"Missing 'Integrity' panel:\n{result.output}"
)
assert "Export completed" in result.output, (
f"Missing 'Export completed':\n{result.output}"
)
assert sid in result.output, f"Session ID missing from output:\n{result.output}"
print("session-cli-export-rich-panels-ok")
finally:
if os.path.exists(path):
os.unlink(path)
_teardown()
def export_stdout_rich_panels() -> None:
"""Test that stdout export also renders Rich panels."""
sid = str(ULID())
svc = _setup_service()
session = _mock_session(session_id=sid, actor_name="openai/gpt-4")
svc.get.return_value = session
export_data = session.as_export_dict()
svc.export_session.return_value = export_data
try:
result = runner.invoke(session_app, ["export", sid])
assert result.exit_code == 0, f"exit={result.exit_code}: {result.output}"
assert "Session Export" in result.output, (
f"Missing 'Session Export' panel:\n{result.output}"
)
assert "(stdout)" in result.output, (
f"Missing '(stdout)' indicator:\n{result.output}"
)
assert "Export completed" in result.output, (
f"Missing 'Export completed':\n{result.output}"
)
print("session-cli-export-stdout-rich-panels-ok")
finally:
_teardown()
# ---------------------------------------------------------------------------
# Dispatcher
# ---------------------------------------------------------------------------
_COMMANDS: dict[str, object] = {
"create-default": create_default,
"create-actor": create_actor,
"list-empty": list_empty,
"list-populated": list_populated,
"show-valid": show_valid,
"show-not-found": show_not_found,
"delete-yes": delete_yes,
"export-import-roundtrip": export_import_roundtrip,
"import-rich-panels": import_rich_panels,
"export-rich-panels": export_rich_panels,
"export-stdout-rich-panels": export_stdout_rich_panels,
"tell-message": tell_message,
}
if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
print(f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>", file=sys.stderr)
sys.exit(1)
cmd = _COMMANDS[sys.argv[1]]
cmd() # type: ignore[operator]