cli/session: add BDD tests for --format flag and JSON envelope output to session tell #11148
@@ -198,6 +198,13 @@ ensuring data is stored with proper parameter values.
|
||||
(use → execute → apply) with a `WF18 Test Teardown` keyword for diagnostic
|
||||
logging on failure.
|
||||
|
||||
- **`agents session tell --format` outputs JSON envelope** (#10466): Added the
|
||||
`--format`/`-f` flag to `agents session tell`, enabling machine-readable output
|
||||
(JSON, YAML, plain, table) alongside the existing Rich console text. When
|
||||
non-rich formats are selected the response is wrapped in the spec-required JSON
|
||||
envelope containing `command`, `status`, `exit_code`, `data`, `timing`, and
|
||||
`messages` fields. The default `rich` output path is unchanged — no regression.
|
||||
|
||||
- **`agents session tell` invokes real LLM orchestrator actor** (#5784): Replaced the
|
||||
M3 echo-stub with real actor invocation via `SessionWorkflow`, routing through
|
||||
`LangChainSessionCaller` → `ToolCallingRuntime.run_tool_loop()`. The user prompt
|
||||
|
||||
@@ -114,3 +114,4 @@ Below are some specific details of individual PR contributions.
|
||||
* HAL 9000 has contributed the ACMS execute phase ContextAssemblyPipeline wiring (PR #10027): replaces the base ``ACMSPipeline`` default with ``ContextAssemblyPipeline`` in ``ACMSExecutePhaseContextAssembler``, enabling production Phase 1 components (confidence-weighted strategy selection, proportional budget allocation, parallel execution with circuit breaking) and per-stage timing instrumentation by default. Includes Behave test coverage verifying the default pipeline type.
|
||||
* HAL 9000 has contributed the path containment security hardening fix (PR #7801 / issue #7478): replaced insecure ``str.startswith(root + "/")`` string-prefix path containment checks with semantic ``os.path.relpath`` comparisons in ``tool/path_mapper.py`` (_is_under) and ``application/services/llm_actors.py`` (_write_to_sandbox), eliminating the sibling-directory prefix-collision path traversal bypass vulnerability.
|
||||
* HAL 9000 has contributed the data-integrity fix for ProjectRepository (#8179): removed unconditional ``session.rollback()`` calls from exception handlers in ``ProjectRepository.create()`` and ``NamespacedProjectRepository.create/update/delete``, delegating transaction rollback to the Unit of Work outer-layer handler where it belongs.
|
||||
* Jeffrey Phillips Freeman has contributed the `--format`/`-f` flag to `agents session tell` (issue #10466): adds JSON envelope output for machine-readable workflows alongside existing Rich console output, with Behave BDD test coverage verifying all four non-rich format paths (JSON, YAML, plain, table) and the short `-f` flag alias.
|
||||
|
||||
@@ -252,3 +252,48 @@ Feature: Session CLI commands
|
||||
When I run session CLI tell to a non-existent session
|
||||
Then the session CLI should exit with error
|
||||
And the session CLI output should contain "Session not found"
|
||||
|
||||
# Tell command format flag tests (issue #10466)
|
||||
@format_flag
|
||||
Scenario: Tell with JSON format outputs spec-compliant envelope
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with --format json and prompt "What files were changed?"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI output should be valid JSON
|
||||
And the session CLI tell JSON should contain a data envelope
|
||||
|
||||
@format_flag
|
||||
Scenario: Tell YAML format outputs structured output
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with --format yaml and prompt "Hello"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI tell output should be valid YAML
|
||||
|
||||
@format_flag
|
||||
Scenario: Tell plain format outputs key-value lines
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with --format plain and prompt "Test"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI tell output should contain "session_id:"
|
||||
|
||||
@format_flag
|
||||
Scenario: Tell table format outputs ASCII table
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with --format table and prompt "Table test"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI tell output should contain a table border character
|
||||
|
||||
@format_flag
|
||||
Scenario: Tell short flag -f works the same as --format
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with short format flag json and prompt "Short flag"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI output should be valid JSON
|
||||
|
||||
@format_flag
|
||||
Scenario: Tell default Rich output has no regression
|
||||
Given there is a mocked session for tell
|
||||
When I run session CLI tell with a prompt "Rich output test"
|
||||
Then the session CLI tell should succeed
|
||||
And the session CLI output should contain "user:"
|
||||
And the session CLI output should contain "assistant:"
|
||||
|
||||
@@ -809,3 +809,147 @@ def step_exit_with_error(context: Context) -> None:
|
||||
f"Expected non-zero exit code, got {context.result.exit_code}: "
|
||||
f"{context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tell format flag step definitions (issue #10466)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I run session CLI tell with --format json and prompt "{prompt}"')
|
||||
def step_tell_with_format_json(context: Context, prompt: str) -> None:
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Response: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "--format", "json", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I run session CLI tell with --format yaml and prompt "{prompt}"')
|
||||
def step_tell_with_format_yaml(context: Context, prompt: str) -> None:
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Response: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "--format", "yaml", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I run session CLI tell with --format plain and prompt "{prompt}"')
|
||||
def step_tell_with_format_plain(context: Context, prompt: str) -> None:
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Response: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "--format", "plain", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I run session CLI tell with --format table and prompt "{prompt}"')
|
||||
def step_tell_with_format_table(context: Context, prompt: str) -> None:
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Response: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "--format", "table", prompt],
|
||||
)
|
||||
|
||||
|
||||
@when('I run session CLI tell with short format flag {fmt} and prompt "{prompt}"')
|
||||
def step_tell_with_short_format_flag(context: Context, fmt: str, prompt: str) -> None:
|
||||
"""Run tell with the -f shorthand for --format."""
|
||||
context.mock_workflow.tell.return_value = TellResult(
|
||||
session_id=context.session_id,
|
||||
user_message=prompt,
|
||||
assistant_message=f"Response: {prompt}",
|
||||
input_tokens=len(prompt) // 4,
|
||||
output_tokens=5,
|
||||
cost=0.0,
|
||||
duration_ms=0.0,
|
||||
tool_calls_count=0,
|
||||
)
|
||||
context.result = context.runner.invoke(
|
||||
session_app,
|
||||
["tell", "--session", context.session_id, "-f", fmt, prompt],
|
||||
)
|
||||
|
||||
|
||||
@then("the session CLI tell JSON should contain a data envelope")
|
||||
def step_tell_json_has_envelope(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}: {context.result.output}"
|
||||
)
|
||||
parsed = json.loads(context.result.output)
|
||||
envelope_keys = {"command", "status", "exit_code", "data", "timing", "messages"}
|
||||
assert envelope_keys.issubset(parsed.keys()), (
|
||||
f"JSON output missing envelope keys: {envelope_keys - set(parsed.keys())}. "
|
||||
f"Got keys: {parsed.keys()}"
|
||||
)
|
||||
|
||||
|
||||
@then("the session CLI tell output should be valid YAML")
|
||||
def step_tell_output_valid_yaml(context: Context) -> None:
|
||||
import yaml
|
||||
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}: {context.result.output}"
|
||||
)
|
||||
try:
|
||||
yaml.safe_load(context.result.output)
|
||||
except yaml.YAMLError as exc:
|
||||
raise AssertionError(f"Output is not valid YAML: {exc}") from exc
|
||||
|
||||
|
||||
@then('the session CLI tell output should contain "session_id:"')
|
||||
def step_tell_plain_contains_session_id(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}: {context.result.output}"
|
||||
)
|
||||
assert "session_id:" in context.result.output, (
|
||||
f"Expected 'session_id:' in plain output, got: {context.result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the session CLI tell output should contain a table border character")
|
||||
def step_tell_table_has_border(context: Context) -> None:
|
||||
assert context.result.exit_code == 0, (
|
||||
f"Expected exit code 0, got {context.result.exit_code}: {context.result.output}"
|
||||
)
|
||||
# ASCII table uses border characters like | and +-+
|
||||
has_border = "|" in context.result.output or "+" in context.result.output
|
||||
assert has_border, (
|
||||
f"Expected table border character in output, got: {context.result.output}"
|
||||
)
|
||||
|
||||
@@ -23,6 +23,7 @@ from io import StringIO
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
from rich import box
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
@@ -130,8 +131,10 @@ def _format_table(data: dict[str, Any] | list[dict[str, Any]]) -> str:
|
||||
if key not in columns:
|
||||
columns.append(key)
|
||||
|
||||
# Build Rich table and capture to string
|
||||
table = Table(show_header=True, show_edge=True)
|
||||
# Build Rich table and capture to string. Use ASCII box style so the
|
||||
# output uses '|' and '+' border characters (matches the docstring
|
||||
# contract "ASCII table" and what `--format table` callers expect).
|
||||
table = Table(show_header=True, show_edge=True, box=box.ASCII)
|
||||
for col in columns:
|
||||
table.add_column(col)
|
||||
for row in rows:
|
||||
|
||||
Reference in New Issue
Block a user