fix(cli): replace one-liner with structured Rich panel in validation attach output
CI / lint (pull_request) Successful in 50s
CI / typecheck (pull_request) Successful in 1m10s
CI / build (pull_request) Successful in 44s
CI / push-validation (pull_request) Successful in 26s
CI / security (pull_request) Successful in 1m23s
CI / quality (pull_request) Successful in 1m15s
CI / helm (pull_request) Successful in 47s
CI / integration_tests (pull_request) Successful in 3m55s
CI / e2e_tests (pull_request) Failing after 4m16s
CI / unit_tests (pull_request) Successful in 5m18s
CI / docker (pull_request) Failing after 0s
CI / coverage (pull_request) Successful in 11m16s
CI / status-check (pull_request) Failing after 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 1h5m12s

Replaces the simple one-liner output in the agents validation attach
command with a structured Rich Panel titled "Validation Attached"
containing Attachment ID, Validation, Mode, Resource, and Scope fields,
per spec lines 9572-9640.

Direct (unscoped) attachments now show an informational note:
"This validation will run for ALL plans/projects that access this resource."

Adds a footer message: "✓ OK Validation attached"

Also adds:
- BDD TDD regression test (features/tdd_validation_attach_rich_panel.feature)
  tagged @tdd_issue @tdd_issue_1423
- Robot Framework integration test (robot/validation_attach_rich_output.robot)
- Updates existing tool_cli.feature assertion to check for "Validation Attached"
- Updates CHANGELOG.md

ISSUES CLOSED: #1423
This commit is contained in:
2026-04-24 10:57:56 +00:00
parent 663a6d2397
commit 49bc440f73
7 changed files with 325 additions and 5 deletions
+5
View File
@@ -5,6 +5,11 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Fixed
- **agents validation attach output now uses structured Rich Panel** (#1423): Replaced the one-liner output with a Rich Panel titled Validation Attached containing Attachment ID, Validation, Mode, Resource, and Scope fields. Direct (unscoped) attachments now show an informational note.
### Changed
- **Diagnostics spec examples expanded to all 9 providers** (#5320): Updated the
@@ -0,0 +1,101 @@
"""Step definitions for TDD bug #1423.
agents validation attach output is a one-liner instead of structured panel.
This test captures bug #1423. The specification
(docs/specification.md lines 9572-9640) states that the
agents validation attach command should output a structured Rich Panel.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.cli.commands.validation import app as validation_app
_runner = CliRunner()
_PATCH_SVC = "cleveragents.cli.commands.validation._get_tool_registry_service"
def _make_attachment(
attachment_id: str = "01HXM5A1B2C3D4E5F6G7H8J9K0",
validation_name: str = "local/run-tests",
resource_id: str = "local/api-repo",
mode: str = "required",
project_name: str | None = None,
plan_id: str | None = None,
) -> dict[str, Any]:
return {
"attachment_id": attachment_id,
"validation_name": validation_name,
"resource_id": resource_id,
"mode": mode,
"project_name": project_name,
"plan_id": plan_id,
"created_at": "2026-01-01T00:00:00",
}
@given("a tdd 1423 CLI test runner with mocked services")
def step_tdd_1423_background(context: Context) -> None:
context.tdd1423_runner = _runner
context.tdd1423_result = None
context.tdd1423_mock_service = MagicMock()
context.tdd1423_attachment = _make_attachment(project_name="myproject")
context.tdd1423_mock_service.attach_validation.return_value = (
context.tdd1423_attachment
)
@given("a tdd 1423 mocked validation attachment with project scope")
def step_tdd_1423_attachment_project(context: Context) -> None:
context.tdd1423_attachment = _make_attachment(project_name="myproject")
context.tdd1423_mock_service.attach_validation.return_value = (
context.tdd1423_attachment
)
@given("a tdd 1423 mocked validation attachment without scope")
def step_tdd_1423_attachment_no_scope(context: Context) -> None:
context.tdd1423_attachment = _make_attachment()
context.tdd1423_mock_service.attach_validation.return_value = (
context.tdd1423_attachment
)
@given("a tdd 1423 mocked validation attachment with plan scope")
def step_tdd_1423_attachment_plan(context: Context) -> None:
context.tdd1423_attachment = _make_attachment(plan_id="plan-123")
context.tdd1423_mock_service.attach_validation.return_value = (
context.tdd1423_attachment
)
@when("I tdd 1423 invoke validation attach")
def step_tdd_1423_invoke_attach(context: Context) -> None:
with patch(_PATCH_SVC, return_value=context.tdd1423_mock_service):
context.tdd1423_result = context.tdd1423_runner.invoke(
validation_app,
["attach", "local/api-repo", "local/run-tests"],
)
@then("the tdd 1423 CLI result should succeed")
def step_tdd_1423_result_succeed(context: Context) -> None:
result = context.tdd1423_result
assert result is not None, "No CLI result captured"
assert result.exit_code == 0, (
f"Expected exit code 0 but got {result.exit_code}. Output: {result.output}"
)
@then('the tdd 1423 output should contain "{text}"')
def step_tdd_1423_output_contains(context: Context, text: str) -> None:
result = context.tdd1423_result
assert result is not None, "No CLI result captured"
assert text in result.output, f"Expected '{text}' in output. Got: {result.output}"
@@ -0,0 +1,56 @@
# TDD bug-capture test for bug #1423.
#
# The specification (docs/specification.md lines 9572-9640) states that the
# agents validation attach command should output a structured Rich Panel.
#
# Bug #1423 reported that the output was a simple one-liner instead of
# the spec-required structured panel.
#
# The fix replaces the one-liner with a Rich Panel titled Validation Attached.
@tdd_issue @tdd_issue_1423
Feature: Bug #1423 -- validation attach output is a one-liner instead of structured panel
As a user of the CleverAgents CLI
I want the agents validation attach command to output a structured Rich Panel
So that the output matches the spec format at lines 9572-9640
Background:
Given a tdd 1423 CLI test runner with mocked services
Scenario: Attach validation shows structured panel with all required fields
Given a tdd 1423 mocked validation attachment with project scope
When I tdd 1423 invoke validation attach
Then the tdd 1423 CLI result should succeed
And the tdd 1423 output should contain "Validation Attached"
And the tdd 1423 output should contain "Attachment ID:"
And the tdd 1423 output should contain "Validation:"
And the tdd 1423 output should contain "Mode:"
And the tdd 1423 output should contain "Resource:"
And the tdd 1423 output should contain "Scope:"
Scenario: Attach validation without scope shows direct attachment note
Given a tdd 1423 mocked validation attachment without scope
When I tdd 1423 invoke validation attach
Then the tdd 1423 CLI result should succeed
And the tdd 1423 output should contain "Validation Attached"
And the tdd 1423 output should contain "direct (always active)"
And the tdd 1423 output should contain "ALL plans/projects"
Scenario: Attach validation shows OK footer message
Given a tdd 1423 mocked validation attachment with project scope
When I tdd 1423 invoke validation attach
Then the tdd 1423 CLI result should succeed
And the tdd 1423 output should contain "OK"
And the tdd 1423 output should contain "Validation attached"
Scenario: Attach validation with project scope shows project in scope field
Given a tdd 1423 mocked validation attachment with project scope
When I tdd 1423 invoke validation attach
Then the tdd 1423 CLI result should succeed
And the tdd 1423 output should contain "project myproject"
Scenario: Attach validation with plan scope shows plan in scope field
Given a tdd 1423 mocked validation attachment with plan scope
When I tdd 1423 invoke validation attach
Then the tdd 1423 CLI result should succeed
And the tdd 1423 output should contain "plan plan-123"
+1 -1
View File
@@ -195,7 +195,7 @@ Feature: Tool and Validation CLI commands
Given a mocked validation exists for attaching
When I run validation CLI attach "resource/r1" "local/test-val"
Then the validation CLI attach should succeed
And the validation CLI output should contain "Attached validation"
And the validation CLI output should contain "Validation Attached"
Scenario: Attach validation with project scope
Given a mocked validation exists for attaching
@@ -0,0 +1,105 @@
"""Helper script for validation_attach_rich_output.robot integration tests.
Tests that agents validation attach outputs a structured Rich Panel
per spec lines 9572-9640 (bug #1423 regression guard).
"""
from __future__ import annotations
import sys
from collections.abc import Callable
from pathlib import Path
from unittest.mock import MagicMock, patch
_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from helpers_common import reset_global_state # noqa: E402
from typer.testing import CliRunner # noqa: E402
from cleveragents.cli.commands.validation import app as validation_app # noqa: E402
runner = CliRunner()
_ATTACHMENT_ULID = "01HXM5A1B2C3D4E5F6G7H8J9K0"
def attach_rich_panel() -> None:
"""Verify that validation attach outputs a structured Rich Panel."""
mock_svc = MagicMock()
mock_attachment = {
"attachment_id": _ATTACHMENT_ULID,
"validation_name": "local/run-tests",
"resource_id": "local/api-repo",
"mode": "required",
"project_name": "myproject",
"plan_id": None,
"created_at": "2026-01-01T00:00:00",
}
mock_svc.attach_validation.return_value = mock_attachment
with patch(
"cleveragents.cli.commands.validation._get_tool_registry_service",
return_value=mock_svc,
):
result = runner.invoke(
validation_app,
["attach", "local/api-repo", "local/run-tests"],
)
required_fields = [
"Validation Attached",
"Attachment ID:",
"Validation:",
"Mode:",
"Resource:",
"Scope:",
"project myproject",
]
if result.exit_code == 0 and all(f in result.output for f in required_fields):
print("validation-attach-rich-panel-ok")
else:
print(f"FAIL: exit={result.exit_code} output={result.output!r}")
sys.exit(1)
def attach_direct_scope() -> None:
"""Verify that direct (unscoped) attachment shows the note."""
mock_svc = MagicMock()
mock_attachment = {
"attachment_id": _ATTACHMENT_ULID,
"validation_name": "local/run-tests",
"resource_id": "local/api-repo",
"mode": "required",
"project_name": None,
"plan_id": None,
"created_at": "2026-01-01T00:00:00",
}
mock_svc.attach_validation.return_value = mock_attachment
with patch(
"cleveragents.cli.commands.validation._get_tool_registry_service",
return_value=mock_svc,
):
result = runner.invoke(
validation_app,
["attach", "local/api-repo", "local/run-tests"],
)
required = ["Validation Attached", "direct (always active)", "ALL plans/projects"]
if result.exit_code == 0 and all(f in result.output for f in required):
print("validation-attach-direct-scope-ok")
else:
print(f"FAIL: exit={result.exit_code} output={result.output!r}")
sys.exit(1)
_COMMANDS: dict[str, Callable[[], None]] = {
"attach-rich-panel": attach_rich_panel,
"attach-direct-scope": attach_direct_scope,
}
if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
print(f"Usage: {sys.argv[0]} <{chr(124).join(_COMMANDS)}>")
sys.exit(1)
reset_global_state()
fn = _COMMANDS[sys.argv[1]]
fn()
+28
View File
@@ -0,0 +1,28 @@
*** Settings ***
Documentation Integration tests for the validation attach rich panel output.
... Verifies that agents validation attach outputs a structured Rich Panel
... per spec lines 9572-9640 (bug #1423 regression guard).
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment With Database Isolation
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER} ${CURDIR}/helper_validation_attach_rich_output.py
*** Test Cases ***
Validation Attach Shows Structured Rich Panel
[Documentation] agents validation attach must output a structured Rich Panel
... with Attachment ID, Validation, Mode, Resource, Scope fields.
${result}= Run Process ${PYTHON} ${HELPER} attach-rich-panel cwd=${WORKSPACE} timeout=120s on_timeout=kill
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} validation-attach-rich-panel-ok
Validation Attach Direct Scope Shows Note
[Documentation] agents validation attach without scope shows direct attachment note.
${result}= Run Process ${PYTHON} ${HELPER} attach-direct-scope cwd=${WORKSPACE} timeout=120s on_timeout=kill
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} validation-attach-direct-scope-ok
+29 -4
View File
@@ -351,10 +351,35 @@ def attach(
return
att_id = att_data.get("attachment_id", "")
console.print(
f"[green]Attached validation:[/green] {validation_name} -> "
f"{resource} (id: {att_id})"
)
mode = att_data.get("mode", "required")
project_name = att_data.get("project_name")
plan_id_val = att_data.get("plan_id")
# Determine scope string per spec lines 9572-9640
if project_name:
scope = f"project {project_name}"
elif plan_id_val:
scope = f"plan {plan_id_val}"
else:
scope = "direct (always active)"
panel_lines = [
f"[bold]Attachment ID:[/bold] {att_id}",
f"[bold]Validation:[/bold] {validation_name}",
f"[bold]Mode:[/bold] {mode}",
f"[bold]Resource:[/bold] {resource}",
f"[bold]Scope:[/bold] {scope}",
]
if not project_name and not plan_id_val:
panel_lines.append(
"[dim]This validation will run for ALL plans/projects\n"
"that access this resource.[/dim]"
)
panel_content = "\n".join(panel_lines)
console.print(Panel(panel_content, title="Validation Attached", expand=False))
console.print("[green]✓ OK[/green] Validation attached")
except NotFoundError as exc:
console.print(f"[red]Validation not found:[/red] {validation_name}")