test(plan-tree): add failing BDD scenario proving corrected nodes not visually marked #8671
+6
-1
@@ -497,7 +497,12 @@ _ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
- **Quick Start Guide** (PR #9245): Added `docs/quickstart.md` with an end-to-end quick start guide covering prerequisites, installation, project creation, resource registration, plan/apply workflow, and troubleshooting. Updated `mkdocs.yml` navigation to include the Quick Start page.
|
||||
|
||||
- **Plan checkpoint management CLI commands** (#8683): Added `agents plan checkpoint-list <plan-id>` and `agents plan checkpoint-delete <checkpoint-id>` commands. Listing output now highlights checkpoint ID, type, created timestamp, reason, phase, and decision linkage with a concise field summary footer across rich/table/json/yaml formats. Deletion supports batch IDs, interactive confirmation (skip with `--yes`), and structured JSON/YAML responses for automation-friendly scripting.
|
||||
- **Invariant Remove CLI Command** (#8530): Implemented `agents invariant remove <id>` command that soft-deletes an invariant by ID. The command displays a confirmation prompt before removal (bypassable with `--yes`/`-y`), outputs the removed invariant ID on success, and shows a clear error message when the invariant ID does not exist. Supports `--format` flag for JSON and YAML output. Full BDD test coverage and Robot Framework integration tests included.
|
||||
- **Invariant Remove CLI Command** (#8530): Implemented `agents invariant remove <id>` command that soft-deletes an invariant by ID. The command displays a confirmation prompt before removal (bypassable with `--yes`/ `-y`), outputs the removed invariant ID on success, and shows a clear error message when the invariant ID does not exist. Supports `--format` flag for JSON and YAML output. Full BDD test coverage and Robot Framework integration tests included.
|
||||
- **TDD: plan tree does not visually mark corrected nodes** (#8576): Added a failing
|
||||
BDD scenario proving that corrected nodes (decisions with `is_correction=True`) are
|
||||
not visually distinguished in the `agents plan tree` output. The scenario is tagged
|
||||
`@tdd_expected_fail` and will pass (by inversion) until the underlying gap described
|
||||
in Spec Requirement #7 is fixed.
|
||||
- **TDD: MCPToolAdapter.infer_resource_slots() TypeError with null properties** (#10470):
|
||||
Added a TDD issue-capture Behave scenario that reproduces the bug where
|
||||
`MCPToolAdapter.infer_resource_slots()` raises `TypeError` when the input schema
|
||||
|
||||
@@ -68,3 +68,4 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed the plan correct JSON output envelope fix (PR #8662 / issue #8584): restructured `agents plan correct --format json` output to nest correction fields under `data.correction` and pass `command="plan correct"` to `format_output`, producing the spec-required CLI envelope. Added three BDD scenarios validating `data.correction.mode` (revert and append modes) and the `command` field.
|
||||
* HAL 9000 has contributed BDD feature file tag coverage improvements (#9124 / pr #9183): added required `@a2a`, `@session`, and `@cli` Gherkin tags to 30 feature files (8 A2A, 7 session, 15 CLI) to enable selective tag-based test filtering via `behave --tags=a2a,session,cli`.
|
||||
* HAL 9000 has contributed the plan tree JSON `decision_id` fix (#9096): updated `step_tree_json_valid` in features/steps/plan_explain_steps.py to correctly handle the {"data": [...]} envelope structure produced by format_output, and removed @tdd_expected_fail from the @tdd_issue_4254 scenario so it runs as a permanent regression guard.
|
||||
* HAL 9000 has contributed the TDD scenario for plan tree correction visual marking (PR #8671 / issue #8576): added a failing BDD scenario proving that corrected nodes (decisions with is_correction=True) are not visually distinguished in the plan tree output, formalizing Spec Requirement #7 as an executable specification.
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Step definitions for tdd_plan_tree_correction_visual.feature.
|
||||
|
||||
Tests that corrected nodes (decisions with is_correction=True) are visually
|
||||
marked in the plan tree output. This is a TDD scenario documenting the gap
|
||||
where the current implementation does NOT visually distinguish corrected nodes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from ulid import ULID
|
||||
|
||||
from cleveragents.cli.commands.plan import build_decision_tree
|
||||
|
|
||||
from cleveragents.domain.models.core.decision import Decision, DecisionType
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PLAN_ID = str(ULID())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a set of test decisions with a corrected decision")
|
||||
def step_decisions_with_correction(context: Context) -> None:
|
||||
"""Create a set of decisions where one is marked as a correction."""
|
||||
root_id = str(ULID())
|
||||
corrected_id = str(ULID())
|
||||
corrects_id = str(ULID())
|
||||
|
||||
context.tdd_decisions = [
|
||||
# Root decision
|
||||
Decision(
|
||||
decision_id=root_id,
|
||||
plan_id=_PLAN_ID,
|
||||
sequence_number=0,
|
||||
decision_type=DecisionType.PROMPT_DEFINITION,
|
||||
question="What to build?",
|
||||
chosen_option="REST API",
|
||||
),
|
||||
# Original decision that will be corrected
|
||||
Decision(
|
||||
decision_id=corrects_id,
|
||||
plan_id=_PLAN_ID,
|
||||
parent_decision_id=root_id,
|
||||
sequence_number=1,
|
||||
decision_type=DecisionType.STRATEGY_CHOICE,
|
||||
question="Which framework?",
|
||||
chosen_option="Django",
|
||||
),
|
||||
# Corrected decision (is_correction=True)
|
||||
Decision(
|
||||
decision_id=corrected_id,
|
||||
plan_id=_PLAN_ID,
|
||||
parent_decision_id=root_id,
|
||||
sequence_number=2,
|
||||
decision_type=DecisionType.STRATEGY_CHOICE,
|
||||
question="Which framework?",
|
||||
chosen_option="FastAPI",
|
||||
is_correction=True,
|
||||
corrects_decision_id=corrects_id,
|
||||
),
|
||||
]
|
||||
context.tdd_corrected_id = corrected_id
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I build the correction TDD test decision tree")
|
||||
def step_build_tree_default(context: Context) -> None:
|
||||
"""Build the decision tree from the test decisions."""
|
||||
context.tdd_tree_data = build_decision_tree(
|
||||
context.tdd_decisions,
|
||||
show_superseded=False,
|
||||
max_depth=0,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the tree should include a visual marker for the corrected decision")
|
||||
def step_tree_has_correction_marker(context: Context) -> None:
|
||||
"""Verify that the tree data includes a marker for the corrected decision."""
|
||||
tree_nodes: list[dict[str, object]] = context.tdd_tree_data
|
||||
|
||||
# The tree_data is a list of root-level node dicts.
|
||||
# We need to search through it to find the corrected decision node
|
||||
# and verify it has a visual marker.
|
||||
|
||||
def find_corrected_node(
|
||||
nodes: list[dict[str, object]],
|
||||
) -> dict[str, object] | None:
|
||||
"""Recursively search for the corrected decision node."""
|
||||
for node in nodes:
|
||||
if node.get("decision_id") == context.tdd_corrected_id:
|
||||
return node
|
||||
children = node.get("children", [])
|
||||
assert isinstance(children, list)
|
||||
result = find_corrected_node(children)
|
||||
if result is not None:
|
||||
return result
|
||||
return None
|
||||
|
||||
corrected_node = find_corrected_node(tree_nodes)
|
||||
assert corrected_node is not None, (
|
||||
f"Corrected decision {context.tdd_corrected_id} not found in tree"
|
||||
)
|
||||
|
||||
# Store the node for the next assertion
|
||||
context.tdd_corrected_node = corrected_node
|
||||
|
||||
|
||||
@then(
|
||||
'the corrected decision label should contain "[corrected]" or "✎" or similar marker'
|
||||
)
|
||||
def step_corrected_label_has_marker(context: Context) -> None:
|
||||
"""Verify that the corrected decision's label contains a visual marker.
|
||||
|
||||
This assertion is expected to FAIL (via @tdd_expected_fail) because
|
||||
build_decision_tree does not currently include a 'label' key in its
|
||||
node dicts, and even if it did, it does not add visual markers for
|
||||
corrected decisions (is_correction=True). This TDD scenario documents
|
||||
the gap described in Spec Requirement #7.
|
||||
"""
|
||||
corrected_node = context.tdd_corrected_node
|
||||
|
||||
# The label should be in the node's label field.
|
||||
# build_decision_tree currently does NOT include a 'label' key in its
|
||||
# node dicts -- this assertion will fail with AssertionError, proving
|
||||
# the bug exists (Spec Requirement #7: corrected nodes must be visually
|
||||
# marked in the plan tree output).
|
||||
label = str(corrected_node.get("label", ""))
|
||||
|
||||
# Check for common visual markers for corrections
|
||||
markers = ["[corrected]", "\u270e", "[correction]", "\u270f", "[fix]", "[amended]"]
|
||||
|
||||
has_marker = any(marker in label for marker in markers)
|
||||
|
||||
assert has_marker, (
|
||||
f"Corrected decision label '{label}' does not contain any visual marker. "
|
||||
f"Expected one of: {markers}. "
|
||||
f"build_decision_tree does not currently add visual markers for "
|
||||
f"decisions with is_correction=True (Spec Requirement #7 gap)."
|
||||
)
|
||||
@@ -0,0 +1,23 @@
|
||||
@tdd_expected_fail
|
||||
@tdd_issue
|
||||
@tdd_issue_8576
|
||||
Feature: TDD: plan tree visually marks corrected nodes
|
||||
Spec Requirement #7 states that `agents plan tree` should visually
|
||||
distinguish corrected nodes (decisions with is_correction=True).
|
||||
|
||||
This TDD scenario documents the gap: the current implementation does NOT
|
||||
visually mark corrected nodes in the tree output. The tree renderer builds
|
||||
node labels without checking decision.is_correction.
|
||||
|
||||
Expected behavior: Corrected nodes should be marked with a visual indicator
|
||||
such as [corrected], ✎, or similar annotation.
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Scenario: Corrected node is visually marked in tree output
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
Scenario: Corrected decision is visually marked in plan tree
|
||||
Given a set of test decisions with a corrected decision
|
||||
When I build the correction TDD test decision tree
|
||||
Then the tree should include a visual marker for the corrected decision
|
||||
And the corrected decision label should contain "[corrected]" or "✎" or similar marker
|
||||
Reference in New Issue
Block a user
BLOCKER — CI / unit_tests Failing
This step imports
build_decision_treefromcleveragents.cli.commands.plan. If this import triggers a runtime error (e.g., a missing dependency or side effect in the large CLI module), the step will fail with a non-AssertionErrorexception — bypassing the@tdd_expected_failinversion mechanism inenvironment.py, which only invertsAssertionError.To diagnose:
nox -s unit_tests -- features/tdd_plan_tree_correction_visual.featurelocallyImportErrororAttributeError, trace the import chain and fix the root causeOnce the exact failure is known and resolved, CI should pass because the scenario logic itself is sound —
_node_dict()has nolabelkey, sostr(corrected_node.get("label", ""))returns"",has_markerisFalse, andAssertionErroris raised (which IS inverted to a pass).Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker
BLOCKER — CI / unit_tests Failing: Likely Non-
AssertionErrorException Preventing TDD InversionThis module-level import:
pulls in
src/cleveragents/cli/commands/plan.py, a ~5000-line CLI module with heavy module-level initialization (get_container,SQLAlchemyError,A2aRequest,GitWorktreeSandbox, etc.). In the Behave CI test environment, this import may raise a non-AssertionErrorexception (e.g.,ImportError,ModuleNotFoundError,RuntimeError).The
apply_tdd_inversionfunction infeatures/environment.pyexplicitly skips inversion for any exception that is not anAssertionError:This means a module-level import failure causes the test to report as a hard CI failure instead of an expected-fail pass.
Diagnosis:
nox -s unit_tests -- features/tdd_plan_tree_correction_visual.featurelocally to reproduceSuggested fix — defer the import inside the When step:
A deferred import ensures the exception (if any) occurs inside the step and provides a better diagnostic. Longer-term, consider moving
build_decision_treeto a lighter domain-layer module (e.g.,cleveragents.domain.services.plan_tree) so test files can import it without loading the entire CLI module.Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker