fix(tdd): restore tdd_expected_fail tags and fix Robot helper inversion
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 21s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 42s
CI / unit_tests (pull_request) Successful in 2m35s
CI / docker (pull_request) Successful in 8s
CI / integration_tests (pull_request) Successful in 3m30s
CI / coverage (pull_request) Successful in 5m17s
CI / benchmark-regression (pull_request) Successful in 35m10s

Bug #554 is NOT yet fixed — the DI container still lacks a db provider.
The previous commit incorrectly removed tdd_expected_fail tags, exposing
the real failures in Behave unit tests (13 scenarios).

Root cause: the Robot helper had its own pass/fail inversion (a workaround
from before tdd_expected_fail_listener existed).  The master merge brought
in the listener, causing double inversion that made Robot tests appear to
pass.  This commit:

- Rewrites the helper to report real outcomes (exit 0 = bug fixed)
- Restores tdd_expected_fail on all 16 Robot + Behave scenarios
- Lets the listener (Robot) and environment.py (Behave) handle inversion
This commit is contained in:
Brent E. Edwards
2026-03-11 00:40:35 +00:00
parent 3c7915099f
commit bed7072dde
4 changed files with 45 additions and 45 deletions
+13 -12
View File
@@ -1,5 +1,6 @@
# TDD tests for bug #554 — the DI container fix has landed.
# The @tdd_expected_fail tags have been removed; all scenarios now pass.
# TDD tests for bug #554 — expected to fail until the DI container fix lands.
# Once the fix is applied, remove the @tdd_expected_fail tags and verify all
# scenarios pass.
Feature: Session list command handles missing database gracefully
As a developer using the agents CLI
I want "agents session list" to work after a fresh init
@@ -8,28 +9,28 @@ Feature: Session list command handles missing database gracefully
Background:
Given a session-list-error CLI runner using the real DI path
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list returns empty list when no sessions exist
When I invoke session-list-error list with default format
Then the session-list-error command should exit successfully
And the session-list-error output should contain "No sessions found"
And the session-list-error output should not contain "AttributeError"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list after init does not raise DI error
When I invoke session-list-error list with default format
Then the session-list-error command should exit successfully
And the session-list-error output should not contain "AttributeError"
And the session-list-error output should not contain "INTERNAL"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list returns sessions after creation via service
Given a session-list-error service with a pre-populated session
When I invoke session-list-error list with default format
Then the session-list-error command should exit successfully
And the session-list-error output should contain "Sessions ("
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list works with rich output format
Given a session-list-error service with a pre-populated session
When I invoke session-list-error list with format "rich"
@@ -37,21 +38,21 @@ Feature: Session list command handles missing database gracefully
And the session-list-error output should contain "Sessions ("
And the session-list-error output should not contain "AttributeError"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list works with JSON output format
Given a session-list-error service with a pre-populated session
When I invoke session-list-error list with format "json"
Then the session-list-error command should exit successfully
And the session-list-error output should be valid JSON containing "sessions"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list works with plain output format
Given a session-list-error service with a pre-populated session
When I invoke session-list-error list with format "plain"
Then the session-list-error command should exit successfully
And the session-list-error output should contain "Sessions ("
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Session list works with YAML output format
Given a session-list-error service with a pre-populated session
When I invoke session-list-error list with format "yaml"
@@ -62,21 +63,21 @@ Feature: Session list command handles missing database gracefully
# with explicit output formats. The production code currently bypasses
# --format for empty lists, so these document the expected behaviour.
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Empty session list with JSON format produces valid JSON
When I invoke session-list-error list with format "json"
Then the session-list-error command should exit successfully
And the session-list-error output should be valid JSON containing "sessions"
And the session-list-error output should not contain "AttributeError"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Empty session list with YAML format produces valid YAML
When I invoke session-list-error list with format "yaml"
Then the session-list-error command should exit successfully
And the session-list-error output should be valid YAML containing "sessions"
And the session-list-error output should not contain "AttributeError"
@tdd_bug @tdd_bug_554
@tdd_bug @tdd_bug_554 @tdd_expected_fail
Scenario: Empty session list with plain format does not error
When I invoke session-list-error list with format "plain"
Then the session-list-error command should exit successfully
+3
View File
@@ -9,16 +9,19 @@ Feature: TDD Bug #554 — session list DI container missing db provider
`container.db()`, but the Container class has no `db` provider, causing
an AttributeError at runtime.
@tdd_expected_fail
Scenario: Session list command succeeds via DI container
Given a CLI runner using the real session DI path
When I invoke the session list command
Then the session list command should exit successfully
@tdd_expected_fail
Scenario: Session list DI path resolves a SessionService
Given a CLI runner using the real session DI path
When I request the session service from the DI container
Then the session service should be a valid SessionService instance
@tdd_expected_fail
Scenario: Session list command produces structured output via DI
Given a CLI runner using the real session DI path
When I invoke the session list command with format json
+21 -26
View File
@@ -1,11 +1,10 @@
"""Helper script for tdd_session_list_di.robot smoke tests.
Each subcommand exercises the real DI path (no mocks) to reproduce bug #554.
Because Robot Framework does not yet have ``@tdd_expected_fail`` inversion
logic (issue #628), this helper **inverts the result itself**: it prints the
sentinel and exits 0 when the bug is detected (expected), and exits 1 if
the bug appears to be fixed (meaning the ``@tdd_expected_fail`` tag should
be removed).
The helper reports the **real** outcome: it exits 0 and prints the sentinel
when the operation succeeds (bug is fixed), and exits 1 when the bug is
still present. The ``tdd_expected_fail_listener`` on the Robot side handles
pass/fail inversion while the bug remains open.
"""
from __future__ import annotations
@@ -64,18 +63,17 @@ def _teardown(db_path: str) -> None:
def list_di_error() -> None:
"""Invoke ``session list`` through the real DI path.
Expected: the command fails because ``container.db()`` does not exist.
Exits 0 with sentinel when the command succeeds (bug fixed).
Exits 1 when the command fails (bug still present).
"""
db_path = _setup_real_di()
try:
result = runner.invoke(session_app, ["list"])
if result.exit_code != 0:
# Bug present — expected failure.
if result.exit_code == 0:
print("tdd-session-list-di-error-ok")
else:
print(
"ERROR: session list succeeded — bug #554 appears fixed. "
"Remove @tdd_expected_fail tag.",
f"session list failed with exit code {result.exit_code}",
file=sys.stderr,
)
sys.exit(1)
@@ -86,23 +84,21 @@ def list_di_error() -> None:
def service_resolution() -> None:
"""Call ``_get_session_service()`` to verify DI resolution.
Expected: raises ``AttributeError`` because ``container.db()`` is missing.
Exits 0 with sentinel when the service resolves (bug fixed).
Exits 1 when resolution raises AttributeError (bug still present).
"""
db_path = _setup_real_di()
try:
try:
session_mod._get_session_service()
except AttributeError:
# Bug present — expected failure.
print("tdd-session-list-service-resolution-ok")
return
except AttributeError as exc:
print(
f"_get_session_service() raised {exc!r}",
file=sys.stderr,
)
sys.exit(1)
print(
"ERROR: _get_session_service() succeeded — bug #554 appears fixed. "
"Remove @tdd_expected_fail tag.",
file=sys.stderr,
)
sys.exit(1)
print("tdd-session-list-service-resolution-ok")
finally:
_teardown(db_path)
@@ -110,18 +106,17 @@ def service_resolution() -> None:
def list_json() -> None:
"""Invoke ``session list --format json`` through the real DI path.
Expected: the command fails because ``container.db()`` does not exist.
Exits 0 with sentinel when the command succeeds (bug fixed).
Exits 1 when the command fails (bug still present).
"""
db_path = _setup_real_di()
try:
result = runner.invoke(session_app, ["list", "--format", "json"])
if result.exit_code != 0:
# Bug present — expected failure.
if result.exit_code == 0:
print("tdd-session-list-json-ok")
else:
print(
"ERROR: session list --format json succeeded — bug #554 appears "
"fixed. Remove @tdd_expected_fail tag.",
f"session list --format json failed with exit code {result.exit_code}",
file=sys.stderr,
)
sys.exit(1)
+8 -7
View File
@@ -1,7 +1,8 @@
*** Settings ***
Documentation TDD Bug #554 — session list DI container missing db provider
... Integration smoke tests verifying that the session list command
... works correctly now that bug #554 is fixed.
... succeeds once the DI container has a proper ``db`` provider.
... Tagged ``tdd_expected_fail`` until bug #554 is resolved.
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
@@ -11,8 +12,8 @@ ${HELPER} ${CURDIR}/helper_tdd_session_list_di.py
*** Test Cases ***
TDD Session List DI Error Via CLI
[Documentation] Verify that ``session list`` triggers the DI db error
[Tags] tdd_bug tdd_bug_554
[Documentation] Verify that ``session list`` succeeds via the real DI path
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
${result}= Run Process ${PYTHON} ${HELPER} list-di-error cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
@@ -20,8 +21,8 @@ TDD Session List DI Error Via CLI
Should Contain ${result.stdout} tdd-session-list-di-error-ok
TDD Session List DI Service Resolution
[Documentation] Verify that ``_get_session_service()`` raises AttributeError due to missing db provider
[Tags] tdd_bug tdd_bug_554
[Documentation] Verify that ``_get_session_service()`` resolves a valid service
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
${result}= Run Process ${PYTHON} ${HELPER} service-resolution cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
@@ -29,8 +30,8 @@ TDD Session List DI Service Resolution
Should Contain ${result.stdout} tdd-session-list-service-resolution-ok
TDD Session List DI JSON Output
[Documentation] Verify that ``session list --format json`` fails due to DI db error
[Tags] tdd_bug tdd_bug_554
[Documentation] Verify that ``session list --format json`` succeeds via the real DI path
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
${result}= Run Process ${PYTHON} ${HELPER} list-json cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}