fix(tdd): restore tdd_expected_fail tags and fix Robot helper inversion
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 21s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 42s
CI / unit_tests (pull_request) Successful in 2m35s
CI / docker (pull_request) Successful in 8s
CI / integration_tests (pull_request) Successful in 3m30s
CI / coverage (pull_request) Successful in 5m17s
CI / benchmark-regression (pull_request) Successful in 35m10s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 21s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 42s
CI / unit_tests (pull_request) Successful in 2m35s
CI / docker (pull_request) Successful in 8s
CI / integration_tests (pull_request) Successful in 3m30s
CI / coverage (pull_request) Successful in 5m17s
CI / benchmark-regression (pull_request) Successful in 35m10s
Bug #554 is NOT yet fixed — the DI container still lacks a db provider. The previous commit incorrectly removed tdd_expected_fail tags, exposing the real failures in Behave unit tests (13 scenarios). Root cause: the Robot helper had its own pass/fail inversion (a workaround from before tdd_expected_fail_listener existed). The master merge brought in the listener, causing double inversion that made Robot tests appear to pass. This commit: - Rewrites the helper to report real outcomes (exit 0 = bug fixed) - Restores tdd_expected_fail on all 16 Robot + Behave scenarios - Lets the listener (Robot) and environment.py (Behave) handle inversion
This commit is contained in:
@@ -1,5 +1,6 @@
|
||||
# TDD tests for bug #554 — the DI container fix has landed.
|
||||
# The @tdd_expected_fail tags have been removed; all scenarios now pass.
|
||||
# TDD tests for bug #554 — expected to fail until the DI container fix lands.
|
||||
# Once the fix is applied, remove the @tdd_expected_fail tags and verify all
|
||||
# scenarios pass.
|
||||
Feature: Session list command handles missing database gracefully
|
||||
As a developer using the agents CLI
|
||||
I want "agents session list" to work after a fresh init
|
||||
@@ -8,28 +9,28 @@ Feature: Session list command handles missing database gracefully
|
||||
Background:
|
||||
Given a session-list-error CLI runner using the real DI path
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list returns empty list when no sessions exist
|
||||
When I invoke session-list-error list with default format
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should contain "No sessions found"
|
||||
And the session-list-error output should not contain "AttributeError"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list after init does not raise DI error
|
||||
When I invoke session-list-error list with default format
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should not contain "AttributeError"
|
||||
And the session-list-error output should not contain "INTERNAL"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list returns sessions after creation via service
|
||||
Given a session-list-error service with a pre-populated session
|
||||
When I invoke session-list-error list with default format
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should contain "Sessions ("
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list works with rich output format
|
||||
Given a session-list-error service with a pre-populated session
|
||||
When I invoke session-list-error list with format "rich"
|
||||
@@ -37,21 +38,21 @@ Feature: Session list command handles missing database gracefully
|
||||
And the session-list-error output should contain "Sessions ("
|
||||
And the session-list-error output should not contain "AttributeError"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list works with JSON output format
|
||||
Given a session-list-error service with a pre-populated session
|
||||
When I invoke session-list-error list with format "json"
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should be valid JSON containing "sessions"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list works with plain output format
|
||||
Given a session-list-error service with a pre-populated session
|
||||
When I invoke session-list-error list with format "plain"
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should contain "Sessions ("
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Session list works with YAML output format
|
||||
Given a session-list-error service with a pre-populated session
|
||||
When I invoke session-list-error list with format "yaml"
|
||||
@@ -62,21 +63,21 @@ Feature: Session list command handles missing database gracefully
|
||||
# with explicit output formats. The production code currently bypasses
|
||||
# --format for empty lists, so these document the expected behaviour.
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Empty session list with JSON format produces valid JSON
|
||||
When I invoke session-list-error list with format "json"
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should be valid JSON containing "sessions"
|
||||
And the session-list-error output should not contain "AttributeError"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Empty session list with YAML format produces valid YAML
|
||||
When I invoke session-list-error list with format "yaml"
|
||||
Then the session-list-error command should exit successfully
|
||||
And the session-list-error output should be valid YAML containing "sessions"
|
||||
And the session-list-error output should not contain "AttributeError"
|
||||
|
||||
@tdd_bug @tdd_bug_554
|
||||
@tdd_bug @tdd_bug_554 @tdd_expected_fail
|
||||
Scenario: Empty session list with plain format does not error
|
||||
When I invoke session-list-error list with format "plain"
|
||||
Then the session-list-error command should exit successfully
|
||||
|
||||
@@ -9,16 +9,19 @@ Feature: TDD Bug #554 — session list DI container missing db provider
|
||||
`container.db()`, but the Container class has no `db` provider, causing
|
||||
an AttributeError at runtime.
|
||||
|
||||
@tdd_expected_fail
|
||||
Scenario: Session list command succeeds via DI container
|
||||
Given a CLI runner using the real session DI path
|
||||
When I invoke the session list command
|
||||
Then the session list command should exit successfully
|
||||
|
||||
@tdd_expected_fail
|
||||
Scenario: Session list DI path resolves a SessionService
|
||||
Given a CLI runner using the real session DI path
|
||||
When I request the session service from the DI container
|
||||
Then the session service should be a valid SessionService instance
|
||||
|
||||
@tdd_expected_fail
|
||||
Scenario: Session list command produces structured output via DI
|
||||
Given a CLI runner using the real session DI path
|
||||
When I invoke the session list command with format json
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
"""Helper script for tdd_session_list_di.robot smoke tests.
|
||||
|
||||
Each subcommand exercises the real DI path (no mocks) to reproduce bug #554.
|
||||
Because Robot Framework does not yet have ``@tdd_expected_fail`` inversion
|
||||
logic (issue #628), this helper **inverts the result itself**: it prints the
|
||||
sentinel and exits 0 when the bug is detected (expected), and exits 1 if
|
||||
the bug appears to be fixed (meaning the ``@tdd_expected_fail`` tag should
|
||||
be removed).
|
||||
The helper reports the **real** outcome: it exits 0 and prints the sentinel
|
||||
when the operation succeeds (bug is fixed), and exits 1 when the bug is
|
||||
still present. The ``tdd_expected_fail_listener`` on the Robot side handles
|
||||
pass/fail inversion while the bug remains open.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -64,18 +63,17 @@ def _teardown(db_path: str) -> None:
|
||||
def list_di_error() -> None:
|
||||
"""Invoke ``session list`` through the real DI path.
|
||||
|
||||
Expected: the command fails because ``container.db()`` does not exist.
|
||||
Exits 0 with sentinel when the command succeeds (bug fixed).
|
||||
Exits 1 when the command fails (bug still present).
|
||||
"""
|
||||
db_path = _setup_real_di()
|
||||
try:
|
||||
result = runner.invoke(session_app, ["list"])
|
||||
if result.exit_code != 0:
|
||||
# Bug present — expected failure.
|
||||
if result.exit_code == 0:
|
||||
print("tdd-session-list-di-error-ok")
|
||||
else:
|
||||
print(
|
||||
"ERROR: session list succeeded — bug #554 appears fixed. "
|
||||
"Remove @tdd_expected_fail tag.",
|
||||
f"session list failed with exit code {result.exit_code}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
@@ -86,23 +84,21 @@ def list_di_error() -> None:
|
||||
def service_resolution() -> None:
|
||||
"""Call ``_get_session_service()`` to verify DI resolution.
|
||||
|
||||
Expected: raises ``AttributeError`` because ``container.db()`` is missing.
|
||||
Exits 0 with sentinel when the service resolves (bug fixed).
|
||||
Exits 1 when resolution raises AttributeError (bug still present).
|
||||
"""
|
||||
db_path = _setup_real_di()
|
||||
try:
|
||||
try:
|
||||
session_mod._get_session_service()
|
||||
except AttributeError:
|
||||
# Bug present — expected failure.
|
||||
print("tdd-session-list-service-resolution-ok")
|
||||
return
|
||||
except AttributeError as exc:
|
||||
print(
|
||||
f"_get_session_service() raised {exc!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print(
|
||||
"ERROR: _get_session_service() succeeded — bug #554 appears fixed. "
|
||||
"Remove @tdd_expected_fail tag.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
print("tdd-session-list-service-resolution-ok")
|
||||
finally:
|
||||
_teardown(db_path)
|
||||
|
||||
@@ -110,18 +106,17 @@ def service_resolution() -> None:
|
||||
def list_json() -> None:
|
||||
"""Invoke ``session list --format json`` through the real DI path.
|
||||
|
||||
Expected: the command fails because ``container.db()`` does not exist.
|
||||
Exits 0 with sentinel when the command succeeds (bug fixed).
|
||||
Exits 1 when the command fails (bug still present).
|
||||
"""
|
||||
db_path = _setup_real_di()
|
||||
try:
|
||||
result = runner.invoke(session_app, ["list", "--format", "json"])
|
||||
if result.exit_code != 0:
|
||||
# Bug present — expected failure.
|
||||
if result.exit_code == 0:
|
||||
print("tdd-session-list-json-ok")
|
||||
else:
|
||||
print(
|
||||
"ERROR: session list --format json succeeded — bug #554 appears "
|
||||
"fixed. Remove @tdd_expected_fail tag.",
|
||||
f"session list --format json failed with exit code {result.exit_code}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
*** Settings ***
|
||||
Documentation TDD Bug #554 — session list DI container missing db provider
|
||||
... Integration smoke tests verifying that the session list command
|
||||
... works correctly now that bug #554 is fixed.
|
||||
... succeeds once the DI container has a proper ``db`` provider.
|
||||
... Tagged ``tdd_expected_fail`` until bug #554 is resolved.
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment
|
||||
Suite Teardown Cleanup Test Environment
|
||||
@@ -11,8 +12,8 @@ ${HELPER} ${CURDIR}/helper_tdd_session_list_di.py
|
||||
|
||||
*** Test Cases ***
|
||||
TDD Session List DI Error Via CLI
|
||||
[Documentation] Verify that ``session list`` triggers the DI db error
|
||||
[Tags] tdd_bug tdd_bug_554
|
||||
[Documentation] Verify that ``session list`` succeeds via the real DI path
|
||||
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
|
||||
${result}= Run Process ${PYTHON} ${HELPER} list-di-error cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
@@ -20,8 +21,8 @@ TDD Session List DI Error Via CLI
|
||||
Should Contain ${result.stdout} tdd-session-list-di-error-ok
|
||||
|
||||
TDD Session List DI Service Resolution
|
||||
[Documentation] Verify that ``_get_session_service()`` raises AttributeError due to missing db provider
|
||||
[Tags] tdd_bug tdd_bug_554
|
||||
[Documentation] Verify that ``_get_session_service()`` resolves a valid service
|
||||
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
|
||||
${result}= Run Process ${PYTHON} ${HELPER} service-resolution cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
@@ -29,8 +30,8 @@ TDD Session List DI Service Resolution
|
||||
Should Contain ${result.stdout} tdd-session-list-service-resolution-ok
|
||||
|
||||
TDD Session List DI JSON Output
|
||||
[Documentation] Verify that ``session list --format json`` fails due to DI db error
|
||||
[Tags] tdd_bug tdd_bug_554
|
||||
[Documentation] Verify that ``session list --format json`` succeeds via the real DI path
|
||||
[Tags] tdd_bug tdd_bug_554 tdd_expected_fail
|
||||
${result}= Run Process ${PYTHON} ${HELPER} list-json cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
|
||||
Reference in New Issue
Block a user