test(cli): TDD failing tests for session list missing database (bug #680)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 17s
CI / typecheck (pull_request) Successful in 38s
CI / security (pull_request) Successful in 53s
CI / unit_tests (pull_request) Successful in 2m58s
CI / integration_tests (pull_request) Successful in 3m30s
CI / docker (pull_request) Successful in 1m9s
CI / coverage (pull_request) Successful in 5m30s
CI / benchmark-regression (pull_request) Successful in 35m22s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 16s
CI / quality (push) Successful in 21s
CI / security (push) Successful in 35s
CI / typecheck (push) Successful in 48s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 3m25s
CI / unit_tests (push) Successful in 3m43s
CI / docker (push) Successful in 10s
CI / coverage (push) Successful in 5m39s
CI / benchmark-publish (push) Successful in 19m41s

Add Behave and Robot Framework TDD tests that exercise the session list
command when no database file exists.  Tagged @tdd_expected_fail so that
the tests pass CI while the bug is present (the Container has no db
provider, causing AttributeError).  Once the bugfix removes the tag,
these become permanent regression tests.

ISSUES CLOSED: #683
This commit was merged in pull request #702.
This commit is contained in:
Brent E. Edwards
2026-03-11 22:50:37 +00:00
parent 3959565723
commit b5fc903425
4 changed files with 325 additions and 0 deletions
@@ -0,0 +1,118 @@
"""Step definitions for TDD Bug #680 — session list with missing database.
These steps exercise the ``session list`` CLI command when **no database file
exists**. Unlike the #554 TDD tests (which create a temp database), these
tests deliberately point ``CLEVERAGENTS_DATABASE_URL`` at a non-existent
path so that both the DI wiring **and** auto-creation of the database are
exercised.
On ``master`` (before the fix), the Container has no ``db`` provider, so
``_get_session_service()`` raises ``AttributeError``. After the fix, the
Container provides a ``db`` Singleton that auto-creates the database via
``Base.metadata.create_all()``, and ``session list`` returns an empty list.
"""
from __future__ import annotations
import json
import os
import tempfile
from pathlib import Path
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.application.container import reset_container
from cleveragents.cli.commands import session as session_mod
from cleveragents.cli.commands.session import app as session_app
from cleveragents.config.settings import Settings
@given("a CLI runner with no database file present")
def step_runner_no_db(context: Context) -> None:
"""Set up a CLI runner pointing at a database path that does not exist.
The database URL is set to a file inside a temporary directory, but
the file is **not** created. This forces the application to either
auto-create the database or fail with an appropriate error.
"""
context.runner = CliRunner()
context.missing_db_tmpdir = tempfile.mkdtemp(prefix="tdd_missing_db_680_")
# Point at a file that does NOT exist — this is the key difference from
# the shared step ``a CLI runner using the real session DI path`` which
# creates the file via ``tempfile.mkstemp``.
db_path = os.path.join(context.missing_db_tmpdir, "nonexistent.db")
context.missing_db_path = db_path
os.environ["CLEVERAGENTS_DATABASE_URL"] = f"sqlite:///{db_path}"
reset_container()
session_mod._service = None
def _cleanup() -> None:
session_mod._service = None
os.environ.pop("CLEVERAGENTS_DATABASE_URL", None)
reset_container()
Settings._instance = None
# The fix may auto-create the database file, so clean it up.
Path(db_path).unlink(missing_ok=True)
# Remove the temp directory.
import shutil
shutil.rmtree(context.missing_db_tmpdir, ignore_errors=True)
context.add_cleanup(_cleanup)
@when("I invoke session list with missing db")
def step_invoke_list(context: Context) -> None:
"""Invoke ``session list`` through the real CLI app."""
context.result = context.runner.invoke(session_app, ["list"])
@when("I invoke session list with missing db and format json")
def step_invoke_list_json(context: Context) -> None:
"""Invoke ``session list --format json`` through the real CLI app."""
context.result = context.runner.invoke(session_app, ["list", "--format", "json"])
@then("the session list missing db command should exit successfully")
def step_exit_ok(context: Context) -> None:
"""Assert the command exits with code 0."""
assert context.result.exit_code == 0, (
f"Expected exit code 0, got {context.result.exit_code}.\n"
f"Output:\n{context.result.output}\n"
f"Exception: {getattr(context.result, 'exception', None)!r}"
)
@then('the session list missing db output should not contain "{text}"')
def step_output_not_contains(context: Context, text: str) -> None:
"""Assert the output does NOT contain a specific string."""
assert text not in context.result.output, (
f"Output unexpectedly contained '{text}':\n{context.result.output}"
)
@then("the session list missing db output should be valid JSON")
def step_output_valid_json(context: Context) -> None:
"""Assert the output is parseable JSON."""
try:
json.loads(context.result.output)
except json.JSONDecodeError as exc:
raise AssertionError(
f"Output is not valid JSON:\n{context.result.output}"
) from exc
@then("the session list missing db output should indicate no sessions")
def step_output_no_sessions(context: Context) -> None:
"""Assert the output indicates an empty session list.
The CLI shows 'No sessions found' for human-readable formats
(rich, plain, color).
"""
output = context.result.output
assert "No sessions found" in output or "sessions" in output.lower(), (
f"Expected indication of no sessions in output:\n{output}"
)
@@ -0,0 +1,34 @@
@tdd_bug @tdd_bug_680
Feature: TDD Bug #680 — session list with missing database
As a developer
I want to verify that `agents session list` handles a missing database
gracefully
So that the bug is captured and will be caught by a regression test
The CLI `session list` command should work even when no database file
exists. Currently `_get_session_service()` calls `container.db()`,
but the Container class has no `db` provider, causing an
AttributeError at runtime. Even after the DI wiring is fixed, the
database file itself must be auto-created so the command returns an
empty list rather than crashing.
@tdd_expected_fail
Scenario: Session list with missing database exits successfully
Given a CLI runner with no database file present
When I invoke session list with missing db
Then the session list missing db command should exit successfully
And the session list missing db output should not contain "AttributeError"
@tdd_expected_fail
Scenario: Session list with missing database produces valid JSON
Given a CLI runner with no database file present
When I invoke session list with missing db and format json
Then the session list missing db command should exit successfully
And the session list missing db output should be valid JSON
@tdd_expected_fail
Scenario: Session list with missing database shows empty list
Given a CLI runner with no database file present
When I invoke session list with missing db
Then the session list missing db command should exit successfully
And the session list missing db output should indicate no sessions
+141
View File
@@ -0,0 +1,141 @@
"""Helper script for tdd_session_list_missing_db.robot smoke tests.
Each subcommand exercises the real DI path (no mocks) to reproduce bug #680.
Unlike the #554 helpers, these tests point ``CLEVERAGENTS_DATABASE_URL`` at
a **non-existent** database file to verify that the application handles a
missing database gracefully (auto-creating it or returning a clean error).
The helper reports the **real** outcome: it exits 0 and prints the sentinel
when the operation succeeds (bug is fixed), and exits 1 when the bug is
still present. The ``tdd_expected_fail_listener`` on the Robot side handles
pass/fail inversion while the bug remains open.
"""
from __future__ import annotations
import os
import shutil
import sys
import tempfile
from collections.abc import Callable
from pathlib import Path
# Ensure local source tree AND robot/ directory are importable.
_ROOT = Path(__file__).resolve().parents[1]
_SRC = str(_ROOT / "src")
_ROBOT = str(_ROOT / "robot")
for _p in (_SRC, _ROBOT):
if _p not in sys.path:
sys.path.insert(0, _p)
from typer.testing import CliRunner # noqa: E402
from cleveragents.application.container import reset_container # noqa: E402
from cleveragents.cli.commands import session as session_mod # noqa: E402
from cleveragents.cli.commands.session import app as session_app # noqa: E402
from cleveragents.config.settings import Settings # noqa: E402
runner = CliRunner()
# ---------------------------------------------------------------------------
# Setup / Teardown — missing-database variant
# ---------------------------------------------------------------------------
def setup_missing_db() -> str:
"""Prepare the environment with a database URL pointing to a non-existent file.
Returns the path to the temp directory (caller must clean up).
"""
session_mod._service = None
reset_container()
tmpdir = tempfile.mkdtemp(prefix="tdd_missing_db_680_")
db_path = os.path.join(tmpdir, "nonexistent.db")
os.environ["CLEVERAGENTS_DATABASE_URL"] = f"sqlite:///{db_path}"
return tmpdir
def teardown_missing_db(tmpdir: str) -> None:
"""Clean up after a test."""
session_mod._service = None
os.environ.pop("CLEVERAGENTS_DATABASE_URL", None)
reset_container()
Settings._instance = None
shutil.rmtree(tmpdir, ignore_errors=True)
# ---------------------------------------------------------------------------
# Subcommands
# ---------------------------------------------------------------------------
def list_missing_db() -> None:
"""Invoke ``session list`` when no database file exists.
Exits 0 with sentinel when the command succeeds (bug fixed).
Exits 1 when the command fails (bug still present).
"""
tmpdir = setup_missing_db()
try:
result = runner.invoke(session_app, ["list"])
if result.exit_code == 0:
print("tdd-session-list-missing-db-ok")
else:
print(
f"session list failed with exit code {result.exit_code}",
file=sys.stderr,
)
if result.exception:
import traceback
traceback.print_exception(
type(result.exception),
result.exception,
result.exception.__traceback__,
file=sys.stderr,
)
sys.exit(1)
finally:
teardown_missing_db(tmpdir)
def list_missing_db_json() -> None:
"""Invoke ``session list --format json`` when no database file exists.
Exits 0 with sentinel when the command succeeds (bug fixed).
Exits 1 when the command fails (bug still present).
"""
tmpdir = setup_missing_db()
try:
result = runner.invoke(session_app, ["list", "--format", "json"])
if result.exit_code == 0:
print("tdd-session-list-missing-db-json-ok")
else:
print(
f"session list --format json failed with exit code {result.exit_code}",
file=sys.stderr,
)
sys.exit(1)
finally:
teardown_missing_db(tmpdir)
# ---------------------------------------------------------------------------
# Dispatcher
# ---------------------------------------------------------------------------
_COMMANDS: dict[str, Callable[[], None]] = {
"list-missing-db": list_missing_db,
"list-missing-db-json": list_missing_db_json,
}
if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
print(
f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>",
file=sys.stderr,
)
sys.exit(1)
cmd = _COMMANDS[sys.argv[1]]
cmd()
+32
View File
@@ -0,0 +1,32 @@
*** Settings ***
Documentation TDD Bug #680 — session list with missing database
... Integration smoke tests verifying that the session list command
... handles a missing database gracefully. The database URL points
... to a non-existent file; the command should auto-create the DB
... and return an empty list rather than crashing.
... Tagged ``tdd_expected_fail`` until bug #680 is resolved.
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER} ${CURDIR}/helper_tdd_session_list_missing_db.py
*** Test Cases ***
TDD Session List Missing DB Via CLI
[Documentation] Verify that ``session list`` succeeds when no database file exists
[Tags] tdd_bug tdd_bug_680 tdd_expected_fail
${result}= Run Process ${PYTHON} ${HELPER} list-missing-db cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} tdd-session-list-missing-db-ok
TDD Session List Missing DB JSON Output
[Documentation] Verify that ``session list --format json`` succeeds when no database file exists
[Tags] tdd_bug tdd_bug_680 tdd_expected_fail
${result}= Run Process ${PYTHON} ${HELPER} list-missing-db-json cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} tdd-session-list-missing-db-json-ok