Files
temp/features/steps/tdd_e2e_implicit_init_steps.py
brent.edwards efacfd61c3 test: add TDD bug-capture test for #1023 — implicit init requirement (#1113)
## Summary

Adds TDD bug-capture tests proving that CLI commands (`resource add`, `project create`) fail in a fresh environment without explicit `agents init`, even when `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true` is set.

Bug #1023 reports that `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true` triggers schema migrations on an existing database but does NOT create the database file or its parent directory structure, causing `sqlite3.OperationalError: unable to open database file`.

## Changes

### Behave Tests
- `features/tdd_e2e_implicit_init.feature` — Two scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_1023`:
  - `resource add` in a fresh environment without prior init
  - `project create` in a fresh environment without prior init
  - Both scenarios include output content assertions
- `features/steps/tdd_e2e_implicit_init_steps.py` — Step definitions using Typer's CliRunner with real CLI invocation in a pristine temp directory.
  - Env isolation: saves/removes `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS`, `CLEVERAGENTS_DATABASE_URL`, `CLEVERAGENTS_TEST_DATABASE_URL`, `CLEVERAGENTS_HOME`, `BEHAVE_TESTING`, `CLEVERAGENTS_TEMPLATE_DB`, `CLEVERAGENTS_TESTING_USE_MOCK_AI`
  - CliRunner invoked without `catch_exceptions=False` to ensure proper `@tdd_expected_fail` inversion

### Robot Tests
- `robot/tdd_e2e_implicit_init.robot` — Two test cases tagged `tdd_bug tdd_bug_1023 tdd_expected_fail` (listener inverts results while bug is unfixed).
- `robot/helper_tdd_e2e_implicit_init.py` — Helper script exercising CLI in a fresh environment.
  - Env isolation includes `ROBOT_TESTING`, `CLEVERAGENTS_TEMPLATE_DB`, `CLEVERAGENTS_TESTING_USE_MOCK_AI`
  - Env restoration in `finally` block ensures cleanup even on exceptions
  - CliRunner invoked without `catch_exceptions=False` for correct inversion

### Changelog
- Updated `CHANGELOG.md` with entry for this TDD test addition.

## Quality Gates

| Gate | Result |
|------|--------|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (462 features, 12232 scenarios, 0 failed) |
| `nox -s integration_tests` | PASS (pre-existing failures only; our suite correct) |
| `nox -s e2e_tests` | N/A (TDD test only, no E2E suite changes) |
| `nox -s coverage_report` | PASS (98% coverage, threshold 97%) |

Closes #1033

Reviewed-on: cleveragents/cleveragents-core#1113
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-03-28 00:30:26 +00:00

174 lines
6.7 KiB
Python

"""Step definitions for TDD Bug #1023 — implicit init requirement.
These steps exercise real CLI commands in a fresh environment where
``agents init`` has NOT been explicitly run. The environment variable
``CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true`` is set, which *should*
cause the database to be created implicitly.
Bug #1023 reports that ``CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true``
triggers schema migrations on an existing database but does NOT create
the database file or its parent directory structure. As a result, any
CLI command that touches the database (e.g., ``resource add``,
``project create``) fails with:
sqlite3.OperationalError: unable to open database file
The assertions here expect the commands to succeed (exit code 0),
proving implicit init works and preventing regressions.
Mock strategy
~~~~~~~~~~~~~
No mocks are needed — we exercise the real CLI via Typer's CliRunner
in a pristine temporary directory. We only manipulate environment
variables to create a fresh, uninitialised CLEVERAGENTS_HOME. We
remove ``CLEVERAGENTS_DATABASE_URL``, ``CLEVERAGENTS_TEST_DATABASE_URL``,
``BEHAVE_TESTING``, and ``CLEVERAGENTS_TEMPLATE_DB`` so the container
derives the DB path from ``CLEVERAGENTS_HOME``, migration prompts are
not auto-approved by the testing guard, and the template-DB fast-path
does not silently create the database for us.
"""
from __future__ import annotations
import os
import shutil
import tempfile
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.application.container import reset_container
from cleveragents.cli.main import app
runner = CliRunner()
# ---------------------------------------------------------------------------
# Given steps
# ---------------------------------------------------------------------------
@given("a fresh isolated environment for tdd-implicit-init")
def step_fresh_environment(context: Context) -> None:
"""Set up a fresh temporary environment with no existing database.
Creates a temporary directory to serve as ``CLEVERAGENTS_HOME``.
Does NOT run ``agents init`` — the entire point of this TDD test is
to verify that CLI commands work without it.
Removes ``BEHAVE_TESTING`` so the migration runner's auto-approve
guard does not mask the real code path.
"""
tmpdir = tempfile.mkdtemp(prefix="tdd_implicit_init_")
context.tdd_implicit_init_tmpdir = tmpdir
# Save and remove env vars that point to pre-migrated databases,
# template-DB fast-paths, or testing guards that auto-approve
# migration prompts. CLEVERAGENTS_TEMPLATE_DB is particularly
# important: when set, the Behave template-DB patch copies a
# pre-migrated database, silently bypassing the directory-creation
# bug this test is designed to capture.
saved_env: dict[str, str | None] = {
k: os.environ.pop(k, None)
for k in (
"CLEVERAGENTS_AUTO_APPLY_MIGRATIONS",
"CLEVERAGENTS_DATABASE_URL",
"CLEVERAGENTS_TEST_DATABASE_URL",
"CLEVERAGENTS_HOME",
"BEHAVE_TESTING",
"CLEVERAGENTS_TEMPLATE_DB",
"CLEVERAGENTS_TESTING_USE_MOCK_AI",
)
}
context.tdd_implicit_init_saved_env = saved_env
# Point CLEVERAGENTS_HOME at our pristine temp directory.
os.environ["CLEVERAGENTS_HOME"] = tmpdir
# Force an isolated SQLite database path under this scenario's temp
# directory. This avoids leaking state from repository-level local
# database files created by prior test runs.
db_url = f"sqlite:///{tmpdir}/db/cleveragents.db"
os.environ["CLEVERAGENTS_DATABASE_URL"] = db_url
os.environ["CLEVERAGENTS_TEST_DATABASE_URL"] = db_url
# Reset the DI container so it picks up our env overrides.
reset_container()
def _cleanup() -> None:
for key, val in saved_env.items():
if val is not None:
os.environ[key] = val
else:
os.environ.pop(key, None)
reset_container()
shutil.rmtree(tmpdir, ignore_errors=True)
context.add_cleanup(_cleanup)
@given("CLEVERAGENTS_AUTO_APPLY_MIGRATIONS is set to true")
def step_set_auto_apply_migrations(context: Context) -> None:
"""Ensure CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true is in the environment.
This is the env var that should, in theory, trigger implicit
database creation and migration. Bug #1023 reports it does not
create the directory structure.
"""
os.environ["CLEVERAGENTS_AUTO_APPLY_MIGRATIONS"] = "true"
# ---------------------------------------------------------------------------
# When steps
# ---------------------------------------------------------------------------
@when('I run the CLI command "{command}" without prior init')
def step_run_cli_command_without_init(context: Context, command: str) -> None:
"""Invoke a CLI command via CliRunner without running ``agents init`` first.
The command string is split into tokens and passed to the main CLI
app. ``CLEVERAGENTS_HOME`` is set to the pristine temp directory
and ``CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true`` is in the
environment.
Exceptions are caught by CliRunner (the default behaviour) so that
the result object always carries an ``exit_code`` and ``output``.
This is essential for ``@tdd_expected_fail`` inversion — if
exceptions propagated as non-AssertionError, the inversion guard
in ``apply_tdd_inversion`` would skip inversion and the scenario
would be reported as a hard failure instead of an expected failure.
"""
# Reset container again to ensure a clean UoW derivation.
reset_container()
args = command.split()
context.tdd_implicit_init_result = runner.invoke(app, args)
# ---------------------------------------------------------------------------
# Then steps
# ---------------------------------------------------------------------------
@then("the tdd-implicit-init command should exit with code {code:d}")
def step_exit_code(context: Context, code: int) -> None:
"""Assert the CLI command exited with the expected return code."""
result = context.tdd_implicit_init_result
assert result is not None, "CLI command was not invoked"
actual = result.exit_code
assert actual == code, (
f"Expected exit code {code}, got {actual}.\nOutput:\n{result.output}"
)
@then('the tdd-implicit-init command output should contain "{text}"')
def step_output_contains(context: Context, text: str) -> None:
"""Assert the CLI command output contains the expected text."""
result = context.tdd_implicit_init_result
assert result is not None, "CLI command was not invoked"
assert text in result.output, (
f"Expected output to contain {text!r}.\nActual output:\n{result.output}"
)