forked from cleveragents/cleveragents-core
test(resource): add failing tests for built-in fs-directory type bootstrap
Add TDD-style Behave BDD tests for the built-in fs-directory resource type bootstrap (bug #523). Three Gherkin scenarios: one failing TDD test reproducing the bug (no bootstrap called during init, tagged @wip), and two regression tests verifying bootstrap_builtin_types() seeds correct data and resource add fs-directory succeeds after bootstrap. Includes Robot Framework regression tests. Review feedback addressed: - Removed all 21 unnecessary # type: ignore comments (hurui200320 M1) - Fixed is not True to is False for clarity (Aditya F2) - Fixed Robot common.resource path to ${CURDIR}/common.resource (hurui200320 L1) - Squashed all commits into one and rebased onto master (C1, C2) - Added CHANGELOG entry with correct scenario count Closes #537
This commit is contained in:
@@ -2,6 +2,11 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Added TDD-style Behave BDD tests for the built-in `fs-directory` resource type
|
||||
bootstrap. Three scenarios: one failing TDD test reproducing bug #523 (no bootstrap
|
||||
called during init), and two regression tests verifying `bootstrap_builtin_types()`
|
||||
seeds correct data and `agents resource add fs-directory` succeeds. Includes Robot
|
||||
Framework regression tests. (#537)
|
||||
- Added general-purpose domain event system under
|
||||
`cleveragents.infrastructure.events`. `EventType` StrEnum defines 38 typed
|
||||
event identifiers across 9 domains (plan lifecycle, decision, invariant, actor,
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# These tests target bug #523 and are expected to fail until the fix is applied.
|
||||
# bootstrap_builtin_types() is never called during initialization, so the
|
||||
# built-in fs-directory resource type is not seeded into the database.
|
||||
|
||||
Feature: Built-in fs-directory Resource Type Bootstrap
|
||||
As a CleverAgents user
|
||||
I want the built-in fs-directory resource type to be available after initialization
|
||||
So that I can run "agents resource add fs-directory" without "Resource type not found"
|
||||
|
||||
# ── Bug reproduction: init path does NOT call bootstrap ────
|
||||
# This scenario is the TDD failing test. It creates a registry WITHOUT
|
||||
# calling bootstrap_builtin_types() and asserts fs-directory exists.
|
||||
# It will fail until the fix integrates bootstrap into the init path.
|
||||
#
|
||||
# NOTE FOR FIX AUTHOR: If the chosen fix is to call bootstrap inside
|
||||
# init_command() / initialize_project() rather than in
|
||||
# ResourceRegistryService.__init__(), you will need to update the Given
|
||||
# step to exercise the init path instead of constructing a bare service.
|
||||
|
||||
@tdd @bug523 @wip
|
||||
Scenario: fs-directory type exists after init without explicit bootstrap call
|
||||
Given a fresh in-memory resource registry without bootstrap
|
||||
When I query the fs bootstrap resource type registry for "fs-directory"
|
||||
Then the bootstrap fs resource type "fs-directory" should exist
|
||||
|
||||
# ── Regression: bootstrap function itself works correctly ──
|
||||
|
||||
@tdd @bug523
|
||||
Scenario: After initialization fs-directory type exists in the registry
|
||||
Given a fresh in-memory resource registry with bootstrap
|
||||
When I query the fs bootstrap resource type registry for "fs-directory"
|
||||
Then the bootstrap fs resource type "fs-directory" should exist
|
||||
And the bootstrap fs resource type kind should be "physical"
|
||||
And the bootstrap fs resource type sandbox_strategy should be "copy_on_write"
|
||||
|
||||
# ── CLI add command ────────────────────────────────────────
|
||||
|
||||
@tdd @bug523
|
||||
Scenario: resource add fs-directory succeeds after bootstrap
|
||||
Given a fresh in-memory resource registry with bootstrap
|
||||
When I run resource add for type "fs-directory" named "local/test" with path "/tmp/test"
|
||||
Then the resource add command should succeed
|
||||
And the resource add output should contain "Added resource"
|
||||
And the resource add output should not contain "Resource type not found"
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Step definitions for resource_type_bootstrap_fs.feature.
|
||||
|
||||
These tests target bug #523. The first scenario (without bootstrap) is a
|
||||
genuine TDD failing test: it creates a fresh registry *without* calling
|
||||
``bootstrap_builtin_types()`` and asserts ``fs-directory`` exists — which
|
||||
fails until the init path is fixed to call bootstrap.
|
||||
|
||||
The remaining scenarios verify that ``bootstrap_builtin_types()`` itself
|
||||
seeds the correct data and that the CLI ``resource add`` path works once
|
||||
the types are present.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from io import StringIO
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from rich.console import Console
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from cleveragents.application.services.resource_registry_service import (
|
||||
ResourceRegistryService,
|
||||
)
|
||||
from cleveragents.core.exceptions import NotFoundError
|
||||
from cleveragents.infrastructure.database.models import Base
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────
|
||||
|
||||
_PATCH_SERVICE = "cleveragents.cli.commands.resource._get_registry_service"
|
||||
_PATCH_CONSOLE = "cleveragents.cli.commands.resource.console"
|
||||
|
||||
|
||||
def _make_service(context: Context, *, run_bootstrap: bool) -> ResourceRegistryService:
|
||||
"""Create an in-memory ResourceRegistryService.
|
||||
|
||||
When *run_bootstrap* is ``True`` the built-in types are seeded; when
|
||||
``False`` the registry is left empty (reproducing the bug).
|
||||
"""
|
||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
Base.metadata.create_all(engine)
|
||||
factory = sessionmaker(bind=engine, expire_on_commit=False)
|
||||
service = ResourceRegistryService(session_factory=factory)
|
||||
if run_bootstrap:
|
||||
service.bootstrap_builtin_types()
|
||||
context.bootstrap_fs_service = service
|
||||
return service
|
||||
|
||||
|
||||
def _capture_output(func: Any, *args: Any, **kwargs: Any) -> tuple[str, bool]:
|
||||
"""Run a CLI function capturing its console output and success status."""
|
||||
buf = StringIO()
|
||||
console = Console(
|
||||
file=buf, width=200, no_color=True, highlight=False, force_terminal=False
|
||||
)
|
||||
|
||||
failed = False
|
||||
failure_reason = ""
|
||||
with patch(_PATCH_CONSOLE, console):
|
||||
try:
|
||||
func(*args, **kwargs)
|
||||
except SystemExit as exc:
|
||||
# Defensive: currently resource_add raises typer.Abort (a
|
||||
# RuntimeError subclass) on failure, not SystemExit. This
|
||||
# handler guards against future changes where the function
|
||||
# is invoked through the Typer CLI runner, which converts
|
||||
# exceptions into SystemExit.
|
||||
if exc.code not in (None, 0):
|
||||
failed = True
|
||||
failure_reason = f"SystemExit: {exc.code}"
|
||||
except Exception as exc:
|
||||
failed = True
|
||||
# typer.Abort() carries no message; str(exc) is empty.
|
||||
# Fall back to the class name so failure_reason is not blank.
|
||||
reason = str(exc) if str(exc) else "(no message)"
|
||||
failure_reason = f"{type(exc).__name__}: {reason}"
|
||||
|
||||
output = buf.getvalue()
|
||||
if failure_reason:
|
||||
output = output + "\n" + failure_reason
|
||||
return output, failed
|
||||
|
||||
|
||||
# ── Given steps ──────────────────────────────────────────────
|
||||
|
||||
|
||||
@given("a fresh in-memory resource registry with bootstrap")
|
||||
def step_fresh_registry_with_bootstrap(context: Context) -> None:
|
||||
"""Set up a fresh in-memory database and run bootstrap_builtin_types."""
|
||||
_make_service(context, run_bootstrap=True)
|
||||
context.bootstrap_fs_output = ""
|
||||
context.bootstrap_fs_failed = False
|
||||
|
||||
|
||||
@given("a fresh in-memory resource registry without bootstrap")
|
||||
def step_fresh_registry_without_bootstrap(context: Context) -> None:
|
||||
"""Set up a fresh in-memory database WITHOUT running bootstrap.
|
||||
|
||||
This reproduces bug #523: the registry is empty because
|
||||
``bootstrap_builtin_types()`` is never called during initialization.
|
||||
"""
|
||||
_make_service(context, run_bootstrap=False)
|
||||
context.bootstrap_fs_output = ""
|
||||
context.bootstrap_fs_failed = False
|
||||
|
||||
|
||||
# ── When steps ───────────────────────────────────────────────
|
||||
|
||||
|
||||
@when('I query the fs bootstrap resource type registry for "{name}"')
|
||||
def step_query_registry_for_type(context: Context, name: str) -> None:
|
||||
"""Query the registry for a specific resource type by name."""
|
||||
service: ResourceRegistryService = context.bootstrap_fs_service
|
||||
try:
|
||||
context.bootstrap_fs_type_spec = service.show_type(name)
|
||||
context.bootstrap_fs_type_found = True
|
||||
except NotFoundError:
|
||||
context.bootstrap_fs_type_spec = None
|
||||
context.bootstrap_fs_type_found = False
|
||||
|
||||
|
||||
@when('I run resource add for type "{type_name}" named "{name}" with path "{path}"')
|
||||
def step_run_resource_add_fs(
|
||||
context: Context, type_name: str, name: str, path: str
|
||||
) -> None:
|
||||
"""Run resource add command via the CLI function."""
|
||||
from cleveragents.cli.commands.resource import resource_add
|
||||
|
||||
with patch(
|
||||
_PATCH_SERVICE,
|
||||
return_value=context.bootstrap_fs_service,
|
||||
):
|
||||
output, failed = _capture_output(
|
||||
resource_add,
|
||||
type_name=type_name,
|
||||
name=name,
|
||||
path=path,
|
||||
branch=None,
|
||||
description=None,
|
||||
image=None,
|
||||
read_only=False,
|
||||
fmt="rich",
|
||||
)
|
||||
context.bootstrap_fs_output = output
|
||||
context.bootstrap_fs_failed = failed
|
||||
|
||||
|
||||
# ── Then steps ───────────────────────────────────────────────
|
||||
|
||||
|
||||
@then('the bootstrap fs resource type "{name}" should exist')
|
||||
def step_type_should_exist(context: Context, name: str) -> None:
|
||||
"""Assert the resource type was found in the registry."""
|
||||
assert context.bootstrap_fs_type_found is True, (
|
||||
f"Expected resource type '{name}' to exist in the registry after "
|
||||
f"bootstrap, but it was not found."
|
||||
)
|
||||
|
||||
|
||||
@then('the bootstrap fs resource type kind should be "{kind}"')
|
||||
def step_type_kind(context: Context, kind: str) -> None:
|
||||
"""Assert the resource type has the expected kind."""
|
||||
spec = context.bootstrap_fs_type_spec
|
||||
actual = spec.resource_kind
|
||||
actual_str = actual.value if hasattr(actual, "value") else str(actual)
|
||||
assert actual_str == kind, f"Expected kind '{kind}', got '{actual_str}'"
|
||||
|
||||
|
||||
@then('the bootstrap fs resource type sandbox_strategy should be "{strategy}"')
|
||||
def step_type_sandbox(context: Context, strategy: str) -> None:
|
||||
"""Assert the resource type has the expected sandbox strategy."""
|
||||
spec = context.bootstrap_fs_type_spec
|
||||
actual = spec.sandbox_strategy
|
||||
actual_str = actual.value if hasattr(actual, "value") else str(actual)
|
||||
assert actual_str == strategy, (
|
||||
f"Expected sandbox_strategy '{strategy}', got '{actual_str}'"
|
||||
)
|
||||
|
||||
|
||||
@then("the resource add command should succeed")
|
||||
def step_resource_add_should_succeed(context: Context) -> None:
|
||||
"""Assert the resource add command did not fail."""
|
||||
assert context.bootstrap_fs_failed is False, (
|
||||
f"Expected 'resource add' to succeed, but it failed. "
|
||||
f"Output: {context.bootstrap_fs_output!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the resource add output should contain "{text}"')
|
||||
def step_fs_output_should_contain(context: Context, text: str) -> None:
|
||||
"""Assert the output contains the given text."""
|
||||
assert text in context.bootstrap_fs_output, (
|
||||
f"Expected output to contain '{text}', but it did not. "
|
||||
f"Full output: {context.bootstrap_fs_output!r}"
|
||||
)
|
||||
|
||||
|
||||
@then('the resource add output should not contain "{text}"')
|
||||
def step_output_should_not_contain(context: Context, text: str) -> None:
|
||||
"""Assert the output does not contain the given text."""
|
||||
assert text not in context.bootstrap_fs_output, (
|
||||
f"Output unexpectedly contained '{text}'. "
|
||||
f"Full output: {context.bootstrap_fs_output!r}"
|
||||
)
|
||||
@@ -0,0 +1,63 @@
|
||||
*** Settings ***
|
||||
Documentation Regression tests for built-in fs-directory type bootstrap (bug #523).
|
||||
... These tests verify that bootstrap_builtin_types() correctly seeds
|
||||
... fs-directory into the registry and that resource registration succeeds
|
||||
... after bootstrap. Both tests call bootstrap explicitly and should pass.
|
||||
Library Process
|
||||
Library OperatingSystem
|
||||
Resource ${CURDIR}/common.resource
|
||||
|
||||
Suite Setup Setup Test Environment
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Test Cases ***
|
||||
Bootstrap Seeds Fs Directory Type Into Registry
|
||||
[Documentation] Verify that bootstrap_builtin_types() seeds fs-directory into the
|
||||
... database so that show_type("fs-directory") succeeds.
|
||||
${script}= Catenate SEPARATOR=\n
|
||||
... from sqlalchemy import create_engine
|
||||
... from sqlalchemy.orm import sessionmaker
|
||||
... from cleveragents.infrastructure.database.models import Base
|
||||
... from cleveragents.application.services.resource_registry_service import ResourceRegistryService
|
||||
... engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
... Base.metadata.create_all(engine)
|
||||
... factory = sessionmaker(bind=engine, expire_on_commit=False)
|
||||
... service = ResourceRegistryService(session_factory=factory)
|
||||
... registered = service.bootstrap_builtin_types()
|
||||
... assert "fs-directory" in registered, f"fs-directory not in registered: {registered}"
|
||||
... spec = service.show_type("fs-directory")
|
||||
... assert spec.name == "fs-directory", f"name mismatch: {spec.name}"
|
||||
... kind = spec.resource_kind.value if hasattr(spec.resource_kind, "value") else str(spec.resource_kind)
|
||||
... assert kind == "physical", f"kind: {kind}"
|
||||
... strategy = spec.sandbox_strategy.value if hasattr(spec.sandbox_strategy, "value") else str(spec.sandbox_strategy)
|
||||
... assert strategy == "copy_on_write", f"strategy: {strategy}"
|
||||
... print("fs-directory bootstrap validated successfully")
|
||||
${result}= Run Process ${PYTHON} -c ${script} timeout=60s
|
||||
Should Be Equal As Integers ${result.rc} 0 fs-directory bootstrap failed: ${result.stderr}
|
||||
Should Contain ${result.stdout} fs-directory bootstrap validated successfully
|
||||
|
||||
Resource Add Fs Directory Succeeds After Bootstrap
|
||||
[Documentation] Verify that after bootstrap, registering an fs-directory resource
|
||||
... instance succeeds (no "Resource type not found" error).
|
||||
${script}= Catenate SEPARATOR=\n
|
||||
... from sqlalchemy import create_engine
|
||||
... from sqlalchemy.orm import sessionmaker
|
||||
... from cleveragents.infrastructure.database.models import Base
|
||||
... from cleveragents.application.services.resource_registry_service import ResourceRegistryService
|
||||
... engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
... Base.metadata.create_all(engine)
|
||||
... factory = sessionmaker(bind=engine, expire_on_commit=False)
|
||||
... service = ResourceRegistryService(session_factory=factory)
|
||||
... service.bootstrap_builtin_types()
|
||||
... resource = service.register_resource(
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}type_name="fs-directory",
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}name="local/test",
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}location="/tmp/test",
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}description="Test fs-directory resource",
|
||||
... )
|
||||
... assert resource.resource_type_name == "fs-directory", f"type: {resource.resource_type_name}"
|
||||
... assert resource.name == "local/test", f"name: {resource.name}"
|
||||
... print("resource add fs-directory succeeded after bootstrap")
|
||||
${result}= Run Process ${PYTHON} -c ${script} timeout=60s
|
||||
Should Be Equal As Integers ${result.rc} 0 resource add fs-directory failed: ${result.stderr}
|
||||
Should Contain ${result.stdout} resource add fs-directory succeeded after bootstrap
|
||||
Reference in New Issue
Block a user