Files
temp/features/steps/async_audit_recording_steps.py
freemo 02250473ad fix(ci): restore all CI quality gates to passing on master
Fix all failing CI quality gates (lint, unit_tests, format) without
suppressing any quality enforcement.

Root causes and fixes:

1. Format: features/steps/plan_namespaced_name_tdd_steps.py had trailing
   whitespace; fixed by running ruff format.

2. Unit tests - A2A JSON-RPC 2.0 migration (commit 9c6d6915) renamed
   A2aRequest fields (operation→method, request_id→id, a2a_version→jsonrpc)
   and A2aResponse fields (status+data→result, request_id→id) but did not
   update all step files and feature files:
   - a2a_jsonrpc_wire_format_steps.py: added use_step_matcher('re') and
     reset to 'parse' at end to prevent parallel test interference
   - a2a_facade_wiring_steps.py: updated operation= to method=, .status/.data
     to .result
   - a2a_facade_steps.py: updated request_id→id, a2a_version→jsonrpc,
     A2aResponse(request_id=..., status=...) to new API
   - m6_facade_steps.py: updated all old API usage
   - devcontainer_cleanup_steps.py: updated A2aRequest(operation=...)
   - plan_prompt_command_steps.py: updated A2aRequest(operation=...)
   - wf03_plan_prompt_confidence_steps.py: updated A2aRequest(operation=...)
   - consolidated_misc.feature: updated old A2aRequest/A2aResponse scenarios

3. Unit tests - Session CLI output changed (commit 0d5d9cf0 and others):
   - 'Session Created' → 'Session created' (lowercase)
   - 'Session Details' → 'Session Summary'
   - 'Sessions (N total)' → 'Sessions'
   - session list JSON: top-level 'total' → nested 'summary.total'
   - Fixed in: session_cli.feature, session_cli_coverage_boost.feature,
     session_cli_uncovered_branches.feature, session_list_error.feature,
     tdd_session_create_persist_steps.py

4. Unit tests - Plan list output changed (commit 1a07a891):
   - 'V3 Lifecycle Plans' → 'Plans'
   - 'Lifecycle Plans' → 'Plans'
   - Name column removed (restored in source)
   - Invariants column removed (restored in source)
   - Project truncation removed (restored in source)
   - Fixed in: plan_cli_cancel_revert_coverage.feature,
     plan_lifecycle_cli_coverage.feature, plan_cli_coverage_boost_steps.py,
     plan.py (source code restored)

5. Unit tests - Plan apply command now requires ULID (commit 300a5d6d):
   - plan_cli_coverage_r3.feature: updated 'PLAN-001' to valid ULID
   - plan_cli_coverage_r3_steps.py: added --yes flag, added new step for
     no-eligible-plans path

6. Unit tests - Various source code bugs:
   - ThoughtBlock: converted from @dataclass to Pydantic BaseModel
     (architecture test requires all dataclasses to use Pydantic)
   - session.py: added DatabaseError handling to export, import, tell commands
   - database.py: fixed rollback_to() to reuse checkpoint connection for writes
   - database.py: added _get_checkpoint_conn() helper
   - check-tls-cert.py: fixed SSLCertVerificationError.reason AttributeError

7. Unit tests - Test step bugs:
   - error_recovery_coverage_boost_steps.py: fixed invalid ULID _PLAN_ID
   - session_service_coverage_steps.py: fixed 'sha256:' prefix bug in checksum
   - database_models_new_coverage_steps.py: added 'name' field to session mock
   - async_audit_recording_steps.py: fixed Settings(audit_async=False) via env var
   - coverage_threshold_config_steps.py: added --coverage-min pattern support
   - m5_acms_smoke_steps.py: updated usage hint text
   - actor_cli_yaml_steps.py: updated 'Removed actor' → 'Actor removed'
   - aimodelscredentials_steps.py: set context.imported_class in import step
   - domain_base_model.feature: added missing 'When I examine model_config' step
   - tui_first_run_steps.py: fixed module reload to restore cleveragents.tui.*
     modules after test (prevented patch interference in subsequent tests)
   - tui_first_run_steps.py: added set_search('') step for empty string
   - resource_handler_base_coverage_r3_steps.py: use _MinimalHandler instead
     of DatabaseResourceHandler for NotImplementedError tests
   - resource_handler_crud.feature: updated to test new DatabaseHandler behavior
   - resource_handler_sandbox.feature: updated to test new DatabaseHandler behavior
   - tdd_json_decode_crash_persistence.feature: fixed @tdd_bug → @tdd_issue tags

8. Parallel test interference:
   - All step files using use_step_matcher('re') now reset to 'parse' at end
     to prevent global matcher state leaking to subsequent step files
2026-04-04 20:38:16 +00:00

349 lines
13 KiB
Python

"""Step definitions for async audit recording (issue #718).
Tests the write-behind queue, non-blocking record(), flush(), and
graceful shutdown behaviour of :class:`AuditService` in async mode.
Design note: async mode is only active when the service owns its
session (no injected session). Tests therefore create the service
with a ``database_url`` pointing to a temporary SQLite file so that
the background writer thread can open its own connection.
"""
from __future__ import annotations
import tempfile
import time
from behave import given, then, when
from behave.runner import Context
from sqlalchemy import create_engine
from cleveragents.application.services.audit_service import AuditService
from cleveragents.config.settings import Settings
from cleveragents.infrastructure.database.models import Base
__all__: list[str] = []
# ── Helpers ───────────────────────────────────────────────────────
def _make_settings(**overrides: object) -> Settings:
"""Create a fresh Settings instance with optional overrides."""
Settings._instance = None
base = Settings()
if overrides:
return base.model_copy(update=overrides)
return base
def _make_async_service(context: Context) -> AuditService:
"""Create an AuditService in async mode using a temp SQLite file.
The service owns its session so the background writer thread can
open its own connection to the same file.
"""
tmp = tempfile.mktemp(suffix=".db", prefix="audit_async_test_")
context._async_db_path = tmp
db_url = f"sqlite:///{tmp}"
settings = _make_settings(audit_async=True, database_url=db_url)
# Pre-create the table so the background thread doesn't race.
engine = create_engine(db_url, connect_args={"check_same_thread": False})
Base.metadata.create_all(engine, tables=[])
from cleveragents.infrastructure.database.models import AuditLogModel
Base.metadata.create_all(engine, tables=[AuditLogModel.__table__])
engine.dispose()
return AuditService(settings=settings, database_url=db_url)
def _make_sync_service(context: Context) -> AuditService:
"""Create an AuditService in synchronous mode using a temp SQLite file."""
tmp = tempfile.mktemp(suffix=".db", prefix="audit_sync_test_")
context._sync_db_path = tmp
db_url = f"sqlite:///{tmp}"
settings = _make_settings(audit_async=False, database_url=db_url)
from cleveragents.infrastructure.database.models import AuditLogModel
engine = create_engine(db_url)
Base.metadata.create_all(engine, tables=[AuditLogModel.__table__])
engine.dispose()
return AuditService(settings=settings, database_url=db_url)
# ── Given ─────────────────────────────────────────────────────────
@given("an async audit service")
def step_async_audit_service(context: Context) -> None:
context.async_service = _make_async_service(context)
context.async_entry = None
context.async_error = None
context.async_closed = False
@given("an async audit service used as context manager")
def step_async_audit_service_ctx(context: Context) -> None:
context.async_service = _make_async_service(context)
context.async_entry = None
context.async_error = None
context.async_closed = False
context.ctx_entry = None
@given("a synchronous audit service")
def step_sync_audit_service(context: Context) -> None:
context.async_service = _make_sync_service(context)
context.async_entry = None
context.async_error = None
# ── When ──────────────────────────────────────────────────────────
@when('I record a "{event_type}" event in async mode')
def step_record_async(context: Context, event_type: str) -> None:
start = time.monotonic()
context.async_entry = context.async_service.record(event_type=event_type)
context.record_elapsed = time.monotonic() - start
@when('I record {count:d} "{event_type}" events in async mode')
def step_record_multiple_async(context: Context, count: int, event_type: str) -> None:
for i in range(count):
context.async_service.record(event_type=event_type, plan_id=f"plan-async-{i}")
@when("I flush the async audit service")
def step_flush_async(context: Context) -> None:
context.async_error = None
try:
context.async_service.flush()
except Exception as exc:
context.async_error = exc
@when("I flush the async audit service again")
def step_flush_async_again(context: Context) -> None:
try:
context.async_service.flush()
except Exception as exc:
context.async_error = exc
@when("I close the async audit service")
def step_close_async(context: Context) -> None:
context.async_error = None
context.async_closed = True
try:
context.async_service.close()
except Exception as exc:
context.async_error = exc
@when("I close the async audit service again")
def step_close_async_again(context: Context) -> None:
try:
context.async_service.close()
except Exception as exc:
context.async_error = exc
@when('I record a "{event_type}" event synchronously')
def step_record_sync(context: Context, event_type: str) -> None:
context.async_entry = context.async_service.record(event_type=event_type)
@when('I record a "{event_type}" event inside the context manager')
def step_record_inside_ctx(context: Context, event_type: str) -> None:
with context.async_service as svc:
context.ctx_entry = svc.record(event_type=event_type)
context.async_closed = True
@when('I record events with plan_ids "{p1}" "{p2}" "{p3}" in async mode')
def step_record_ordered_async(context: Context, p1: str, p2: str, p3: str) -> None:
for pid in (p1, p2, p3):
context.async_service.record(event_type="plan_applied", plan_id=pid)
context.ordered_plan_ids = [p1, p2, p3]
@when('I record an event with invalid type "{event_type}" in async mode')
def step_record_invalid_async(context: Context, event_type: str) -> None:
try:
context.async_service.record(event_type=event_type)
context.async_error = None
except ValueError as exc:
context.async_error = exc
# ── Then ──────────────────────────────────────────────────────────
@then("the record call should return immediately")
def step_record_returned_immediately(context: Context) -> None:
# "Immediately" is defined as < 100 ms — a synchronous SQLite write
# typically takes 5-50 ms; the async enqueue should be < 1 ms.
assert context.record_elapsed < 0.1, (
f"record() took {context.record_elapsed:.3f}s — expected < 0.1s"
)
@then('the returned entry should have event_type "{expected}"')
def step_returned_entry_event_type(context: Context, expected: str) -> None:
assert context.async_entry is not None
assert context.async_entry.event_type == expected, (
f"Expected event_type={expected!r}, got {context.async_entry.event_type!r}"
)
@then("the returned entry id should be -1")
def step_returned_entry_id_minus_one(context: Context) -> None:
assert context.async_entry is not None
assert context.async_entry.id == -1, (
f"Expected id=-1 (async placeholder), got {context.async_entry.id}"
)
@then("the returned entry id should be a positive integer")
def step_returned_entry_id_positive(context: Context) -> None:
assert context.async_entry is not None
assert context.async_entry.id > 0, (
f"Expected positive id (sync mode), got {context.async_entry.id}"
)
@then("the audit log should contain {count:d} persisted entry")
@then("the audit log should contain {count:d} persisted entries")
def step_persisted_count(context: Context, count: int) -> None:
actual = context.async_service.count()
assert actual == count, f"Expected {count} persisted entries, got {actual}"
@then("the audit log should contain {count:d} persisted entries after close")
def step_persisted_count_after_close(context: Context, count: int) -> None:
# After close() we need a fresh read session to verify persistence.
db_path = getattr(context, "_async_db_path", None)
if db_path:
from sqlalchemy import create_engine as _ce
from sqlalchemy.orm import sessionmaker as _sm
from cleveragents.infrastructure.database.models import AuditLogModel as _M
engine = _ce(f"sqlite:///{db_path}")
session = _sm(bind=engine)()
actual = session.query(_M).count()
session.close()
engine.dispose()
else:
actual = context.async_service.count()
assert actual == count, (
f"Expected {count} persisted entries after close, got {actual}"
)
@then("the audit log should contain 1 persisted entry after context exit")
def step_persisted_after_ctx_exit(context: Context) -> None:
db_path = getattr(context, "_async_db_path", None)
if db_path:
from sqlalchemy import create_engine as _ce
from sqlalchemy.orm import sessionmaker as _sm
from cleveragents.infrastructure.database.models import AuditLogModel as _M
engine = _ce(f"sqlite:///{db_path}")
session = _sm(bind=engine)()
actual = session.query(_M).count()
session.close()
engine.dispose()
else:
actual = context.async_service.count()
assert actual == 1, f"Expected 1 persisted entry after context exit, got {actual}"
@then("the audit log should contain 1 persisted entry immediately")
def step_persisted_immediately(context: Context) -> None:
actual = context.async_service.count()
assert actual == 1, (
f"Expected 1 persisted entry immediately (sync mode), got {actual}"
)
@then("the background writer thread should be alive")
def step_writer_thread_alive(context: Context) -> None:
thread = context.async_service._writer_thread
assert thread is not None, "Expected a background writer thread"
assert thread.is_alive(), "Expected background writer thread to be alive"
@then("the background writer thread should be stopped")
def step_writer_thread_stopped(context: Context) -> None:
thread = context.async_service._writer_thread
assert thread is not None, "Expected a background writer thread"
assert not thread.is_alive(), "Expected background writer thread to be stopped"
@then("the entries should be persisted in enqueue order")
def step_entries_in_order(context: Context) -> None:
entries = context.async_service.list_entries(event_type="plan_applied", limit=100)
# list_entries returns newest-first; reverse to get enqueue order.
persisted_ids = [e.plan_id for e in reversed(entries)]
assert persisted_ids == context.ordered_plan_ids, (
f"Expected order {context.ordered_plan_ids}, got {persisted_ids}"
)
@then("a ValueError should be raised immediately")
def step_value_error_raised(context: Context) -> None:
assert isinstance(context.async_error, ValueError), (
f"Expected ValueError, got {type(context.async_error)}"
)
@then("no async audit error should be raised")
def step_no_async_audit_error_raised(context: Context) -> None:
assert context.async_error is None, f"Expected no error, got {context.async_error}"
# ── Settings steps ────────────────────────────────────────────────
@then("audit_async should be True")
def step_audit_async_true(context: Context) -> None:
assert context.settings.audit_async is True, (
f"Expected audit_async=True, got {context.settings.audit_async}"
)
@then("audit_async should be False")
def step_audit_async_false(context: Context) -> None:
assert context.settings.audit_async is False, (
f"Expected audit_async=False, got {context.settings.audit_async}"
)
@then("audit_queue_maxsize should be {expected:d}")
def step_audit_queue_maxsize(context: Context, expected: int) -> None:
assert context.settings.audit_queue_maxsize == expected, (
f"Expected audit_queue_maxsize={expected}, "
f"got {context.settings.audit_queue_maxsize}"
)
# ── Settings Given steps ──────────────────────────────────────────
@given("settings with audit_async False")
def step_settings_audit_async_false(context: Context) -> None:
import os
Settings._instance = None
old_val = os.environ.get("CLEVERAGENTS_AUDIT_ASYNC")
os.environ["CLEVERAGENTS_AUDIT_ASYNC"] = "false"
context.settings = Settings()
# Restore env var after test
if old_val is None:
os.environ.pop("CLEVERAGENTS_AUDIT_ASYNC", None)
else:
os.environ["CLEVERAGENTS_AUDIT_ASYNC"] = old_val
Settings._instance = None