00881a3e5f
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m26s
CI / build (pull_request) Successful in 14s
CI / typecheck (pull_request) Successful in 4m2s
CI / quality (pull_request) Successful in 4m2s
CI / security (pull_request) Successful in 4m23s
CI / integration_tests (pull_request) Successful in 9m35s
CI / unit_tests (pull_request) Successful in 9m39s
CI / docker (pull_request) Successful in 2m15s
CI / e2e_tests (pull_request) Successful in 12m32s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 23s
CI / lint (push) Successful in 3m17s
CI / typecheck (push) Successful in 4m11s
CI / quality (push) Successful in 4m11s
CI / security (push) Successful in 4m28s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 9m13s
CI / unit_tests (push) Successful in 9m26s
CI / e2e_tests (push) Successful in 10m20s
CI / docker (push) Successful in 1m16s
CI / coverage (push) Successful in 12m2s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 33m17s
CI / benchmark-regression (pull_request) Successful in 57m23s
Verified and completed the Event System Domain Event Taxonomy implementation. Gap analysis: - EventType enum (38 types across 12 domains): ALREADY EXISTED, verified complete - DomainEvent Pydantic model (all 9 fields): ALREADY EXISTED, verified complete - EventBus Protocol (emit + subscribe): ALREADY EXISTED, verified complete - ReactiveEventBus (RxPY Subject, stream property): ALREADY EXISTED, verified - LoggingEventBus (structlog-based): ALREADY EXISTED, verified complete Added: - In-memory audit_log on ReactiveEventBus (list[DomainEvent] with defensive copy) - Emit ordering aligned with specification: RxPY stream push, then audit append, then handler dispatch (§Event System) - Clarified audit_log docstring: volatile in-memory log, not durable SQLite persistence (durable persistence wired separately via audit service layer) - Behave feature: features/observability/event_system_taxonomy.feature (36 scenarios) - Step definitions: features/steps/event_system_taxonomy_steps.py - Robot integration test: robot/event_system_taxonomy_integration.robot (13 test cases) - Robot helper: robot/helper_event_system_taxonomy.py (13 subcommands) - ASV benchmark: benchmarks/bench_event_bus.py (5 benchmark suites) - vulture_whitelist.py: added audit_log property Code review fixes applied: - Reordered emit() to match spec: on_next → audit_log → handlers (was: audit → on_next → handlers) - Clarified audit_log docstring to distinguish volatile in-memory log from spec SQLite table - Fixed benchmark TaxonomyAuditLogSuite: parameterized setup instead of inline construction - Added teardown to TaxonomyLoggingEmitSuite to restore logging.disable(NOTSET) - Clear _received list between benchmark iterations to reduce noise - Catch specific pydantic.ValidationError in frozen model mutation tests - Consolidated duplicate singular/plural step definitions - Tightened EventType count threshold from >=30 to >=38 Quality gates: - lint: PASSED - typecheck: PASSED (0 errors) - unit_tests: 36/36 new scenarios PASSED (pre-existing 12 flaky failures unchanged) - integration_tests: 1483/1483 PASSED - security_scan: PASSED - dead_code: PASSED ISSUES CLOSED: #587
175 lines
5.2 KiB
Python
175 lines
5.2 KiB
Python
"""ASV benchmarks for Event System Domain Event Taxonomy.
|
|
|
|
Measures:
|
|
- DomainEvent construction throughput (minimal, with plan_id, with details)
|
|
- ReactiveEventBus emit with audit_log overhead
|
|
- ReactiveEventBus subscribe registration
|
|
- Fan-out: emit to N subscribers with audit persistence
|
|
- Audit log retrieval cost
|
|
- Log correlation (filtering events by correlation_id)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import importlib
|
|
import logging
|
|
import sys
|
|
from pathlib import Path
|
|
from typing import ClassVar
|
|
|
|
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
|
if _SRC not in sys.path:
|
|
sys.path.insert(0, _SRC)
|
|
|
|
import cleveragents # noqa: E402
|
|
|
|
importlib.reload(cleveragents)
|
|
|
|
from cleveragents.infrastructure.events import ( # noqa: E402
|
|
DomainEvent,
|
|
EventType,
|
|
LoggingEventBus,
|
|
ReactiveEventBus,
|
|
)
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DomainEvent construction
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TaxonomyDomainEventSuite:
|
|
"""Benchmark DomainEvent construction and serialisation."""
|
|
|
|
timeout = 60
|
|
|
|
def setup(self) -> None:
|
|
self.event_type = EventType.PLAN_CREATED
|
|
self.plan_id = "01ARZ3NDEKTSV4RRFFQ69G5FAV"
|
|
|
|
def time_create_minimal(self) -> None:
|
|
DomainEvent(event_type=self.event_type)
|
|
|
|
def time_create_with_all_fields(self) -> None:
|
|
DomainEvent(
|
|
event_type=self.event_type,
|
|
plan_id=self.plan_id,
|
|
root_plan_id=self.plan_id,
|
|
session_id=self.plan_id,
|
|
actor_name="local/strategist",
|
|
project_name="my-project",
|
|
details={"phase": "strategize", "actor": "strategists/planner"},
|
|
)
|
|
|
|
def time_json_roundtrip(self) -> None:
|
|
event = DomainEvent(
|
|
event_type=self.event_type,
|
|
plan_id=self.plan_id,
|
|
details={"phase": "execute"},
|
|
)
|
|
DomainEvent.model_validate_json(event.model_dump_json())
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ReactiveEventBus emit with audit_log
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TaxonomyReactiveEmitSuite:
|
|
"""Benchmark ReactiveEventBus.emit() with audit_log persistence."""
|
|
|
|
timeout = 60
|
|
|
|
def setup(self) -> None:
|
|
self.bus = ReactiveEventBus()
|
|
self.event = DomainEvent(event_type=EventType.PLAN_CREATED)
|
|
self.decision_event = DomainEvent(event_type=EventType.DECISION_CREATED)
|
|
|
|
self._received: list[DomainEvent] = []
|
|
self.bus.subscribe(EventType.PLAN_CREATED, self._received.append)
|
|
|
|
def time_emit_no_subscribers(self) -> None:
|
|
self._received.clear()
|
|
self.bus.clear_audit_log()
|
|
self.bus.emit(self.decision_event)
|
|
|
|
def time_emit_with_one_subscriber(self) -> None:
|
|
self._received.clear()
|
|
self.bus.clear_audit_log()
|
|
self.bus.emit(self.event)
|
|
|
|
def time_emit_100_events(self) -> None:
|
|
self._received.clear()
|
|
self.bus.clear_audit_log()
|
|
for _ in range(100):
|
|
self.bus.emit(self.event)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Audit log retrieval
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TaxonomyAuditLogSuite:
|
|
"""Benchmark audit_log property access (defensive copy)."""
|
|
|
|
timeout = 60
|
|
params: ClassVar[list[int]] = [100, 1000]
|
|
param_names: ClassVar[list[str]] = ["num_events"]
|
|
|
|
def setup(self, num_events: int) -> None:
|
|
self.bus = ReactiveEventBus()
|
|
for _ in range(num_events):
|
|
self.bus.emit(DomainEvent(event_type=EventType.PLAN_CREATED))
|
|
|
|
def time_audit_log_snapshot(self, num_events: int) -> None:
|
|
_ = self.bus.audit_log
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Fan-out with audit
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TaxonomyFanOutSuite:
|
|
"""Benchmark event fan-out to multiple subscribers with audit persistence."""
|
|
|
|
timeout = 60
|
|
params: ClassVar[list[int]] = [1, 5, 10, 50]
|
|
param_names: ClassVar[list[str]] = ["num_subscribers"]
|
|
|
|
def setup(self, num_subscribers: int) -> None:
|
|
self.bus = ReactiveEventBus()
|
|
for _ in range(num_subscribers):
|
|
self.bus.subscribe(EventType.PLAN_CREATED, lambda e: None)
|
|
self.event = DomainEvent(event_type=EventType.PLAN_CREATED)
|
|
|
|
def time_emit(self, num_subscribers: int) -> None:
|
|
self.bus.clear_audit_log()
|
|
self.bus.emit(self.event)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# LoggingEventBus emit
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TaxonomyLoggingEmitSuite:
|
|
"""Benchmark LoggingEventBus.emit() overhead."""
|
|
|
|
timeout = 60
|
|
|
|
def setup(self) -> None:
|
|
logging.disable(logging.CRITICAL)
|
|
self.bus = LoggingEventBus()
|
|
self.event = DomainEvent(event_type=EventType.DECISION_CREATED)
|
|
|
|
def teardown(self) -> None:
|
|
logging.disable(logging.NOTSET)
|
|
|
|
def time_emit_single(self) -> None:
|
|
self.bus.emit(self.event)
|
|
|
|
def time_emit_100(self) -> None:
|
|
for _ in range(100):
|
|
self.bus.emit(self.event)
|