Commit Graph

3 Commits

Author SHA1 Message Date
CoreRasurae 00881a3e5f feat(observability): implement Event System Domain Event Taxonomy (full EventType enum + DomainEvent model)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m26s
CI / build (pull_request) Successful in 14s
CI / typecheck (pull_request) Successful in 4m2s
CI / quality (pull_request) Successful in 4m2s
CI / security (pull_request) Successful in 4m23s
CI / integration_tests (pull_request) Successful in 9m35s
CI / unit_tests (pull_request) Successful in 9m39s
CI / docker (pull_request) Successful in 2m15s
CI / e2e_tests (pull_request) Successful in 12m32s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 23s
CI / lint (push) Successful in 3m17s
CI / typecheck (push) Successful in 4m11s
CI / quality (push) Successful in 4m11s
CI / security (push) Successful in 4m28s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 9m13s
CI / unit_tests (push) Successful in 9m26s
CI / e2e_tests (push) Successful in 10m20s
CI / docker (push) Successful in 1m16s
CI / coverage (push) Successful in 12m2s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 33m17s
CI / benchmark-regression (pull_request) Successful in 57m23s
Verified and completed the Event System Domain Event Taxonomy implementation.

Gap analysis:
- EventType enum (38 types across 12 domains): ALREADY EXISTED, verified complete
- DomainEvent Pydantic model (all 9 fields): ALREADY EXISTED, verified complete
- EventBus Protocol (emit + subscribe): ALREADY EXISTED, verified complete
- ReactiveEventBus (RxPY Subject, stream property): ALREADY EXISTED, verified
- LoggingEventBus (structlog-based): ALREADY EXISTED, verified complete

Added:
- In-memory audit_log on ReactiveEventBus (list[DomainEvent] with defensive copy)
- Emit ordering aligned with specification: RxPY stream push, then audit append,
  then handler dispatch (§Event System)
- Clarified audit_log docstring: volatile in-memory log, not durable SQLite
  persistence (durable persistence wired separately via audit service layer)
- Behave feature: features/observability/event_system_taxonomy.feature (36 scenarios)
- Step definitions: features/steps/event_system_taxonomy_steps.py
- Robot integration test: robot/event_system_taxonomy_integration.robot (13 test cases)
- Robot helper: robot/helper_event_system_taxonomy.py (13 subcommands)
- ASV benchmark: benchmarks/bench_event_bus.py (5 benchmark suites)
- vulture_whitelist.py: added audit_log property

Code review fixes applied:
- Reordered emit() to match spec: on_next → audit_log → handlers (was: audit → on_next → handlers)
- Clarified audit_log docstring to distinguish volatile in-memory log from spec SQLite table
- Fixed benchmark TaxonomyAuditLogSuite: parameterized setup instead of inline construction
- Added teardown to TaxonomyLoggingEmitSuite to restore logging.disable(NOTSET)
- Clear _received list between benchmark iterations to reduce noise
- Catch specific pydantic.ValidationError in frozen model mutation tests
- Consolidated duplicate singular/plural step definitions
- Tightened EventType count threshold from >=30 to >=38

Quality gates:
- lint: PASSED
- typecheck: PASSED (0 errors)
- unit_tests: 36/36 new scenarios PASSED (pre-existing 12 flaky failures unchanged)
- integration_tests: 1483/1483 PASSED
- security_scan: PASSED
- dead_code: PASSED

ISSUES CLOSED: #587
2026-03-26 11:52:52 +00:00
CoreRasurae 3e3e9b4b5d feat(observability): wire AuditService.record() into domain services via EventBus auto-dispatch
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 18s
CI / e2e_tests (pull_request) Successful in 30s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 40s
CI / unit_tests (pull_request) Successful in 3m19s
CI / integration_tests (pull_request) Successful in 3m36s
CI / docker (pull_request) Successful in 40s
CI / coverage (pull_request) Successful in 5m38s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 19s
CI / build (push) Successful in 15s
CI / security (push) Successful in 38s
CI / e2e_tests (push) Successful in 28s
CI / typecheck (push) Successful in 42s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m14s
CI / docker (push) Successful in 10s
CI / integration_tests (push) Successful in 3m33s
CI / coverage (push) Successful in 6m3s
CI / benchmark-publish (push) Successful in 19m14s
CI / benchmark-regression (pull_request) Successful in 36m42s
Implements AuditEventSubscriber that subscribes to all 9 security-relevant
EventType members and persists redacted audit entries via AuditService.record().

Key components:
- AuditEventSubscriber: bridges EventBus and AuditService (SEC7)
- SECURITY_EVENT_MAP: maps EventType enum to audit type strings
- Redaction via redact_dict() on event details before persistence
- Graceful error handling: failures logged, never propagated

Post-review fixes applied:
- BUG-1: Remove dead correlation_id null-check guard (DomainEvent.correlation_id
  is always non-None via ULID default_factory)
- SEC-2: Redact exception messages in warning logs via redact_value() to prevent
  potential leakage of sensitive internal state (e.g. DB connection strings)
- PERF-3: Pre-generate unique DomainEvent instances in ASV benchmark setup to
  avoid skew from reusing a single frozen object
- Wire event_bus from the DI container into CorrectionService (plan.py),
  ConfigService (config.py, skill.py x2, server.py), and
  PersistentSessionService (session.py) at their CLI construction sites.

Closes #581
2026-03-12 23:32:48 +00:00
freemo 958eb0c060 feat(observability): implement Metrics Collection Framework (14 metric types with Histogram/Counter/Gauge)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / typecheck (pull_request) Successful in 37s
CI / security (pull_request) Successful in 51s
CI / unit_tests (pull_request) Successful in 2m41s
CI / integration_tests (pull_request) Successful in 3m19s
CI / docker (pull_request) Successful in 51s
CI / coverage (pull_request) Successful in 5m7s
CI / benchmark-regression (pull_request) Successful in 33m26s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 33s
CI / typecheck (push) Successful in 35s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m2s
CI / integration_tests (push) Successful in 3m12s
CI / docker (push) Successful in 49s
CI / coverage (push) Successful in 6m10s
CI / benchmark-publish (push) Has been cancelled
Implement the structured metrics collection framework covering 14
operational metric types with proper Histogram, Counter, and Gauge
semantics per the spec (Architecture > Observability > Metrics
Collection, lines ~43805-43825).

Domain layer:
- Add MetricType enum (HISTOGRAM, COUNTER, GAUGE) to metrics.py
- Add MetricDefinition model and METRIC_DEFINITIONS registry mapping
  all 14 OperationalMetricKey values to their metric types
- Extend MetricCollector with typed factory methods (histogram, counter,
  gauge) and 14 convenience methods (plan_duration, plan_cost,
  plan_decision_count, subplan_count, actor_invocation_count,
  actor_latency, tool_invocation_count, tool_error_rate,
  context_build_time, context_token_count, llm_call_count,
  llm_total_tokens, llm_total_cost, llm_avg_latency)
- Extend MetricEntry with optional metric_type field that auto-resolves
  from METRIC_DEFINITIONS via MetricCollector.record()

Infrastructure layer:
- Add MetricsEmitter (infrastructure/observability/metrics_emitter.py)
  with emit(), emit_batch(), from_settings(), and enabled/disabled
  support for structured log emission in local mode
- Add metrics_log_processor (config/metrics_processor.py) for structlog
  integration

Configuration:
- Add metrics_enabled and metrics_export_prometheus settings
- Register MetricsEmitter as DI Singleton in application container

Instrumentation:
- Add best-effort metric emission in PlanExecutor for plan_duration
  (both runtime and stub execute paths) and plan_decision_count
  (strategize path) via _try_emit_metric helper that tolerates invalid
  plan IDs in test fixtures

Testing:
- 34 Behave BDD scenarios (features/observability/metrics_collection.feature)
- 8 Robot Framework integration tests (robot/metrics_collection.robot)
- ASV benchmark suite (benchmarks/bench_metrics_collection.py)

Closes #579
2026-03-10 17:23:05 +00:00