Commit Graph

5 Commits

Author SHA1 Message Date
HAL9000 8ed8750160 fix(audit): forward DomainEvent.timestamp to AuditService.record()
CI / push-validation (pull_request) Successful in 23s
CI / helm (pull_request) Successful in 30s
CI / build (pull_request) Successful in 43s
CI / lint (pull_request) Successful in 55s
CI / quality (pull_request) Successful in 54s
CI / typecheck (pull_request) Successful in 57s
CI / security (pull_request) Successful in 1m28s
CI / integration_tests (pull_request) Successful in 3m4s
CI / unit_tests (pull_request) Successful in 7m30s
CI / docker (pull_request) Failing after 14m0s
CI / coverage (pull_request) Failing after 21m7s
CI / status-check (pull_request) Has been cancelled
AuditService.record() was generating its own timestamp internally,
discarding the original DomainEvent.timestamp. This means audit entries
recorded when an event was audited, not when the domain event actually
occurred, breaking forensic accuracy per §Audit Logging (SEC7).

Changes:
- Add `timestamp: datetime | None = None` keyword parameter to
  AuditService.record(). When provided, uses it as created_at;
  falls back to datetime.now(tz=UTC) for backward compatibility.
  Applied to both the async queue path and the synchronous DB path.
- AuditEventSubscriber._handle_event() now passes timestamp=event.timestamp
  so the original event creation time is preserved in audit entries.
- Add 3 Behave BDD scenarios covering: full pipeline timestamp
  preservation, direct record() with explicit timestamp, and backward
  compatibility (record() without timestamp auto-generates created_at).
- Add preserve_event_timestamp Robot integration test and helper subcommand.
- Add static source check in security_audit.robot verifying the
  timestamp parameter signature exists.

ISSUES CLOSED: #719
2026-05-29 16:36:39 -04:00
brent.edwards e3d3c7926d feat(events): add user_identity field to DomainEvent and propagate through event pipeline (#1257)
CI / integration_tests (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / build (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / security (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / docker (push) Has been cancelled
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-04-02 16:59:07 +00:00
CoreRasurae 00881a3e5f feat(observability): implement Event System Domain Event Taxonomy (full EventType enum + DomainEvent model)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m26s
CI / build (pull_request) Successful in 14s
CI / typecheck (pull_request) Successful in 4m2s
CI / quality (pull_request) Successful in 4m2s
CI / security (pull_request) Successful in 4m23s
CI / integration_tests (pull_request) Successful in 9m35s
CI / unit_tests (pull_request) Successful in 9m39s
CI / docker (pull_request) Successful in 2m15s
CI / e2e_tests (pull_request) Successful in 12m32s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 23s
CI / lint (push) Successful in 3m17s
CI / typecheck (push) Successful in 4m11s
CI / quality (push) Successful in 4m11s
CI / security (push) Successful in 4m28s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 9m13s
CI / unit_tests (push) Successful in 9m26s
CI / e2e_tests (push) Successful in 10m20s
CI / docker (push) Successful in 1m16s
CI / coverage (push) Successful in 12m2s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 33m17s
CI / benchmark-regression (pull_request) Successful in 57m23s
Verified and completed the Event System Domain Event Taxonomy implementation.

Gap analysis:
- EventType enum (38 types across 12 domains): ALREADY EXISTED, verified complete
- DomainEvent Pydantic model (all 9 fields): ALREADY EXISTED, verified complete
- EventBus Protocol (emit + subscribe): ALREADY EXISTED, verified complete
- ReactiveEventBus (RxPY Subject, stream property): ALREADY EXISTED, verified
- LoggingEventBus (structlog-based): ALREADY EXISTED, verified complete

Added:
- In-memory audit_log on ReactiveEventBus (list[DomainEvent] with defensive copy)
- Emit ordering aligned with specification: RxPY stream push, then audit append,
  then handler dispatch (§Event System)
- Clarified audit_log docstring: volatile in-memory log, not durable SQLite
  persistence (durable persistence wired separately via audit service layer)
- Behave feature: features/observability/event_system_taxonomy.feature (36 scenarios)
- Step definitions: features/steps/event_system_taxonomy_steps.py
- Robot integration test: robot/event_system_taxonomy_integration.robot (13 test cases)
- Robot helper: robot/helper_event_system_taxonomy.py (13 subcommands)
- ASV benchmark: benchmarks/bench_event_bus.py (5 benchmark suites)
- vulture_whitelist.py: added audit_log property

Code review fixes applied:
- Reordered emit() to match spec: on_next → audit_log → handlers (was: audit → on_next → handlers)
- Clarified audit_log docstring to distinguish volatile in-memory log from spec SQLite table
- Fixed benchmark TaxonomyAuditLogSuite: parameterized setup instead of inline construction
- Added teardown to TaxonomyLoggingEmitSuite to restore logging.disable(NOTSET)
- Clear _received list between benchmark iterations to reduce noise
- Catch specific pydantic.ValidationError in frozen model mutation tests
- Consolidated duplicate singular/plural step definitions
- Tightened EventType count threshold from >=30 to >=38

Quality gates:
- lint: PASSED
- typecheck: PASSED (0 errors)
- unit_tests: 36/36 new scenarios PASSED (pre-existing 12 flaky failures unchanged)
- integration_tests: 1483/1483 PASSED
- security_scan: PASSED
- dead_code: PASSED

ISSUES CLOSED: #587
2026-03-26 11:52:52 +00:00
CoreRasurae 3e3e9b4b5d feat(observability): wire AuditService.record() into domain services via EventBus auto-dispatch
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 18s
CI / e2e_tests (pull_request) Successful in 30s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 40s
CI / unit_tests (pull_request) Successful in 3m19s
CI / integration_tests (pull_request) Successful in 3m36s
CI / docker (pull_request) Successful in 40s
CI / coverage (pull_request) Successful in 5m38s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 19s
CI / build (push) Successful in 15s
CI / security (push) Successful in 38s
CI / e2e_tests (push) Successful in 28s
CI / typecheck (push) Successful in 42s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m14s
CI / docker (push) Successful in 10s
CI / integration_tests (push) Successful in 3m33s
CI / coverage (push) Successful in 6m3s
CI / benchmark-publish (push) Successful in 19m14s
CI / benchmark-regression (pull_request) Successful in 36m42s
Implements AuditEventSubscriber that subscribes to all 9 security-relevant
EventType members and persists redacted audit entries via AuditService.record().

Key components:
- AuditEventSubscriber: bridges EventBus and AuditService (SEC7)
- SECURITY_EVENT_MAP: maps EventType enum to audit type strings
- Redaction via redact_dict() on event details before persistence
- Graceful error handling: failures logged, never propagated

Post-review fixes applied:
- BUG-1: Remove dead correlation_id null-check guard (DomainEvent.correlation_id
  is always non-None via ULID default_factory)
- SEC-2: Redact exception messages in warning logs via redact_value() to prevent
  potential leakage of sensitive internal state (e.g. DB connection strings)
- PERF-3: Pre-generate unique DomainEvent instances in ASV benchmark setup to
  avoid skew from reusing a single frozen object
- Wire event_bus from the DI container into CorrectionService (plan.py),
  ConfigService (config.py, skill.py x2, server.py), and
  PersistentSessionService (session.py) at their CLI construction sites.

Closes #581
2026-03-12 23:32:48 +00:00
freemo 958eb0c060 feat(observability): implement Metrics Collection Framework (14 metric types with Histogram/Counter/Gauge)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / typecheck (pull_request) Successful in 37s
CI / security (pull_request) Successful in 51s
CI / unit_tests (pull_request) Successful in 2m41s
CI / integration_tests (pull_request) Successful in 3m19s
CI / docker (pull_request) Successful in 51s
CI / coverage (pull_request) Successful in 5m7s
CI / benchmark-regression (pull_request) Successful in 33m26s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 33s
CI / typecheck (push) Successful in 35s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m2s
CI / integration_tests (push) Successful in 3m12s
CI / docker (push) Successful in 49s
CI / coverage (push) Successful in 6m10s
CI / benchmark-publish (push) Has been cancelled
Implement the structured metrics collection framework covering 14
operational metric types with proper Histogram, Counter, and Gauge
semantics per the spec (Architecture > Observability > Metrics
Collection, lines ~43805-43825).

Domain layer:
- Add MetricType enum (HISTOGRAM, COUNTER, GAUGE) to metrics.py
- Add MetricDefinition model and METRIC_DEFINITIONS registry mapping
  all 14 OperationalMetricKey values to their metric types
- Extend MetricCollector with typed factory methods (histogram, counter,
  gauge) and 14 convenience methods (plan_duration, plan_cost,
  plan_decision_count, subplan_count, actor_invocation_count,
  actor_latency, tool_invocation_count, tool_error_rate,
  context_build_time, context_token_count, llm_call_count,
  llm_total_tokens, llm_total_cost, llm_avg_latency)
- Extend MetricEntry with optional metric_type field that auto-resolves
  from METRIC_DEFINITIONS via MetricCollector.record()

Infrastructure layer:
- Add MetricsEmitter (infrastructure/observability/metrics_emitter.py)
  with emit(), emit_batch(), from_settings(), and enabled/disabled
  support for structured log emission in local mode
- Add metrics_log_processor (config/metrics_processor.py) for structlog
  integration

Configuration:
- Add metrics_enabled and metrics_export_prometheus settings
- Register MetricsEmitter as DI Singleton in application container

Instrumentation:
- Add best-effort metric emission in PlanExecutor for plan_duration
  (both runtime and stub execute paths) and plan_decision_count
  (strategize path) via _try_emit_metric helper that tolerates invalid
  plan IDs in test fixtures

Testing:
- 34 Behave BDD scenarios (features/observability/metrics_collection.feature)
- 8 Robot Framework integration tests (robot/metrics_collection.robot)
- ASV benchmark suite (benchmarks/bench_metrics_collection.py)

Closes #579
2026-03-10 17:23:05 +00:00