cleveragents/cleveragents-core

Fork 3

UAT: `MetricsEmitter` does not actually emit metrics to any persistent store — metrics are lost after process exit #5839

New issue

Open

opened 2026-04-09 10:33:21 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 10:33:21 +00:00

Owner

Bug Report

Feature Area: Observability — Metrics Persistence
Milestone: v3.6.0 (M7)
Severity: Priority/Backlog — metrics are ephemeral and not queryable

What Was Tested

The MetricsEmitter and MetricEntry persistence path were inspected against the specification's metrics requirements.

Expected Behavior (from spec)

The specification states:

A dedicated metrics framework must collect performance data, including histograms, counters, and gauges for key operations.
There are 14 specific metric keys defined for tracking things like tool execution latency and plan phase duration.

Metrics should be:

Collected during plan execution
Persisted for later querying/analysis
Exportable in Prometheus format

Actual Behavior

The MetricsEmitter.emit() method only writes metrics as structured log entries via structlog:

def emit(self, entry: MetricEntry) -> None:
    ...
    logger.info("metric.recorded", **log_data)

This means:

Metrics are only written to the log file (if logging is configured)
Metrics are not persisted to the database — there is no metrics table in the SQLite schema
Metrics are not queryable after the fact
Metrics are not aggregated — each emission is a separate log line with no rollup
The TraceService.compute_metrics() creates MetricEntry objects but they are never passed to MetricsEmitter.emit()

The MetricEntry model is well-defined with all 14 metric keys, but the pipeline from metric creation → emission → persistence → query is broken at the emission step.

Code Location

src/cleveragents/infrastructure/observability/metrics_emitter.py — emit() only logs
src/cleveragents/application/services/trace_service.py — compute_metrics() creates entries but doesn't emit them
src/cleveragents/infrastructure/database/models.py — no metrics table exists

Impact

Plan execution metrics (latency, cost, token counts) are not queryable
Performance regression analysis is impossible without a persistent metrics store
The 14 defined OperationalMetricKey values are effectively unused in production
MetricsEmitter.emit_batch() returns a count but the data is only in logs

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: Observability — Metrics Persistence **Milestone**: v3.6.0 (M7) **Severity**: Priority/Backlog — metrics are ephemeral and not queryable ### What Was Tested The `MetricsEmitter` and `MetricEntry` persistence path were inspected against the specification's metrics requirements. ### Expected Behavior (from spec) The specification states: > A dedicated metrics framework must collect performance data, including histograms, counters, and gauges for key operations. > There are 14 specific metric keys defined for tracking things like tool execution latency and plan phase duration. Metrics should be: 1. Collected during plan execution 2. Persisted for later querying/analysis 3. Exportable in Prometheus format ### Actual Behavior The `MetricsEmitter.emit()` method only writes metrics as structured log entries via `structlog`: ```python def emit(self, entry: MetricEntry) -> None: ... logger.info("metric.recorded", **log_data) ``` This means: 1. Metrics are only written to the log file (if logging is configured) 2. Metrics are **not persisted to the database** — there is no `metrics` table in the SQLite schema 3. Metrics are **not queryable** after the fact 4. Metrics are **not aggregated** — each emission is a separate log line with no rollup 5. The `TraceService.compute_metrics()` creates `MetricEntry` objects but they are never passed to `MetricsEmitter.emit()` The `MetricEntry` model is well-defined with all 14 metric keys, but the pipeline from metric creation → emission → persistence → query is broken at the emission step. ### Code Location - `src/cleveragents/infrastructure/observability/metrics_emitter.py` — `emit()` only logs - `src/cleveragents/application/services/trace_service.py` — `compute_metrics()` creates entries but doesn't emit them - `src/cleveragents/infrastructure/database/models.py` — no `metrics` table exists ### Impact - Plan execution metrics (latency, cost, token counts) are not queryable - Performance regression analysis is impossible without a persistent metrics store - The 14 defined `OperationalMetricKey` values are effectively unused in production - `MetricsEmitter.emit_batch()` returns a count but the data is only in logs --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 10:55:34 +00:00

~~HAL9000 referenced this issue 2026-04-09 10:56:39 +00:00~~

[AUTO-UAT-POOL] UAT Testing Report (Cycle 1) #5750

HAL9000 added

and removed

labels

2026-04-09 11:13:58 +00:00

HAL9000 referenced this issue

2026-04-09 11:21:43 +00:00

[AUTO-ARCH] Architecture Review (Cycle 1) #5767

HAL9000 referenced this issue

2026-04-09 11:21:49 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 29) #5788

HAL9000 added this to the v3.6.0 milestone

2026-04-09 13:21:26 +00:00

HAL9000 commented

2026-04-09 13:26:03 +00:00

Author

Owner

Milestone compliance fix applied:

Assigned to milestone: v3.6.0 (Advanced Concepts & Deferred Features)
Reason: Issue is State/Verified but had no milestone. Issue body explicitly states "Milestone: v3.6.0 (M7)" — metrics persistence belongs to v3.6.0 scope.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Milestone compliance fix applied: - Assigned to milestone: **v3.6.0** (Advanced Concepts & Deferred Features) - Reason: Issue is `State/Verified` but had no milestone. Issue body explicitly states "Milestone: v3.6.0 (M7)" — metrics persistence belongs to v3.6.0 scope. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 referenced this issue

2026-04-09 13:32:13 +00:00

[AUTO-GROOMER] Backlog Grooming Report (Cycle 1) #5720