cleveragents/cleveragents-core

Fork 3

UAT: Prometheus metrics export endpoint not implemented — `metrics_export_prometheus` setting is a no-op #5821

New issue

Open

opened 2026-04-09 10:11:47 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 10:11:47 +00:00

Owner

Bug Report

Feature Area: Observability — Prometheus Metrics Export
Milestone: v3.6.0 (M7)
Severity: Critical — spec requires Prometheus format export for operational metrics

What Was Tested

The MetricsEmitter and metrics_export_prometheus setting were inspected against the specification's observability requirements.

Expected Behavior (from spec)

The specification states:

The system must export operational metrics in Prometheus format.

The Settings class has a metrics_export_prometheus field:

metrics_export_prometheus: bool = Field(
    default=False,
    description="Enable Prometheus metrics export endpoint.",
)

Actual Behavior

The MetricsEmitter class (src/cleveragents/infrastructure/observability/metrics_emitter.py) explicitly documents that Prometheus export is not implemented:

"""
MetricsEmitter accepts MetricEntry objects and emits them as
structured log entries via structlog. Local-mode emission is the
default; Prometheus export is a future extension gated behind a
configuration flag.
"""

The prometheus_enabled property exists but is never used to actually export metrics:

@property
def prometheus_enabled(self) -> bool:
    """Whether Prometheus export is enabled."""
    return self._prometheus_enabled

There is no:

HTTP endpoint serving /metrics in Prometheus text format
prometheus_client library integration
Gauge/Counter/Histogram registration with Prometheus
Any code path that actually exports metrics when metrics_export_prometheus=True

Setting CLEVERAGENTS_METRICS_EXPORT_PROMETHEUS=true has no effect — metrics are only emitted as structured log entries.

Code Location

src/cleveragents/infrastructure/observability/metrics_emitter.py — emit() method only logs, never exports to Prometheus
src/cleveragents/config/settings.py — metrics_export_prometheus setting (no-op)

Impact

Operations teams cannot:

Scrape CleverAgents metrics with Prometheus
Set up Grafana dashboards for plan execution metrics
Alert on metric thresholds (e.g., high tool error rate, budget exceeded)
Integrate with existing Prometheus/Grafana monitoring stacks

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: Observability — Prometheus Metrics Export **Milestone**: v3.6.0 (M7) **Severity**: Critical — spec requires Prometheus format export for operational metrics ### What Was Tested The `MetricsEmitter` and `metrics_export_prometheus` setting were inspected against the specification's observability requirements. ### Expected Behavior (from spec) The specification states: > The system must export operational metrics in **Prometheus** format. The `Settings` class has a `metrics_export_prometheus` field: ```python metrics_export_prometheus: bool = Field( default=False, description="Enable Prometheus metrics export endpoint.", ) ``` ### Actual Behavior The `MetricsEmitter` class (`src/cleveragents/infrastructure/observability/metrics_emitter.py`) explicitly documents that Prometheus export is **not implemented**: ```python """ MetricsEmitter accepts MetricEntry objects and emits them as structured log entries via structlog. Local-mode emission is the default; Prometheus export is a future extension gated behind a configuration flag. """ ``` The `prometheus_enabled` property exists but is never used to actually export metrics: ```python @property def prometheus_enabled(self) -> bool: """Whether Prometheus export is enabled.""" return self._prometheus_enabled ``` There is no: - HTTP endpoint serving `/metrics` in Prometheus text format - `prometheus_client` library integration - Gauge/Counter/Histogram registration with Prometheus - Any code path that actually exports metrics when `metrics_export_prometheus=True` Setting `CLEVERAGENTS_METRICS_EXPORT_PROMETHEUS=true` has no effect — metrics are only emitted as structured log entries. ### Code Location - `src/cleveragents/infrastructure/observability/metrics_emitter.py` — `emit()` method only logs, never exports to Prometheus - `src/cleveragents/config/settings.py` — `metrics_export_prometheus` setting (no-op) ### Impact Operations teams cannot: - Scrape CleverAgents metrics with Prometheus - Set up Grafana dashboards for plan execution metrics - Alert on metric thresholds (e.g., high tool error rate, budget exceeded) - Integrate with existing Prometheus/Grafana monitoring stacks --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 10:18:59 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-09 10:26:09 +00:00

HAL9000 commented

2026-04-09 10:32:45 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed

labels

2026-04-09 10:36:50 +00:00

HAL9000 referenced this issue

2026-04-09 10:48:15 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 29) #5788

HAL9000 added

and removed

labels

2026-04-09 11:00:37 +00:00

~~HAL9000 referenced this issue 2026-04-09 12:21:23 +00:00~~

[AUTO-UAT-POOL] UAT Testing Report (Cycle 1) #5750

HAL9000 commented

2026-04-09 15:29:56 +00:00

Author

Owner

⚠️ Milestone mismatch detected:

Current milestone: v3.5.0 (Autonomy Hardening)
Issue body states: v3.6.0 (Advanced Concepts & Deferred Features)
Issue content: Prometheus metrics export endpoint — this is an observability feature that belongs to v3.6.0 scope (cost/session budgets, safety profiles, E2E workflow specification tests), not v3.5.0

Recommendation: Please review and update the milestone to v3.6.0 if the body is correct.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

⚠️ **Milestone mismatch detected:** - Current milestone: **v3.5.0** (Autonomy Hardening) - Issue body states: **v3.6.0** (Advanced Concepts & Deferred Features) - Issue content: Prometheus metrics export endpoint — this is an observability feature that belongs to v3.6.0 scope (cost/session budgets, safety profiles, E2E workflow specification tests), not v3.5.0 **Recommendation**: Please review and update the milestone to v3.6.0 if the body is correct. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 referenced this issue

2026-04-09 20:05:39 +00:00

[AUTO-IMP-POOL] Implementation Pool Tracking (Cycle 1) #6296