cleveragents/cleveragents-core

Fork 3

UAT: Audit trail does not log plan lifecycle events — `plan_created`, `plan_executed`, `tool_invoked`, `decision_point` events missing #5813

New issue

Open

opened 2026-04-09 10:08:09 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 10:08:09 +00:00

Owner

Bug Report

Feature Area: Audit Logging — Event Coverage
Milestone: v3.5.0 (M6)
Severity: Critical — core audit trail events are not captured

What Was Tested

The AuditEventSubscriber and VALID_EVENT_TYPES were inspected against the specification's list of required auditable events.

Expected Behavior (from spec)

The specification states:

An immutable, cryptographically verifiable audit trail must log every significant event.
Events to be logged include: plan creation/execution/application, decision points, tool invocations, sandbox changes, user approvals, and configuration changes.

Actual Behavior

The VALID_EVENT_TYPES in src/cleveragents/application/services/audit_service.py only includes:

VALID_EVENT_TYPES: frozenset[str] = frozenset({
    "plan_applied",
    "plan_cancelled",
    "resource_modified",
    "correction_applied",
    "config_changed",
    "entity_deleted",
    "session_created",
    "auth_success",
    "auth_failure",
})

Missing spec-required event types:

plan_created / plan.created — Plan creation is not audited
plan_executed / plan.executed — Plan execution start/completion is not audited
tool_invoked / tool.invoked — Tool invocations are not audited (the AuditEventSubscriber comment notes: "RESOURCE_MODIFIED has no producing service yet — the tool execution framework does not emit this event")
decision_point — Decision tree nodes are not audited
user_approval — User approval events are not audited
sandbox_changed — Sandbox mutations are not audited

The AuditEventSubscriber (src/cleveragents/application/services/audit_event_subscriber.py) also explicitly notes that RESOURCE_MODIFIED, AUTH_SUCCESS, and AUTH_FAILURE have no producing services yet.

Code Location

src/cleveragents/application/services/audit_service.py — VALID_EVENT_TYPES frozenset
src/cleveragents/application/services/audit_event_subscriber.py — SECURITY_EVENT_MAP
src/cleveragents/infrastructure/events/types.py — EventType enum (missing event types)

Impact

The audit log is incomplete for compliance purposes. Security auditors cannot:

Track when plans were created (only when applied/cancelled)
Track tool invocations during plan execution
Track decision points in the decision tree
Track user approval actions
Track sandbox mutations

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: Audit Logging — Event Coverage **Milestone**: v3.5.0 (M6) **Severity**: Critical — core audit trail events are not captured ### What Was Tested The `AuditEventSubscriber` and `VALID_EVENT_TYPES` were inspected against the specification's list of required auditable events. ### Expected Behavior (from spec) The specification states: > An immutable, cryptographically verifiable audit trail must log every significant event. > Events to be logged include: **plan creation/execution/application**, **decision points**, **tool invocations**, **sandbox changes**, **user approvals**, and **configuration changes**. ### Actual Behavior The `VALID_EVENT_TYPES` in `src/cleveragents/application/services/audit_service.py` only includes: ```python VALID_EVENT_TYPES: frozenset[str] = frozenset({ "plan_applied", "plan_cancelled", "resource_modified", "correction_applied", "config_changed", "entity_deleted", "session_created", "auth_success", "auth_failure", }) ``` **Missing spec-required event types:** 1. `plan_created` / `plan.created` — Plan creation is not audited 2. `plan_executed` / `plan.executed` — Plan execution start/completion is not audited 3. `tool_invoked` / `tool.invoked` — Tool invocations are not audited (the `AuditEventSubscriber` comment notes: "RESOURCE_MODIFIED has no producing service yet — the tool execution framework does not emit this event") 4. `decision_point` — Decision tree nodes are not audited 5. `user_approval` — User approval events are not audited 6. `sandbox_changed` — Sandbox mutations are not audited The `AuditEventSubscriber` (`src/cleveragents/application/services/audit_event_subscriber.py`) also explicitly notes that `RESOURCE_MODIFIED`, `AUTH_SUCCESS`, and `AUTH_FAILURE` have no producing services yet. ### Code Location - `src/cleveragents/application/services/audit_service.py` — `VALID_EVENT_TYPES` frozenset - `src/cleveragents/application/services/audit_event_subscriber.py` — `SECURITY_EVENT_MAP` - `src/cleveragents/infrastructure/events/types.py` — `EventType` enum (missing event types) ### Impact The audit log is incomplete for compliance purposes. Security auditors cannot: - Track when plans were created (only when applied/cancelled) - Track tool invocations during plan execution - Track decision points in the decision tree - Track user approval actions - Track sandbox mutations --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 10:18:59 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-09 10:26:10 +00:00

HAL9000 commented

2026-04-09 10:32:46 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed

labels

2026-04-09 10:36:52 +00:00

HAL9000 referenced this issue

2026-04-09 10:48:15 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 29) #5788

HAL9000 added