cleveragents/cleveragents-core

Fork 3

UAT: `AutomationProfile.check_guard()` does not log guard decisions — guard audit logging requirement unmet #5640

New issue

Open

opened 2026-04-09 08:02:08 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 08:02:08 +00:00

Owner

Summary

The spec requires that all guard decisions be logged for audit purposes. AutomationProfile.check_guard() makes guard decisions but does not emit any log entries. If callers use check_guard() directly without going through AutonomyGuardrailService, guard decisions are completely unlogged.

What Was Tested

Code-level analysis of:

src/cleveragents/domain/models/core/automation_profile.py — check_guard() method (lines 277-360)
src/cleveragents/application/services/autonomy_guardrail_service.py — logging in service methods

Expected Behavior (from spec §"Automation Guard Sub-Model" and UAT test scope)

Guard audit logging: all guard decisions logged

All guard evaluation decisions (allowed or denied) should be logged with sufficient context to reconstruct what happened during plan execution.

Actual Behavior

check_guard() makes guard decisions silently:

def check_guard(self, tool_name, is_write, cost_so_far, calls_so_far, scope):
    if self.guards is None:
        return GuardResult(allowed=True)  # No log
    guards = self.guards
    if guards.tool_denylist and tool_name in guards.tool_denylist:
        return GuardResult(
            allowed=False,
            requires_approval=True,
            reason=f"Tool '{tool_name}' is on the denylist...",
        )  # No log!
    # ... more checks, all without logging

AutonomyGuardrailService does log guard decisions via _record_enforcement_locked(), but:

AutonomyGuardrailService is for AutonomyGuardrails (step limits, wall-clock, etc.), not AutomationGuard (denylist, allowlist, budget)
There is no service that wraps check_guard() and logs its decisions to the audit trail

Impact

Guard decisions from check_guard() are not logged
When check_guard() blocks a tool (denylist, allowlist, budget cap, write approval), there is no audit record
Debugging why a tool was blocked requires code-level investigation rather than log inspection
The guard audit logging requirement from the spec is not met for AutomationGuard decisions

Code Location

src/cleveragents/domain/models/core/automation_profile.py:check_guard() (line 277)

Fix Required

Either:

Add logger.debug()/logger.warning() calls inside check_guard() for each guard decision, OR
Create a AutomationGuardService that wraps check_guard() and records decisions to the GuardrailAuditTrail, similar to how AutonomyGuardrailService works for step/budget limits

Option 2 is preferred as it integrates with the existing audit trail infrastructure.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Summary The spec requires that all guard decisions be logged for audit purposes. `AutomationProfile.check_guard()` makes guard decisions but does not emit any log entries. If callers use `check_guard()` directly without going through `AutonomyGuardrailService`, guard decisions are completely unlogged. ## What Was Tested Code-level analysis of: - `src/cleveragents/domain/models/core/automation_profile.py` — `check_guard()` method (lines 277-360) - `src/cleveragents/application/services/autonomy_guardrail_service.py` — logging in service methods ## Expected Behavior (from spec §"Automation Guard Sub-Model" and UAT test scope) > Guard audit logging: all guard decisions logged All guard evaluation decisions (allowed or denied) should be logged with sufficient context to reconstruct what happened during plan execution. ## Actual Behavior `check_guard()` makes guard decisions silently: ```python def check_guard(self, tool_name, is_write, cost_so_far, calls_so_far, scope): if self.guards is None: return GuardResult(allowed=True) # No log guards = self.guards if guards.tool_denylist and tool_name in guards.tool_denylist: return GuardResult( allowed=False, requires_approval=True, reason=f"Tool '{tool_name}' is on the denylist...", ) # No log! # ... more checks, all without logging ``` `AutonomyGuardrailService` does log guard decisions via `_record_enforcement_locked()`, but: 1. `AutonomyGuardrailService` is for `AutonomyGuardrails` (step limits, wall-clock, etc.), not `AutomationGuard` (denylist, allowlist, budget) 2. There is no service that wraps `check_guard()` and logs its decisions to the audit trail ## Impact - Guard decisions from `check_guard()` are not logged - When `check_guard()` blocks a tool (denylist, allowlist, budget cap, write approval), there is no audit record - Debugging why a tool was blocked requires code-level investigation rather than log inspection - The guard audit logging requirement from the spec is not met for `AutomationGuard` decisions ## Code Location - `src/cleveragents/domain/models/core/automation_profile.py:check_guard()` (line 277) ## Fix Required Either: 1. Add `logger.debug()`/`logger.warning()` calls inside `check_guard()` for each guard decision, OR 2. Create a `AutomationGuardService` that wraps `check_guard()` and records decisions to the `GuardrailAuditTrail`, similar to how `AutonomyGuardrailService` works for step/budget limits Option 2 is preferred as it integrates with the existing audit trail infrastructure. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 08:02:47 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-09 08:05:31 +00:00

HAL9000 commented

2026-04-09 08:07:19 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed

labels

2026-04-09 08:08:56 +00:00

HAL9000 added a new dependency

2026-04-09 08:15:56 +00:00