feat(automation): add autonomy guardrails and audit trail #443
No reviewers
Labels
No labels
auto/needs-reevaluation
controller-managed
auto/blocked-by-deps
auto/ci-timeout
auto/claimed-implementer
auto/claimed-merge
auto/claimed-reviewer
auto/driver-down
auto/invariant-violation
auto/last-attempt-tier-0
auto/last-attempt-tier-1
auto/last-attempt-tier-2
auto/last-attempt-tier-min
Automation Tracking
auto/needs-conflict-resolution
auto/needs-implementer
auto/postmortem
auto/ready-to-merge
auto/restart-throttled
auto/revert
auto/sentinel
auto/stale-inactivity
auto/unstable
Blocked
Bounty
$100
Bounty
$1000
Bounty
$10000
Bounty
$20
Bounty
$2000
Bounty
$250
Bounty
$50
Bounty
$500
Bounty
$5000
Bounty
$750
MoSCoW
Could have
MoSCoW
Must have
MoSCoW
Should have
Needs Feedback
Points
1
Points
13
Points
2
Points
21
Points
3
Points
34
Points
5
Points
55
Points
8
Points
88
Priority
Backlog
Priority
CI Blocker
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Signed-off: Owner
Signed-off: Scrum Master
Signed-off: Tech Lead
Spike
State
Completed
State
Duplicate
State
In Progress
State
In Review
State
Paused
State
Unverified
State
Verified
State
Wont Do
Type
Automation
Type
Bug
Type
Discussion
Type
Documentation
Type
Epic
Type
Feature
Type
Legendary
Type
Refactor
Type
Support
Type
Task
Type
Testing
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
cleveragents/cleveragents-core!443
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feature/m6-autonomy-guards"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Add runtime autonomy constraints (max steps, tool budget, required confirmations) and a structured audit trail for plan execution, providing guardrail enforcement and a complete record of autonomy-related decisions.
Changes
New Domain Models (
src/cleveragents/domain/models/core/autonomy_guardrails.py)max_steps,tool_budget,required_confirmations,step_count,budget_spentfields, pluscheck_step_limit(),check_tool_budget(),check_confirmation_required(),increment_step(), andrecord_cost()methodsadd_entry(),denied_count, andallowed_countpropertiesNew Service (
src/cleveragents/application/services/autonomy_guardrail_service.py)Tests
features/autonomy_guardrails.featurecovering model validation, step/budget/confirmation checks, audit trail recording, and service operationsrobot/autonomy_guardrails.robotfor CLI flag smoke testingbenchmarks/autonomy_guardrails_bench.pymeasuring enforcement overheadDocumentation
docs/reference/automation_profiles.mdwith autonomy guardrails section, guardrail fields, enforcement behavior, audit trail schema, event types, and metadata persistenceNox Results
ISSUES CLOSED: #204
Code Review & Fixes Applied — Autonomy Guardrails
A thorough code review of commit
8a6f07d0identified 12 findings across bug, security, performance, spec-alignment, architecture, and test-quality categories. All 12 fixes have been applied:Bug Fixes
budget_spentin audit trail — Cost now recorded before audit entry so the trail reflects the post-deduction budget state.STEP_CHECK,BUDGET_CHECK,CONFIRMATION_CHECK(never emitted anywhere).Security & Performance
GuardrailAuditTrailnow has a configurablemax_entries(default 10,000) with oldest-first eviction.allowed_count/denied_count— Now maintained as incremental private counters, updated inadd_entry().load_from_metadata— Size guards (_MAX_METADATA_ENTRIES=50,000,_MAX_CONFIRMATIONS=500) prevent memory exhaustion from oversized payloads.AutonomyGuardrailServicenow protects all state mutations with athreading.RLock.Spec Alignment (lines 27863-27894)
max_wall_clock_seconds,start_time,mark_started(),check_wall_clock()and service methodcheck_wall_clock().ActorLimitsmodel withmax_tool_calls_per_invocation,max_retries_per_failure, andcheck_actor_tool_calls().Architecture
AutonomyGuardrailServicenow registered asSingletonincontainer.py.Test Quality
get_guardrails()instead of accessing_guardrailsdict directly.tool_budget=0.0, malformed/oversized metadata, case-insensitive confirmations, metadata round-trip fidelity, wall-clock checks, actor limits, audit trail eviction.Verification
nox -e lint— passednox -e typecheck— 0 errors, 0 warningsnox -e dead_code— passedtime_blocked,time_allowed,actor_limit_blocked,actor_limit_allowed8a6f07d0e9830d0364caI asked Chat-GPT to do a review, but it was garbage. Trying again.
Review Summary (commit 830d0364cad4f778aa78084)
Reviewed the single commit on this PR. The guardrail models and service are solidly structured, but the enforcement is not wired into runtime plan execution yet, so the feature as described isn’t actually active.
CI status isn’t visible via the API on my side. Please confirm required checks per
docs/development/ci-cd.mdare green (lint, typecheck, security, quality, unit_tests, integration_tests, coverage, build, docker).Findings
P1:must-fix — The guardrails are never invoked by plan execution or CLI flows. The new
AutonomyGuardrailServiceis only used in tests/benchmarks, and no call sites exist in plan executor/lifecycle. As a result, “runtime autonomy constraints” aren’t enforced. Please wire the service into execution (or update the PR scope to clarify it is scaffolding only).src/cleveragents/application/services/autonomy_guardrail_service.pysrc/cleveragents/application/container.pyP2:should-fix —
max_retries_per_failureis defined but never enforced. There’s no check method and the service doesn’t evaluate it, yet it is documented and exposed in the model. Either implement enforcement or remove the field/doc to avoid dead config.src/cleveragents/domain/models/core/autonomy_guardrails.pydocs/reference/automation_profiles.mdP2:should-fix —
start_timeis stored as an unvalidated string.check_wall_clock()callsdatetime.fromisoformat()directly, which will raise on malformed metadata. Consider validating/normalizing in the model (e.g., store as datetime) or guarding parsing errors incheck_wall_clock()/load_from_metadata().src/cleveragents/domain/models/core/autonomy_guardrails.pyPositive Notes
RLockis appropriate for shared service usage.830d0364ca322e75b430Approved.
If you want to make things even better, write a test that verifies that
max_retries_per_failureactually stops when you have too many retries.The only test is:
But you don't need to do that if you don't want to.
322e75b43053455275baNew commits pushed, approval review dismissed automatically according to repository settings