UAT: InvariantEnforcementRecord.decision_id is a random ULID unlinked to the actual Decision tree node — enforcement records cannot be traced back to decisions #2890

New issue

Open

opened 2026-04-05 02:42:42 +00:00 by freemo · 1 comment

freemo commented

2026-04-05 02:42:42 +00:00

Owner

Metadata

Branch: fix/invariant-enforcement-record-decision-id-traceability
Commit Message: fix(invariants): link InvariantEnforcementRecord.decision_id to actual Decision node
Milestone: v3.4.0
Parent Epic: #394

Background and Context

The InvariantEnforcementRecord.decision_id field is documented as "ULID of the associated decision node." Per the specification, when an invariant is applied during the "Strategize" phase it must be recorded as an invariant_enforced decision in the plan's decision tree, ensuring the constraint is documented and traceable. This traceability is a core requirement of the Decision Framework.

Currently, InvariantService.enforce_invariants() generates a fresh random ULID for each InvariantEnforcementRecord.decision_id rather than using the ULID of the Decision object created by DecisionService.record_decision(). This severs the audit trail between enforcement records and the decision tree.

Current Behavior

In InvariantReconciliationActor._record_enforcement_decisions() (src/cleveragents/actor/reconciliation.py):

Calls self._decision_service.record_decision(...) → creates a Decision with a real ULID (e.g., 01JQABC...)
Appends that ULID to decision_ids (returned in ReconciliationResult.enforced_decision_ids)
Calls self._invariant_service.enforce_invariants(plan_id=..., invariants=..., actor_response=...)

In InvariantService.enforce_invariants() (src/cleveragents/application/services/invariant_service.py), for each invariant:

record = InvariantEnforcementRecord(
    invariant_id=inv.id,
    enforced=enforced,
    actor_response=response,
    decision_id=str(ULID()),  # ← generates a NEW random ULID — not linked to any Decision!
)

Result: InvariantEnforcementRecord.decision_id is a phantom ULID with no corresponding node in the decision tree. ReconciliationResult.enforced_decision_ids holds the real decision ULIDs, but the enforcement records are disconnected from them.

Steps to reproduce:

Create an InvariantReconciliationActor with real InvariantService and DecisionService
Add a global invariant
Call actor.run(plan_id="01JQAAAAAAAAAAAAAAAAAAAAA01")
Inspect result.enforced_decision_ids[0] — this is the real decision ULID
Inspect invariant_service._enforcement_records[0].decision_id — this is a different random ULID
Observe: the two ULIDs do not match; the enforcement record cannot be traced to the decision

Expected Behavior

InvariantEnforcementRecord.decision_id must equal the Decision.decision_id returned by DecisionService.record_decision() for the corresponding enforcement action. Every enforcement record must be traceable to a real node in the plan's decision tree.

Acceptance Criteria

InvariantService.enforce_invariants() accepts an optional decision_ids: list[str] | None = None parameter
When decision_ids is provided, each InvariantEnforcementRecord uses the corresponding ID from the list instead of generating a new ULID
InvariantReconciliationActor._record_enforcement_decisions() passes the real decision IDs to enforce_invariants()
result.enforced_decision_ids[i] matches enforcement_records[i].decision_id for all i
No phantom ULIDs appear in any InvariantEnforcementRecord.decision_id field
All existing tests continue to pass; new regression tests cover the fix

Supporting Information

Affected files:
- src/cleveragents/actor/reconciliation.py — _record_enforcement_decisions() method
- src/cleveragents/application/services/invariant_service.py — enforce_invariants() method
Severity: Medium — breaks audit trail and decision-tree traceability required by the spec
Discovered during: UAT code analysis of the Invariants System

Subtasks

Audit InvariantService.enforce_invariants() signature and all call sites
Add optional decision_ids: list[str] | None = None parameter to enforce_invariants()
Update enforce_invariants() to use provided decision_ids entries instead of str(ULID()) when supplied
Update InvariantReconciliationActor._record_enforcement_decisions() to collect real decision ULIDs and pass them to enforce_invariants()
Verify result.enforced_decision_ids[i] == enforcement_records[i].decision_id for all i
Tests (pytest/Behave): Add regression test asserting enforcement record decision_id matches the real Decision.decision_id
Tests (pytest/Behave): Add scenario covering the case where decision_ids is None (backward-compatible fallback)
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/invariant-enforcement-record-decision-id-traceability` - **Commit Message**: `fix(invariants): link InvariantEnforcementRecord.decision_id to actual Decision node` - **Milestone**: v3.4.0 - **Parent Epic**: #394 ## Background and Context The `InvariantEnforcementRecord.decision_id` field is documented as "ULID of the associated decision node." Per the specification, when an invariant is applied during the "Strategize" phase it must be recorded as an `invariant_enforced` decision in the plan's decision tree, ensuring the constraint is documented and traceable. This traceability is a core requirement of the Decision Framework. Currently, `InvariantService.enforce_invariants()` generates a **fresh random ULID** for each `InvariantEnforcementRecord.decision_id` rather than using the ULID of the `Decision` object created by `DecisionService.record_decision()`. This severs the audit trail between enforcement records and the decision tree. ## Current Behavior In `InvariantReconciliationActor._record_enforcement_decisions()` (`src/cleveragents/actor/reconciliation.py`): 1. Calls `self._decision_service.record_decision(...)` → creates a `Decision` with a real ULID (e.g., `01JQABC...`) 2. Appends that ULID to `decision_ids` (returned in `ReconciliationResult.enforced_decision_ids`) 3. Calls `self._invariant_service.enforce_invariants(plan_id=..., invariants=..., actor_response=...)` In `InvariantService.enforce_invariants()` (`src/cleveragents/application/services/invariant_service.py`), for each invariant: ```python record = InvariantEnforcementRecord( invariant_id=inv.id, enforced=enforced, actor_response=response, decision_id=str(ULID()), # ← generates a NEW random ULID — not linked to any Decision! ) ``` **Result:** `InvariantEnforcementRecord.decision_id` is a phantom ULID with no corresponding node in the decision tree. `ReconciliationResult.enforced_decision_ids` holds the real decision ULIDs, but the enforcement records are disconnected from them. **Steps to reproduce:** 1. Create an `InvariantReconciliationActor` with real `InvariantService` and `DecisionService` 2. Add a global invariant 3. Call `actor.run(plan_id="01JQAAAAAAAAAAAAAAAAAAAAA01")` 4. Inspect `result.enforced_decision_ids[0]` — this is the real decision ULID 5. Inspect `invariant_service._enforcement_records[0].decision_id` — this is a **different** random ULID 6. Observe: the two ULIDs do not match; the enforcement record cannot be traced to the decision ## Expected Behavior `InvariantEnforcementRecord.decision_id` must equal the `Decision.decision_id` returned by `DecisionService.record_decision()` for the corresponding enforcement action. Every enforcement record must be traceable to a real node in the plan's decision tree. ## Acceptance Criteria - `InvariantService.enforce_invariants()` accepts an optional `decision_ids: list[str] | None = None` parameter - When `decision_ids` is provided, each `InvariantEnforcementRecord` uses the corresponding ID from the list instead of generating a new ULID - `InvariantReconciliationActor._record_enforcement_decisions()` passes the real decision IDs to `enforce_invariants()` - `result.enforced_decision_ids[i]` matches `enforcement_records[i].decision_id` for all `i` - No phantom ULIDs appear in any `InvariantEnforcementRecord.decision_id` field - All existing tests continue to pass; new regression tests cover the fix ## Supporting Information - **Affected files:** - `src/cleveragents/actor/reconciliation.py` — `_record_enforcement_decisions()` method - `src/cleveragents/application/services/invariant_service.py` — `enforce_invariants()` method - **Severity:** Medium — breaks audit trail and decision-tree traceability required by the spec - **Discovered during:** UAT code analysis of the Invariants System ## Subtasks - [ ] Audit `InvariantService.enforce_invariants()` signature and all call sites - [ ] Add optional `decision_ids: list[str] | None = None` parameter to `enforce_invariants()` - [ ] Update `enforce_invariants()` to use provided `decision_ids` entries instead of `str(ULID())` when supplied - [ ] Update `InvariantReconciliationActor._record_enforcement_decisions()` to collect real decision ULIDs and pass them to `enforce_invariants()` - [ ] Verify `result.enforced_decision_ids[i] == enforcement_records[i].decision_id` for all `i` - [ ] Tests (pytest/Behave): Add regression test asserting enforcement record `decision_id` matches the real `Decision.decision_id` - [ ] Tests (pytest/Behave): Add scenario covering the case where `decision_ids` is `None` (backward-compatible fallback) - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.4.0 milestone

2026-04-05 02:42:52 +00:00

freemo added a new dependency

2026-04-05 02:43:10 +00:00

#394 Epic: Decision Framework

freemo referenced this issue

2026-04-05 02:44:00 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2875

freemo commented

2026-04-05 02:44:48 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium (confirmed) — Invariant enforcement records using random ULIDs instead of actual Decision IDs breaks traceability but doesn't affect enforcement itself.
MoSCoW: Should Have — Traceability between enforcement records and decisions is important for debugging and auditing.

Valid UAT finding.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) — Invariant enforcement records using random ULIDs instead of actual Decision IDs breaks traceability but doesn't affect enforcement itself. - **MoSCoW**: Should Have — Traceability between enforcement records and decisions is important for debugging and auditing. Valid UAT finding. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo removed this from the v3.4.0 milestone

2026-04-06 21:01:41 +00:00