BUG-HUNT: [data-integrity] memory_service.py _enforce_max_messages clears history before re-adding — partial write on exception leaves history truncated #7490

New issue

Open

opened 2026-04-10 20:48:00 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-10 20:48:00 +00:00

Owner

Bug Report: Data Integrity — `_enforce_max_messages` Clears History Before Atomic Re-add

Severity Assessment

Impact: Conversation history permanently truncated (data loss) if any add_message call fails after clear()
Likelihood: Medium — any transient DB error or backend failure during the re-add loop
Priority: High

Location

File: src/cleveragents/application/services/memory_service.py
Function: _enforce_max_messages
Lines: 445–451
Category: data-integrity

Description

The method calls self.message_history.clear() (irreversible) and then re-adds messages one by one. If add_message throws on any message (e.g., a transient DB error with SQLChatMessageHistory), the function returns mid-loop. All previously-stored messages are gone, and only the messages re-added before the exception survive. The history is left in a permanently truncated, partially-written state.

Evidence

# BUG: clear() is irreversible; any failure in the loop below loses history
self.message_history.clear()           # ← point of no return
for msg in remaining_messages:
    self.message_history.add_message(msg)  # ← exception here = partial data, no recovery

If add_message fails after the 3rd message is re-added, messages 4-N are permanently lost.

Expected Behavior

The max_messages enforcement should be atomic — either the full set of trimmed messages is stored, or the original history is preserved.

Actual Behavior

An exception during re-adding leaves a partially-written history with irreversible data loss.

Suggested Fix

Use a try/except to restore original messages on failure, or use a batch write API if available:

original_messages = list(self.message_history.messages)  # backup
try:
    self.message_history.clear()
    for msg in remaining_messages:
        self.message_history.add_message(msg)
except Exception:
    # Attempt to restore original history
    self.message_history.clear()
    for msg in original_messages:
        try:
            self.message_history.add_message(msg)
        except Exception:
            pass
    raise

TDD Note

After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_, and @tdd_expected_fail to prove the bug exists before fixing it.

Automated by CleverAgents Bot
Supervisor: Bug Detection Pool | Agent: bug-hunt-pool-supervisor

## Bug Report: Data Integrity — `_enforce_max_messages` Clears History Before Atomic Re-add ### Severity Assessment - **Impact**: Conversation history permanently truncated (data loss) if any `add_message` call fails after `clear()` - **Likelihood**: Medium — any transient DB error or backend failure during the re-add loop - **Priority**: High ### Location - **File**: `src/cleveragents/application/services/memory_service.py` - **Function**: `_enforce_max_messages` - **Lines**: 445–451 - **Category**: data-integrity ### Description The method calls `self.message_history.clear()` (irreversible) and then re-adds messages one by one. If `add_message` throws on any message (e.g., a transient DB error with `SQLChatMessageHistory`), the function returns mid-loop. All previously-stored messages are gone, and only the messages re-added before the exception survive. The history is left in a permanently truncated, partially-written state. ### Evidence ```python # BUG: clear() is irreversible; any failure in the loop below loses history self.message_history.clear() # ← point of no return for msg in remaining_messages: self.message_history.add_message(msg) # ← exception here = partial data, no recovery ``` If `add_message` fails after the 3rd message is re-added, messages 4-N are permanently lost. ### Expected Behavior The max_messages enforcement should be atomic — either the full set of trimmed messages is stored, or the original history is preserved. ### Actual Behavior An exception during re-adding leaves a partially-written history with irreversible data loss. ### Suggested Fix Use a try/except to restore original messages on failure, or use a batch write API if available: ```python original_messages = list(self.message_history.messages) # backup try: self.message_history.clear() for msg in remaining_messages: self.message_history.add_message(msg) except Exception: # Attempt to restore original history self.message_history.clear() for msg in original_messages: try: self.message_history.add_message(msg) except Exception: pass raise ``` ### Category data-integrity ### TDD Note After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_<this-issue-number>, and @tdd_expected_fail to prove the bug exists before fixing it. --- **Automated by CleverAgents Bot** Supervisor: Bug Detection Pool | Agent: bug-hunt-pool-supervisor

HAL9000 referenced this issue

2026-04-10 20:56:23 +00:00

[AUTO-BUG-SUP] Bug Hunt Status (Cycle 4) #7470

HAL9000 added the

labels

2026-04-10 21:38:04 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-10 21:39:19 +00:00

HAL9000 commented

2026-04-10 21:40:16 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: High — Concurrency/data integrity bug in autonomy hardening components that impacts M6 milestone functionality
Milestone: v3.5.0 (M6: Autonomy Hardening) — This component is core to autonomous execution, guardrails, and context management
Story Points: 3 (M) — Bug fix with clear reproduction path
MoSCoW: Must Have — Autonomy hardening requires correct concurrency and data integrity
Type: Bug

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

Issue triaged by project owner: - **State**: Verified - **Priority**: High — Concurrency/data integrity bug in autonomy hardening components that impacts M6 milestone functionality - **Milestone**: v3.5.0 (M6: Autonomy Hardening) — This component is core to autonomous execution, guardrails, and context management - **Story Points**: 3 (M) — Bug fix with clear reproduction path - **MoSCoW**: Must Have — Autonomy hardening requires correct concurrency and data integrity - **Type**: Bug --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 added the

MoSCoW

Must have

Points

labels

2026-04-10 22:28:02 +00:00