[events/ReactiveEventBus] ReactiveEventBus._audit_log grows unboundedly with default max_audit_log_size=None — memory exhaustion in long-running processes #10322

New issue

Open

opened 2026-04-18 08:43:01 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-18 08:43:01 +00:00

Owner

Metadata

Commit: fix(events): bound ReactiveEventBus audit log by default to prevent memory exhaustion
Branch: fix/reactive-event-bus-unbounded-audit-log

Background and Context

ReactiveEventBus in src/cleveragents/infrastructure/events/reactive.py defaults to max_audit_log_size=None, creating an unbounded deque. Every emit() call appends a deep copy of the event to this deque. In long-running processes (the DI container registers ReactiveEventBus as a Singleton), this causes unbounded memory growth that will eventually exhaust available memory.

Expected Behavior

Either:

A sensible default max_audit_log_size (e.g., 10,000) should be set to bound memory usage, OR
A warning should be logged when max_audit_log_size=None to alert operators of the unbounded configuration

The docstring should clearly document the memory implications of the default configuration.

Acceptance Criteria

Default max_audit_log_size is set to a bounded value (e.g., 10,000), OR a warning is logged for unbounded configuration
Docstring clearly documents memory implications
TDD test (see blocked-by issue #10321) passes after fix
nox passes with coverage ≥ 97%

Subtasks

Set a sensible default for max_audit_log_size (e.g., 10,000) OR add warning for unbounded config
Update docstring to document memory implications
Add/update tests to verify bounded behavior with default config
Run nox to confirm coverage ≥ 97%

Definition of Done

This issue is closed when:

The default configuration either bounds the audit log or warns about unbounded growth
The TDD test (blocked-by issue #10321) passes
nox passes with coverage ≥ 97%
A PR is reviewed and merged to main

Summary

Current Behaviour

# src/cleveragents/infrastructure/events/reactive.py
class ReactiveEventBus:
    def __init__(self, max_audit_log_size: int | None = None) -> None:
        # max_audit_log_size=None → deque with NO size limit
        self._audit_log: deque[DomainEvent] = deque(maxlen=max_audit_log_size)
    
    def emit(self, event: DomainEvent) -> None:
        ...
        # Deep copy appended on EVERY emit — grows forever with default config
        self._audit_log.append(event.model_copy(deep=True))

With the default configuration (max_audit_log_size=None):

deque(maxlen=None) has no maximum size
Every emit() appends a deep copy of the event (including the details: dict[str, Any] payload)
The clear_audit_log() method exists but must be called manually
No warning is emitted when the unbounded configuration is used

Impact

In a production system:

The DI container registers ReactiveEventBus as a Singleton (lives for the entire process lifetime)
A system processing 1,000 events/hour with average event size of 1KB would accumulate ~24MB/day
A system processing 10,000 events/hour would accumulate ~240MB/day
Events with large details payloads (e.g., CONTEXT_BUILT with full context data) could be much larger
Over days or weeks, this will exhaust available memory and crash the process

Fix

Option A (recommended): Set a sensible default:

def __init__(self, max_audit_log_size: int | None = 10_000) -> None:

Option B: Warn on unbounded configuration:

if max_audit_log_size is None:
    _logger.warning(
        "reactive_event_bus_unbounded_audit_log",
        message="max_audit_log_size is None — audit log will grow unboundedly.",
    )

Option C (best): Both A and B.

References

src/cleveragents/infrastructure/events/reactive.py — ReactiveEventBus.__init__() (default max_audit_log_size=None) and emit() (deep copy append)

Automated by CleverAgents Bot
Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor

## Metadata - **Commit**: `fix(events): bound ReactiveEventBus audit log by default to prevent memory exhaustion` - **Branch**: `fix/reactive-event-bus-unbounded-audit-log` ## Background and Context `ReactiveEventBus` in `src/cleveragents/infrastructure/events/reactive.py` defaults to `max_audit_log_size=None`, creating an unbounded `deque`. Every `emit()` call appends a **deep copy** of the event to this deque. In long-running processes (the DI container registers `ReactiveEventBus` as a Singleton), this causes unbounded memory growth that will eventually exhaust available memory. ## Expected Behavior Either: 1. A sensible default `max_audit_log_size` (e.g., 10,000) should be set to bound memory usage, OR 2. A warning should be logged when `max_audit_log_size=None` to alert operators of the unbounded configuration The docstring should clearly document the memory implications of the default configuration. ## Acceptance Criteria - [ ] Default `max_audit_log_size` is set to a bounded value (e.g., 10,000), OR a warning is logged for unbounded configuration - [ ] Docstring clearly documents memory implications - [ ] TDD test (see blocked-by issue #10321) passes after fix - [ ] `nox` passes with coverage ≥ 97% ## Subtasks - [ ] Set a sensible default for `max_audit_log_size` (e.g., 10,000) OR add warning for unbounded config - [ ] Update docstring to document memory implications - [ ] Add/update tests to verify bounded behavior with default config - [ ] Run `nox` to confirm coverage ≥ 97% ## Definition of Done This issue is closed when: 1. The default configuration either bounds the audit log or warns about unbounded growth 2. The TDD test (blocked-by issue #10321) passes 3. `nox` passes with coverage ≥ 97% 4. A PR is reviewed and merged to `main` --- ## Summary `ReactiveEventBus` in `src/cleveragents/infrastructure/events/reactive.py` defaults to `max_audit_log_size=None`, creating an unbounded `deque`. Every `emit()` call appends a **deep copy** of the event to this deque. In long-running processes (the DI container registers `ReactiveEventBus` as a Singleton), this causes unbounded memory growth that will eventually exhaust available memory. ## Current Behaviour ```python # src/cleveragents/infrastructure/events/reactive.py class ReactiveEventBus: def __init__(self, max_audit_log_size: int | None = None) -> None: # max_audit_log_size=None → deque with NO size limit self._audit_log: deque[DomainEvent] = deque(maxlen=max_audit_log_size) def emit(self, event: DomainEvent) -> None: ... # Deep copy appended on EVERY emit — grows forever with default config self._audit_log.append(event.model_copy(deep=True)) ``` With the default configuration (`max_audit_log_size=None`): - `deque(maxlen=None)` has no maximum size - Every `emit()` appends a deep copy of the event (including the `details: dict[str, Any]` payload) - The `clear_audit_log()` method exists but must be called manually - No warning is emitted when the unbounded configuration is used ## Impact In a production system: - The DI container registers `ReactiveEventBus` as a Singleton (lives for the entire process lifetime) - A system processing 1,000 events/hour with average event size of 1KB would accumulate ~24MB/day - A system processing 10,000 events/hour would accumulate ~240MB/day - Events with large `details` payloads (e.g., `CONTEXT_BUILT` with full context data) could be much larger - Over days or weeks, this will exhaust available memory and crash the process ## Fix **Option A** (recommended): Set a sensible default: ```python def __init__(self, max_audit_log_size: int | None = 10_000) -> None: ``` **Option B**: Warn on unbounded configuration: ```python if max_audit_log_size is None: _logger.warning( "reactive_event_bus_unbounded_audit_log", message="max_audit_log_size is None — audit log will grow unboundedly.", ) ``` **Option C** (best): Both A and B. ## References - `src/cleveragents/infrastructure/events/reactive.py` — `ReactiveEventBus.__init__()` (default `max_audit_log_size=None`) and `emit()` (deep copy append) --- **Automated by CleverAgents Bot** Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor

HAL9000 added the

labels

2026-04-18 08:44:56 +00:00