cleveragents/cleveragents-core

Fork 3

BUG-HUNT: [resource] `Agent._tasks` list grows unbounded — completed tasks never pruned, causing memory leak proportional to message volume #6524

New issue

Open

opened 2026-04-09 21:14:59 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 21:14:59 +00:00

Owner

Bug Report: [resource] — `Agent._tasks` unbounded memory growth

Severity Assessment

Impact: Long-lived agent instances processing many messages will grow their _tasks list indefinitely, consuming increasing memory proportional to the total number of messages ever processed. Under sustained load this causes OOM conditions.
Likelihood: High — every message processed adds one entry, none are removed
Priority: High

Location

File: src/cleveragents/agents/base.py
Class: Agent
Method: _setup_processing_pipeline, __init__
Lines: 16, 27–29

Description

In __init__, the agent initialises _tasks as an empty list:

# base.py line 16
self._tasks: list[asyncio.Task[Any]] = []

In _setup_processing_pipeline, every incoming message creates a new asyncio.Task and appends it to _tasks:

# base.py lines 27-29
def _on_next(msg: Any) -> None:
    task = asyncio.create_task(self._process_wrapper(msg))
    self._tasks.append(task)  # Tasks ADDED but NEVER removed

There is no code anywhere in Agent (including dispose()) that removes completed tasks from this list. Once a task finishes, its reference remains in _tasks keeping it (and its result/exception) alive in memory forever.

Expected Behavior

Completed tasks should be removed from _tasks to free memory. This can be done via a done_callback:

def _on_next(msg: Any) -> None:
    task = asyncio.create_task(self._process_wrapper(msg))
    self._tasks.append(task)
    task.add_done_callback(self._tasks.remove)

Actual Behavior

_tasks grows monotonically. With a high-throughput agent (e.g., reactive stream router), this becomes a significant memory leak.

Suggested Fix

Add a done_callback that removes the task from the list upon completion:

task = asyncio.create_task(self._process_wrapper(msg))
task.add_done_callback(self._tasks.discard if hasattr(self._tasks, 'discard') else lambda t: self._tasks.remove(t) if t in self._tasks else None)
self._tasks.append(task)

Or use a weakref-based approach, or simply clear finished tasks periodically.

TDD Note

After this bug is verified, a Type/Testing issue will be created with a TDD test tagged @tdd_issue, @tdd_issue_<this-issue-number>, and @tdd_expected_fail to prove the bug exists before fixing it.

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: bug-hunter

## Bug Report: [resource] — `Agent._tasks` unbounded memory growth ### Severity Assessment - **Impact**: Long-lived agent instances processing many messages will grow their `_tasks` list indefinitely, consuming increasing memory proportional to the total number of messages ever processed. Under sustained load this causes OOM conditions. - **Likelihood**: High — every message processed adds one entry, none are removed - **Priority**: High ### Location - **File**: `src/cleveragents/agents/base.py` - **Class**: `Agent` - **Method**: `_setup_processing_pipeline`, `__init__` - **Lines**: 16, 27–29 ### Description In `__init__`, the agent initialises `_tasks` as an empty list: ```python # base.py line 16 self._tasks: list[asyncio.Task[Any]] = [] ``` In `_setup_processing_pipeline`, every incoming message creates a new `asyncio.Task` and appends it to `_tasks`: ```python # base.py lines 27-29 def _on_next(msg: Any) -> None: task = asyncio.create_task(self._process_wrapper(msg)) self._tasks.append(task) # Tasks ADDED but NEVER removed ``` There is no code anywhere in `Agent` (including `dispose()`) that removes completed tasks from this list. Once a task finishes, its reference remains in `_tasks` keeping it (and its result/exception) alive in memory forever. ### Expected Behavior Completed tasks should be removed from `_tasks` to free memory. This can be done via a `done_callback`: ```python def _on_next(msg: Any) -> None: task = asyncio.create_task(self._process_wrapper(msg)) self._tasks.append(task) task.add_done_callback(self._tasks.remove) ``` ### Actual Behavior `_tasks` grows monotonically. With a high-throughput agent (e.g., reactive stream router), this becomes a significant memory leak. ### Suggested Fix Add a `done_callback` that removes the task from the list upon completion: ```python task = asyncio.create_task(self._process_wrapper(msg)) task.add_done_callback(self._tasks.discard if hasattr(self._tasks, 'discard') else lambda t: self._tasks.remove(t) if t in self._tasks else None) self._tasks.append(task) ``` Or use a `weakref`-based approach, or simply clear finished tasks periodically. ### Category resource / memory-leak ### TDD Note After this bug is verified, a Type/Testing issue will be created with a TDD test tagged `@tdd_issue`, `@tdd_issue_<this-issue-number>`, and `@tdd_expected_fail` to prove the bug exists before fixing it. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: bug-hunter

HAL9000 added the

labels

2026-04-09 21:26:16 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 21:27:53 +00:00

HAL9000 referenced this issue

2026-04-14 20:11:00 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Supervisor (Cycle 2) #9489

HAL9000 added

and removed

labels

2026-04-17 06:28:08 +00:00

HAL9000 commented

2026-04-17 07:04:20 +00:00

Author

Owner

✅ Verified — Valid memory leak. Completed tasks are never pruned from the _tasks list, causing memory growth proportional to message volume. MoSCoW: Should Have (confirmed) — memory leak in long-running agents.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — Valid memory leak. Completed tasks are never pruned from the _tasks list, causing memory growth proportional to message volume. **MoSCoW: Should Have** (confirmed) — memory leak in long-running agents. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor