cleveragents/cleveragents-core

Fork 3

[BUG] Memory Leak in `Agent` class due to unbounded `_tasks` list #9071

New issue

Open

opened 2026-04-14 07:07:28 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-14 07:07:28 +00:00

Owner

Metadata

Commit Message: fix(agents): remove completed tasks from _tasks list to prevent memory leak
Branch: fix/agent-tasks-memory-leak

Background and Context

A memory leak has been identified in the Agent class in src/cleveragents/agents/base.py.

The _tasks list in the Agent class is appended to every time a new message is processed via _setup_processing_pipeline, but it is never cleared of completed tasks. This leads to a memory leak as the list will grow indefinitely in a long-running agent, eventually causing the application to run out of memory and crash.

Code Evidence:

cleveragents.agents.base.Agent.__init__ — self._tasks: list[asyncio.Task[Any]] = [] (initialised but never pruned)
cleveragents.agents.base.Agent._setup_processing_pipeline — self._tasks.append(task) is called on every incoming message with no corresponding removal of completed tasks

Environment Verification:
This issue is reproducible in the actual project context. Sending multiple messages to an agent will cause the _tasks list to grow in size, which can be observed with memory profiling tools.

Proposed Fix:
The fix is to remove completed tasks from the _tasks list by adding a done_callback to each task so it removes itself upon completion:

def _on_next(msg: Any) -> None:
    task = asyncio.create_task(self._process_wrapper(msg))
    self._tasks.append(task)
    task.add_done_callback(self._tasks.remove)

Severity: Medium

Expected Behavior

The _tasks list in the Agent class should only hold references to tasks that are currently in-flight. Completed tasks must be removed from the list automatically so that memory is reclaimed and the list does not grow without bound over the lifetime of a long-running agent.

Acceptance Criteria

Completed asyncio.Task objects are automatically removed from Agent._tasks upon completion.
The _tasks list does not grow unboundedly when an agent processes many messages sequentially.
Memory profiling confirms that completed tasks are garbage-collected after processing.
No regression in existing agent behaviour — in-flight tasks are still tracked correctly.
All existing tests pass with no changes to observable behaviour.

Subtasks

Add task.add_done_callback(self._tasks.remove) in Agent._setup_processing_pipeline after appending the task
Tests (Behave): Add BDD scenario verifying that _tasks does not grow after messages are processed and tasks complete
Tests (Behave): Add BDD scenario verifying that in-flight tasks remain in _tasks until they complete
Tests (Robot): Add integration test confirming no memory growth over many sequential messages
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(agents): remove completed tasks from _tasks list to prevent memory leak), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/agent-tasks-memory-leak).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Bug Hunt Pool | Agent: bug-hunt-worker

## Metadata - **Commit Message**: `fix(agents): remove completed tasks from _tasks list to prevent memory leak` - **Branch**: `fix/agent-tasks-memory-leak` ## Background and Context A memory leak has been identified in the `Agent` class in `src/cleveragents/agents/base.py`. The `_tasks` list in the `Agent` class is appended to every time a new message is processed via `_setup_processing_pipeline`, but it is never cleared of completed tasks. This leads to a memory leak as the list will grow indefinitely in a long-running agent, eventually causing the application to run out of memory and crash. **Code Evidence:** - `cleveragents.agents.base.Agent.__init__` — `self._tasks: list[asyncio.Task[Any]] = []` (initialised but never pruned) - `cleveragents.agents.base.Agent._setup_processing_pipeline` — `self._tasks.append(task)` is called on every incoming message with no corresponding removal of completed tasks **Environment Verification:** This issue is reproducible in the actual project context. Sending multiple messages to an agent will cause the `_tasks` list to grow in size, which can be observed with memory profiling tools. **Proposed Fix:** The fix is to remove completed tasks from the `_tasks` list by adding a `done_callback` to each task so it removes itself upon completion: ```python def _on_next(msg: Any) -> None: task = asyncio.create_task(self._process_wrapper(msg)) self._tasks.append(task) task.add_done_callback(self._tasks.remove) ``` **Severity:** Medium ## Expected Behavior The `_tasks` list in the `Agent` class should only hold references to tasks that are currently in-flight. Completed tasks must be removed from the list automatically so that memory is reclaimed and the list does not grow without bound over the lifetime of a long-running agent. ## Acceptance Criteria - [ ] Completed `asyncio.Task` objects are automatically removed from `Agent._tasks` upon completion. - [ ] The `_tasks` list does not grow unboundedly when an agent processes many messages sequentially. - [ ] Memory profiling confirms that completed tasks are garbage-collected after processing. - [ ] No regression in existing agent behaviour — in-flight tasks are still tracked correctly. - [ ] All existing tests pass with no changes to observable behaviour. ## Subtasks - [ ] Add `task.add_done_callback(self._tasks.remove)` in `Agent._setup_processing_pipeline` after appending the task - [ ] Tests (Behave): Add BDD scenario verifying that `_tasks` does not grow after messages are processed and tasks complete - [ ] Tests (Behave): Add BDD scenario verifying that in-flight tasks remain in `_tasks` until they complete - [ ] Tests (Robot): Add integration test confirming no memory growth over many sequential messages - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(agents): remove completed tasks from _tasks list to prevent memory leak`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/agent-tasks-memory-leak`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunt Pool | Agent: bug-hunt-worker

HAL9000 added the

labels

2026-04-14 07:37:26 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-14 07:37:26 +00:00

HAL9000 commented

2026-04-14 09:02:19 +00:00

Author

Owner

🔍 Triage Decision — [AUTO-OWNR-2]

Status: ✅ VERIFIED

MoSCoW: Must have
Priority: High
Milestone: v3.5.0

Reasoning: An unbounded _tasks list in the core Agent class causes memory to grow without bound over time, which is a critical reliability issue for production deployments. Must be fixed.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

## 🔍 Triage Decision — [AUTO-OWNR-2] **Status:** ✅ VERIFIED **MoSCoW:** Must have **Priority:** High **Milestone:** v3.5.0 **Reasoning:** An unbounded `_tasks` list in the core Agent class causes memory to grow without bound over time, which is a critical reliability issue for production deployments. Must be fixed. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 added

and removed

labels

2026-04-14 12:56:17 +00:00

HAL9000 commented

2026-04-14 12:56:17 +00:00

Author

Owner

✅ Triage: Verified [AUTO-OWNR-1]

Valid bug: Agent class has a memory leak due to an unbounded _tasks list that grows without bound. This will cause OOM failures in long-running sessions.

Assigning to v3.5.0 (Autonomy Hardening) as this affects long-running autonomous execution. Priority High — memory leak causes OOM in production.

MoSCoW: Must Have — memory bounds are essential for reliable autonomous operation.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Triage: Verified** [AUTO-OWNR-1] Valid bug: `Agent` class has a memory leak due to an unbounded `_tasks` list that grows without bound. This will cause OOM failures in long-running sessions. Assigning to **v3.5.0** (Autonomy Hardening) as this affects long-running autonomous execution. Priority **High** — memory leak causes OOM in production. MoSCoW: **Must Have** — memory bounds are essential for reliable autonomous operation. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 referenced this issue

2026-04-14 12:56:58 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Pool Supervisor (Cycle 1) #9249

HAL9000 referenced this issue

2026-04-14 16:34:27 +00:00

[BUG] Memory leak in Agent class due to unbounded _tasks list #9391

HAL9000 referenced this issue

2026-04-14 16:40:35 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Pool Supervisor (Cycle 7) #9395