Task Cleanup Memory Leak in Agent Base Class #10566

New issue

Open

opened 2026-04-18 17:19:15 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-18 17:19:15 +00:00

Owner

Metadata

Commit Message: fix: implement task cleanup mechanism in Agent base class to prevent memory leak

Branch Name: fix/agent-task-cleanup-memory-leak

Background and Context

The Agent base class in src/cleveragents/agents/base.py creates asyncio.Task objects and appends them to the _tasks list (lines 27-28) but never cleans them up. This causes a memory leak when agents process many messages, as completed tasks remain in memory indefinitely. Additionally, the dispose() method (lines 79-82) does not cancel pending tasks, leaving resources in an inconsistent state.

This issue affects all agents that inherit from the Agent base class and is particularly problematic in long-running production environments where agents may process thousands or millions of messages.

Expected Behavior

Completed tasks should be automatically removed from the _tasks list
The dispose() method should cancel all pending tasks before cleanup
Memory usage should remain bounded regardless of the number of messages processed
No memory leaks should occur even after processing 10,000+ messages
Task cleanup should be transparent to subclasses and not require manual intervention

Acceptance Criteria

Completed tasks are removed from _tasks list automatically
dispose() method cancels all pending tasks before cleanup
Memory usage remains stable when processing 10,000+ messages
No tasks remain in _tasks after agent disposal
Existing agent functionality is not affected by cleanup changes
Unit tests verify task cleanup behavior
Integration tests confirm no memory leaks in long-running scenarios

Code Evidence

Issue Location: src/cleveragents/agents/base.py

Lines 27-28: Task creation and appending without cleanup

task = asyncio.create_task(self._process_wrapper(msg))
self._tasks.append(task)

Lines 79-82: dispose() method doesn't cancel pending tasks

def dispose(self):
    # Current implementation doesn't handle task cleanup

Problem: Tasks are appended to _tasks list but never removed
Impact: Long-running agents accumulate tasks in memory indefinitely

Reproduction Steps

from cleveragents.agents.base import Agent

class TestAgent(Agent):
    async def process_message(self, message, context=None):
        return f"Processed: {message}"
    
    def get_capabilities(self):
        return ["test"]

agent = TestAgent("test-agent")

# Send 10000 messages
for i in range(10000):
    agent.send_message(f"message-{i}")

# Check memory usage - _tasks list has 10000 entries
print(len(agent._tasks))  # Output: 10000
# Memory is not freed even after tasks complete

Impact

Severity: HIGH
Scope: All agents inheriting from Agent base class
Risk: Memory leak in long-running agents
Consequences:
- Unbounded growth of _tasks list
- Out-of-memory errors in production environments
- Degraded performance over time
- Potential system crashes in resource-constrained environments

Subtasks

Implement task completion tracking mechanism
Add periodic cleanup of completed tasks from _tasks list
Implement on-completion cleanup callback for tasks
Update dispose() method to cancel pending tasks
Consider using bounded queue instead of unbounded list
Add unit tests for task cleanup behavior
Add integration tests for long-running agent scenarios
Update documentation for task lifecycle management
Performance test with 10,000+ message processing

Definition of Done

All subtasks are completed
Unit test coverage >= 97% for modified code
Integration tests pass with no memory leaks detected
Code review approved by at least one maintainer
No regressions in existing agent functionality
Documentation updated to reflect task cleanup behavior
Changes merged to main branch

Automated by CleverAgents Bot
Agent: new-issue-creator

## Metadata **Commit Message:** fix: implement task cleanup mechanism in Agent base class to prevent memory leak **Branch Name:** fix/agent-task-cleanup-memory-leak --- ## Background and Context The Agent base class in `src/cleveragents/agents/base.py` creates asyncio.Task objects and appends them to the `_tasks` list (lines 27-28) but never cleans them up. This causes a memory leak when agents process many messages, as completed tasks remain in memory indefinitely. Additionally, the `dispose()` method (lines 79-82) does not cancel pending tasks, leaving resources in an inconsistent state. This issue affects all agents that inherit from the Agent base class and is particularly problematic in long-running production environments where agents may process thousands or millions of messages. --- ## Expected Behavior 1. Completed tasks should be automatically removed from the `_tasks` list 2. The `dispose()` method should cancel all pending tasks before cleanup 3. Memory usage should remain bounded regardless of the number of messages processed 4. No memory leaks should occur even after processing 10,000+ messages 5. Task cleanup should be transparent to subclasses and not require manual intervention --- ## Acceptance Criteria - [ ] Completed tasks are removed from `_tasks` list automatically - [ ] `dispose()` method cancels all pending tasks before cleanup - [ ] Memory usage remains stable when processing 10,000+ messages - [ ] No tasks remain in `_tasks` after agent disposal - [ ] Existing agent functionality is not affected by cleanup changes - [ ] Unit tests verify task cleanup behavior - [ ] Integration tests confirm no memory leaks in long-running scenarios --- ## Code Evidence **Issue Location:** `src/cleveragents/agents/base.py` 1. **Lines 27-28:** Task creation and appending without cleanup ```python task = asyncio.create_task(self._process_wrapper(msg)) self._tasks.append(task) ``` 2. **Lines 79-82:** `dispose()` method doesn't cancel pending tasks ```python def dispose(self): # Current implementation doesn't handle task cleanup ``` 3. **Problem:** Tasks are appended to `_tasks` list but never removed 4. **Impact:** Long-running agents accumulate tasks in memory indefinitely --- ## Reproduction Steps ```python from cleveragents.agents.base import Agent class TestAgent(Agent): async def process_message(self, message, context=None): return f"Processed: {message}" def get_capabilities(self): return ["test"] agent = TestAgent("test-agent") # Send 10000 messages for i in range(10000): agent.send_message(f"message-{i}") # Check memory usage - _tasks list has 10000 entries print(len(agent._tasks)) # Output: 10000 # Memory is not freed even after tasks complete ``` --- ## Impact - **Severity:** HIGH - **Scope:** All agents inheriting from Agent base class - **Risk:** Memory leak in long-running agents - **Consequences:** - Unbounded growth of `_tasks` list - Out-of-memory errors in production environments - Degraded performance over time - Potential system crashes in resource-constrained environments --- ## Subtasks - [ ] Implement task completion tracking mechanism - [ ] Add periodic cleanup of completed tasks from `_tasks` list - [ ] Implement on-completion cleanup callback for tasks - [ ] Update `dispose()` method to cancel pending tasks - [ ] Consider using bounded queue instead of unbounded list - [ ] Add unit tests for task cleanup behavior - [ ] Add integration tests for long-running agent scenarios - [ ] Update documentation for task lifecycle management - [ ] Performance test with 10,000+ message processing --- ## Definition of Done - All subtasks are completed - Unit test coverage >= 97% for modified code - Integration tests pass with no memory leaks detected - Code review approved by at least one maintainer - No regressions in existing agent functionality - Documentation updated to reflect task cleanup behavior - Changes merged to main branch --- **Automated by CleverAgents Bot** Agent: new-issue-creator

HAL9000 added the

labels

2026-04-18 17:20:52 +00:00