cleveragents/cleveragents-core

Fork 3

[Bug Hunt][Cycle 2][Reactive] Memory Leak in ReactiveStreamRouter Observables #7136

New issue

Open

opened 2026-04-10 08:07:36 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-10 08:07:36 +00:00

Owner

Background

The ReactiveStreamRouter in the reactive subsystem maintains observables and subscriptions in internal dictionaries without any cleanup or disposal mechanism. This is a resource management defect that causes unbounded memory growth in long-running processes.

Location: src/cleveragents/reactive/stream_router.py, ReactiveStreamRouter.__init__() and stream lifecycle management

Current Behavior

ReactiveStreamRouter.__init__() initialises the following collections with no corresponding teardown logic:

def __init__(self, scheduler: AsyncIOScheduler | None = None):
    self.streams: dict[str, Any] = {}
    self.stream_configs: dict[str, StreamConfig] = {}
    self.observables: dict[str, ObservableType] = {}
    self.agents: dict[str, Any] = {}
    self.subscriptions: list[Any] = []

When streams are created but not explicitly disposed, the associated ObservableType entries in self.observables and subscription handles in self.subscriptions remain in memory indefinitely. There is no eviction policy, no dispose() / cleanup() method, and no __del__ or context-manager support to release resources when streams are removed or the router is shut down.

Expected Behavior

The ReactiveStreamRouter should provide a lifecycle-aware cleanup mechanism:

A dispose_stream(stream_name: str) method (or equivalent) that removes the stream's observable from self.observables, disposes its subscription(s), and removes the entry from self.streams and self.stream_configs
A shutdown() / dispose_all() method that disposes every active observable and clears all internal collections
Optionally, __aenter__ / __aexit__ support so the router can be used as an async context manager

Impact

Memory usage grows unbounded in long-running processes. In production deployments with many short-lived streams (e.g., per-request reactive pipelines), this will eventually cause out-of-memory errors and process crashes.

Acceptance Criteria

ReactiveStreamRouter exposes a disposal API that removes observables and subscriptions from internal collections
After calling the disposal API, self.observables and self.subscriptions are empty (or contain only active streams)
The TDD scenario introduced in #7135 passes without @tdd_expected_fail
No regression in existing reactive subsystem tests
Memory profiling confirms no growth after repeated stream creation and disposal

Metadata

Branch: bugfix/m3.2.0-reactive-stream-router-memory-leak
Commit Message: fix(reactive): add disposal mechanism to ReactiveStreamRouter to prevent observable memory leak
Milestone: v3.2.0
Parent Epic: To be linked — see orphan note below

Subtasks

Confirm the TDD issue (#7135) PR has been merged and @tdd_expected_fail scenario is on master
Implement dispose_stream(stream_name: str) on ReactiveStreamRouter — disposes subscription(s) and removes entries from observables, streams, stream_configs, and agents
Implement dispose_all() / shutdown() — disposes all active subscriptions and clears all internal collections
Optionally add __aenter__ / __aexit__ for async context-manager usage
Remove @tdd_expected_fail from the @tdd_issue_7136 scenario (leave @tdd_issue and @tdd_issue_7136 permanently)
Verify the TDD scenario now passes without @tdd_expected_fail
Update docstrings and inline documentation for the new disposal API
Run nox — all stages must pass
Open PR with Closes #7136 in the description

Definition of Done

dispose_stream() and dispose_all() implemented and type-annotated
@tdd_expected_fail removed from @tdd_issue_7136 scenario
TDD scenario passes on the bugfix branch
All nox stages pass
Coverage >= 97%
PR opened, reviewed, and merged to master
This issue closed when PR is merged

Backlog note: This issue was discovered during autonomous operation
on milestone v3.2.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Bug Hunt | Agent: new-issue-creator

## Background The `ReactiveStreamRouter` in the reactive subsystem maintains observables and subscriptions in internal dictionaries without any cleanup or disposal mechanism. This is a resource management defect that causes unbounded memory growth in long-running processes. **Location**: `src/cleveragents/reactive/stream_router.py`, `ReactiveStreamRouter.__init__()` and stream lifecycle management ## Current Behavior `ReactiveStreamRouter.__init__()` initialises the following collections with no corresponding teardown logic: ```python def __init__(self, scheduler: AsyncIOScheduler | None = None): self.streams: dict[str, Any] = {} self.stream_configs: dict[str, StreamConfig] = {} self.observables: dict[str, ObservableType] = {} self.agents: dict[str, Any] = {} self.subscriptions: list[Any] = [] ``` When streams are created but not explicitly disposed, the associated `ObservableType` entries in `self.observables` and subscription handles in `self.subscriptions` remain in memory indefinitely. There is no eviction policy, no `dispose()` / `cleanup()` method, and no `__del__` or context-manager support to release resources when streams are removed or the router is shut down. ## Expected Behavior The `ReactiveStreamRouter` should provide a lifecycle-aware cleanup mechanism: - A `dispose_stream(stream_name: str)` method (or equivalent) that removes the stream's observable from `self.observables`, disposes its subscription(s), and removes the entry from `self.streams` and `self.stream_configs` - A `shutdown()` / `dispose_all()` method that disposes every active observable and clears all internal collections - Optionally, `__aenter__` / `__aexit__` support so the router can be used as an async context manager ## Impact Memory usage grows unbounded in long-running processes. In production deployments with many short-lived streams (e.g., per-request reactive pipelines), this will eventually cause out-of-memory errors and process crashes. ## Acceptance Criteria - `ReactiveStreamRouter` exposes a disposal API that removes observables and subscriptions from internal collections - After calling the disposal API, `self.observables` and `self.subscriptions` are empty (or contain only active streams) - The TDD scenario introduced in #7135 passes without `@tdd_expected_fail` - No regression in existing reactive subsystem tests - Memory profiling confirms no growth after repeated stream creation and disposal ## Metadata - **Branch**: `bugfix/m3.2.0-reactive-stream-router-memory-leak` - **Commit Message**: `fix(reactive): add disposal mechanism to ReactiveStreamRouter to prevent observable memory leak` - **Milestone**: v3.2.0 - **Parent Epic**: _To be linked — see orphan note below_ ## Subtasks - [ ] Confirm the TDD issue (#7135) PR has been merged and `@tdd_expected_fail` scenario is on `master` - [ ] Implement `dispose_stream(stream_name: str)` on `ReactiveStreamRouter` — disposes subscription(s) and removes entries from `observables`, `streams`, `stream_configs`, and `agents` - [ ] Implement `dispose_all()` / `shutdown()` — disposes all active subscriptions and clears all internal collections - [ ] Optionally add `__aenter__` / `__aexit__` for async context-manager usage - [ ] Remove `@tdd_expected_fail` from the `@tdd_issue_7136` scenario (leave `@tdd_issue` and `@tdd_issue_7136` permanently) - [ ] Verify the TDD scenario now passes without `@tdd_expected_fail` - [ ] Update docstrings and inline documentation for the new disposal API - [ ] Run `nox` — all stages must pass - [ ] Open PR with `Closes #7136` in the description ## Definition of Done - [ ] `dispose_stream()` and `dispose_all()` implemented and type-annotated - [ ] `@tdd_expected_fail` removed from `@tdd_issue_7136` scenario - [ ] TDD scenario passes on the bugfix branch - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] PR opened, reviewed, and merged to `master` - [ ] This issue closed when PR is merged > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.2.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunt | Agent: new-issue-creator

HAL9000 referenced this issue

2026-04-10 08:08:06 +00:00

TDD: [Bug Hunt][Cycle 2][Reactive] Memory Leak in ReactiveStreamRouter Observables #7135

HAL9000 added the

labels

2026-04-10 08:08:53 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-10 08:09:36 +00:00

HAL9000 added a new dependency

2026-04-10 08:09:48 +00:00

#7135 TDD: [Bug Hunt][Cycle 2][Reactive] Memory Leak in ReactiveStreamRouter Observables

HAL9000 commented

2026-04-10 08:10:03 +00:00

Author

Owner

⚠️ Orphan Issue — Needs Manual Parent Epic Linking

This bug issue was created during autonomous Bug Hunt Cycle 2 operation. No parent Epic for the Bug Hunt Cycle 2 Reactive module was found in the issue tracker at creation time.

A maintainer must link this issue to the appropriate parent Epic using Forgejo's dependency system:

This issue should block the parent Epic
The parent Epic should depend on this issue

Suggested parent: the Bug Hunt Cycle 2 Reactive Epic (if one exists), or the v3.2.0 reactive subsystem Epic.

TDD dependency: This issue depends on #7135 (TDD issue must be merged first before the fix can be implemented).

Automated by CleverAgents Bot
Supervisor: Bug Hunt | Agent: new-issue-creator

⚠️ **Orphan Issue — Needs Manual Parent Epic Linking** This bug issue was created during autonomous Bug Hunt Cycle 2 operation. No parent Epic for the Bug Hunt Cycle 2 Reactive module was found in the issue tracker at creation time. A maintainer must link this issue to the appropriate parent Epic using Forgejo's dependency system: - This issue should **block** the parent Epic - The parent Epic should **depend on** this issue Suggested parent: the Bug Hunt Cycle 2 Reactive Epic (if one exists), or the v3.2.0 reactive subsystem Epic. **TDD dependency**: This issue depends on #7135 (TDD issue must be merged first before the fix can be implemented). --- **Automated by CleverAgents Bot** Supervisor: Bug Hunt | Agent: new-issue-creator

HAL9000 referenced this issue

2026-04-10 08:24:18 +00:00

[AUTO-BUG-POOL] Worker 15 Started - Reactive Programming Analysis #7033

HAL9000 added

and removed

labels

2026-04-14 06:48:45 +00:00

HAL9000 commented

2026-04-14 06:48:45 +00:00

Author

Owner

✅ Verified — Critical resource bug: memory leak in ReactiveStreamRouter Observables. MoSCoW: Must-have. Priority: Critical.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — Critical resource bug: memory leak in ReactiveStreamRouter Observables. MoSCoW: Must-have. Priority: Critical. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor