TEST-INFRA: [missing-test-levels] Add Robot integration tests for the agents module #2027

Open
opened 2026-04-03 02:42:40 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: test/agents-robot-tests
  • Commit Message: test(agents): add Robot integration tests for agents module
  • Milestone: v3.8.0
  • Parent Epic: #1678

Description

The agents module in src/cleveragents/agents is missing Robot Framework integration tests for several of its key components.

Per CONTRIBUTING.md's Multi-Level Testing Mandate, every module must have comprehensive test coverage at all three levels:

  • Behave unit tests (in features/) — may be tracked separately
  • Robot integration tests (in robot/) — partially missing
  • ASV performance benchmarks (in benchmarks/) — may be tracked separately

While context_analysis_agent.robot and agent_skills_loader.robot exist, the following components in the agents module currently lack Robot integration test coverage:

  • agents/base.py — The Agent ABC (reactive base class with async process_message, process_message_sync, input/output streams, and RxPY pipeline setup). No integration test exercises the full reactive message-passing lifecycle.
  • agents/plan_generation.py (and agents/graphs/plan_generation.py) — The PlanGenerationGraph and PlanGenerationState LangGraph-based implementation. No Robot test verifies the graph can be instantiated, invoked, and produces a valid plan structure under realistic conditions.
  • agents/graphs/auto_debug.py — The AutoDebugState-driven LangGraph auto-debug agent. No Robot test exercises the debug workflow end-to-end (error analysis → fix suggestion → validation).

As a core orchestration boundary, the agents module warrants integration-level verification to ensure correctness under realistic conditions and to guard against regressions in the reactive pipeline, LangGraph graph compilation, and agent lifecycle management.

Subtasks

  • Audit src/cleveragents/agents/ to identify all public integration points requiring Robot coverage (focus on base.py, plan_generation.py, graphs/plan_generation.py, graphs/auto_debug.py).
  • Add Robot integration tests for agents/base.py in robot/, covering the reactive agent lifecycle: instantiation, message dispatch via input_stream, and output receipt via output_stream. No mocking is permitted in Robot tests.
  • Add Robot integration tests for agents/plan_generation.py (and agents/graphs/plan_generation.py) in robot/, verifying PlanGenerationGraph can be instantiated and invoked with a realistic plan generation request, producing a valid PlanGenerationState output.
  • Add Robot integration tests for agents/graphs/auto_debug.py in robot/, verifying the AutoDebugState-driven workflow can be instantiated and run through at least one debug cycle (error input → analysis → suggested fix).
  • Verify all new Robot tests pass via nox -e integration_tests.
  • Confirm coverage ≥ 97% via nox -e coverage_report.
  • Ensure all nox stages pass (nox).

Definition of Done

  • Robot integration tests exist under robot/ covering agents/base.py, agents/plan_generation.py, agents/graphs/plan_generation.py, and agents/graphs/auto_debug.py.
  • No mocking is used in Robot integration tests.
  • The new tests exercise real agent lifecycle behaviour (reactive pipeline, LangGraph graph compilation, and state transitions) without stubbing dependencies.
  • The new tests pass in the CI pipeline.
  • All nox stages pass.
  • Coverage >= 97%
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • A pull request has been created, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/agents-robot-tests` - **Commit Message**: `test(agents): add Robot integration tests for agents module` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The `agents` module in `src/cleveragents/agents` is missing Robot Framework integration tests for several of its key components. Per CONTRIBUTING.md's Multi-Level Testing Mandate, every module must have comprehensive test coverage at all three levels: - **Behave** unit tests (in `features/`) — may be tracked separately - **Robot** integration tests (in `robot/`) — **partially missing** - **ASV** performance benchmarks (in `benchmarks/`) — may be tracked separately While `context_analysis_agent.robot` and `agent_skills_loader.robot` exist, the following components in the `agents` module currently lack Robot integration test coverage: - **`agents/base.py`** — The `Agent` ABC (reactive base class with async `process_message`, `process_message_sync`, input/output streams, and RxPY pipeline setup). No integration test exercises the full reactive message-passing lifecycle. - **`agents/plan_generation.py`** (and `agents/graphs/plan_generation.py`) — The `PlanGenerationGraph` and `PlanGenerationState` LangGraph-based implementation. No Robot test verifies the graph can be instantiated, invoked, and produces a valid plan structure under realistic conditions. - **`agents/graphs/auto_debug.py`** — The `AutoDebugState`-driven LangGraph auto-debug agent. No Robot test exercises the debug workflow end-to-end (error analysis → fix suggestion → validation). As a core orchestration boundary, the `agents` module warrants integration-level verification to ensure correctness under realistic conditions and to guard against regressions in the reactive pipeline, LangGraph graph compilation, and agent lifecycle management. ## Subtasks - [ ] Audit `src/cleveragents/agents/` to identify all public integration points requiring Robot coverage (focus on `base.py`, `plan_generation.py`, `graphs/plan_generation.py`, `graphs/auto_debug.py`). - [ ] Add Robot integration tests for `agents/base.py` in `robot/`, covering the reactive agent lifecycle: instantiation, message dispatch via `input_stream`, and output receipt via `output_stream`. No mocking is permitted in Robot tests. - [ ] Add Robot integration tests for `agents/plan_generation.py` (and `agents/graphs/plan_generation.py`) in `robot/`, verifying `PlanGenerationGraph` can be instantiated and invoked with a realistic plan generation request, producing a valid `PlanGenerationState` output. - [ ] Add Robot integration tests for `agents/graphs/auto_debug.py` in `robot/`, verifying the `AutoDebugState`-driven workflow can be instantiated and run through at least one debug cycle (error input → analysis → suggested fix). - [ ] Verify all new Robot tests pass via `nox -e integration_tests`. - [ ] Confirm coverage ≥ 97% via `nox -e coverage_report`. - [ ] Ensure all nox stages pass (`nox`). ## Definition of Done - [ ] Robot integration tests exist under `robot/` covering `agents/base.py`, `agents/plan_generation.py`, `agents/graphs/plan_generation.py`, and `agents/graphs/auto_debug.py`. - [ ] No mocking is used in Robot integration tests. - [ ] The new tests exercise real agent lifecycle behaviour (reactive pipeline, LangGraph graph compilation, and state transitions) without stubbing dependencies. - [ ] The new tests pass in the CI pipeline. - [ ] All nox stages pass. - [ ] Coverage >= 97% - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] A pull request has been created, reviewed, and merged before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 02:42:45 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2027
No description provided.