Tests (Robot): Add missing Robot Framework integration test for the agents module #2786

Open
opened 2026-04-04 19:30:15 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/agents-robot-integration
  • Commit Message: test(agents): add Robot Framework integration test suite for agents module
  • Milestone: v3.7.0
  • Parent Epic: #933

Background and Context

The src/cleveragents/agents/ module is the core AI agent execution engine of the CleverAgents platform. Per the project specification (ADR-011, §Architecture), all LangGraph-based agent workflows are the primary execution engine driving the Plan Lifecycle (Action → Strategize → Execute → Apply). The module contains the following sub-modules and public classes (confirmed via source inspection):

  • base — Abstract Agent base class and AgentWithMemory mixin (observable message processing, capability introspection, lifecycle management)
  • graphs/auto_debugAutoDebugAgent / AutoDebugState (LangGraph-based error analysis → fix generation → fix validation workflow)
  • graphs/context_analysisContextAnalysisAgent / ContextAnalysisState (file loading → dependency analysis → chunking → relevance scoring → summarisation)
  • graphs/plan_generationPlanGenerationGraph / PlanGenerationState (prompt → context load → requirements analysis → plan generation → validation workflow)

A scan of /app/robot/ confirms that no .robot file currently targets the core agents module at the integration level — agent_skills_loader.robot covers only the skills loader sub-system and context_analysis_agent.robot covers only the context analysis agent in isolation, leaving the base agent abstractions, AutoDebugAgent, and PlanGenerationGraph entirely untested at the Robot Framework integration level.

A companion Behave unit-test issue exists as #2783; this issue covers the Robot Framework integration layer, which must use real services and dependencies (no mocking permitted per CONTRIBUTING.md).

Subtasks

  • Audit each agents sub-module (base, graphs/auto_debug, graphs/context_analysis, graphs/plan_generation) and identify all integration-level behaviours to be covered
  • Create robot/agents.robot with Robot Framework test cases for each identified behaviour, exercising real agent invocations against real LLM/graph services
  • Create robot/helper_agents.py with the corresponding Python keyword library
  • Cover AutoDebugAgent.invoke — synchronous error analysis and fix generation (happy path and retry scenarios)
  • Cover AutoDebugAgent.stream — streaming execution of the debug workflow
  • Cover PlanGenerationGraph.invoke — synchronous plan generation (happy path and validation-retry scenarios)
  • Cover PlanGenerationGraph.stream — streaming plan generation
  • Cover Agent base class lifecycle: instantiation, configuration, capability introspection, and dispose
  • Cover AgentWithMemory — memory-augmented agent construction and message processing
  • Ensure all test cases use real services — no mocking permitted in integration tests
  • Run nox -e integration_tests locally and confirm all new test cases pass
  • Verify coverage >= 97% via nox -e coverage_report
  • Run nox (all default sessions) and fix any errors

Definition of Done

  • All subtasks above are completed and checked off
  • robot/agents.robot exists and covers all public integration-level behaviours of the agents module
  • robot/helper_agents.py exists with all required keyword implementations
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(agents): add Robot Framework integration test suite for agents module), followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/agents-robot-integration)
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/agents-robot-integration` - **Commit Message**: `test(agents): add Robot Framework integration test suite for agents module` - **Milestone**: v3.7.0 - **Parent Epic**: #933 ## Background and Context The `src/cleveragents/agents/` module is the core AI agent execution engine of the CleverAgents platform. Per the project specification (ADR-011, §Architecture), all LangGraph-based agent workflows are the primary execution engine driving the Plan Lifecycle (Action → Strategize → Execute → Apply). The module contains the following sub-modules and public classes (confirmed via source inspection): - `base` — Abstract `Agent` base class and `AgentWithMemory` mixin (observable message processing, capability introspection, lifecycle management) - `graphs/auto_debug` — `AutoDebugAgent` / `AutoDebugState` (LangGraph-based error analysis → fix generation → fix validation workflow) - `graphs/context_analysis` — `ContextAnalysisAgent` / `ContextAnalysisState` (file loading → dependency analysis → chunking → relevance scoring → summarisation) - `graphs/plan_generation` — `PlanGenerationGraph` / `PlanGenerationState` (prompt → context load → requirements analysis → plan generation → validation workflow) A scan of `/app/robot/` confirms that **no `.robot` file currently targets the core `agents` module** at the integration level — `agent_skills_loader.robot` covers only the skills loader sub-system and `context_analysis_agent.robot` covers only the context analysis agent in isolation, leaving the base agent abstractions, `AutoDebugAgent`, and `PlanGenerationGraph` entirely untested at the Robot Framework integration level. A companion Behave unit-test issue exists as #2783; this issue covers the Robot Framework integration layer, which must use **real services and dependencies** (no mocking permitted per CONTRIBUTING.md). ## Subtasks - [ ] Audit each `agents` sub-module (`base`, `graphs/auto_debug`, `graphs/context_analysis`, `graphs/plan_generation`) and identify all integration-level behaviours to be covered - [ ] Create `robot/agents.robot` with Robot Framework test cases for each identified behaviour, exercising real agent invocations against real LLM/graph services - [ ] Create `robot/helper_agents.py` with the corresponding Python keyword library - [ ] Cover `AutoDebugAgent.invoke` — synchronous error analysis and fix generation (happy path and retry scenarios) - [ ] Cover `AutoDebugAgent.stream` — streaming execution of the debug workflow - [ ] Cover `PlanGenerationGraph.invoke` — synchronous plan generation (happy path and validation-retry scenarios) - [ ] Cover `PlanGenerationGraph.stream` — streaming plan generation - [ ] Cover `Agent` base class lifecycle: instantiation, configuration, capability introspection, and `dispose` - [ ] Cover `AgentWithMemory` — memory-augmented agent construction and message processing - [ ] Ensure all test cases use real services — no mocking permitted in integration tests - [ ] Run `nox -e integration_tests` locally and confirm all new test cases pass - [ ] Verify coverage >= 97% via `nox -e coverage_report` - [ ] Run `nox` (all default sessions) and fix any errors ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] `robot/agents.robot` exists and covers all public integration-level behaviours of the `agents` module - [ ] `robot/helper_agents.py` exists with all required keyword implementations - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(agents): add Robot Framework integration test suite for agents module`), followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/agents-robot-integration`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-04 19:30:20 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified | MoSCoW: Should Have — Missing Robot Framework integration test for the agents module.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified | **MoSCoW**: Should Have — Missing Robot Framework integration test for the agents module. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2786
No description provided.