cleveragents/cleveragents-core

Fork 3

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile) #775

New issue

Open

opened 2026-03-12 19:39:53 +00:00 by freemo · 1 comment

freemo commented

2026-03-12 19:39:53 +00:00

Owner

Metadata

Commit Message: test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)
Branch: test/int-wf11-graph-actor

Background

Integration test for Specification Workflow Example 11: Complex Graph Actor for Multi-Stage Code Review. Exercises a custom graph-type actor (5 nodes, 6 edges) with parallel fan-out to security/performance/style reviewer nodes and result synthesis using mocked LLM providers. The action is read-only.

Runs within the standard nox -s integration_tests session using mocked LLM providers.

Expected Behavior

The integration test validates graph actor execution with mocked LLM responses. Parallel fan-out, specialized sub-actors, and result synthesis are verified. No files are modified (read-only action).

Acceptance Criteria

Robot Framework test suite in robot/ directory (standard integration tests)
Test registers a custom graph-type actor with YAML topology
Test uses integration-appropriate mocking (mocked LLM providers)
Test verifies parallel fan-out to 3 reviewer nodes
Test verifies unified review report synthesis
Test verifies no file modifications (read-only)
Test passes via nox -s integration_tests
Coverage >=97% maintained

Subtasks

Write Robot Framework integration test suite for workflow example 11
Create graph actor YAML fixture
Configure mocked LLM responses for code review
Implement graph actor review workflow
Verify via nox -s integration_tests
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)` - **Branch**: `test/int-wf11-graph-actor` ## Background Integration test for Specification Workflow Example 11: Complex Graph Actor for Multi-Stage Code Review. Exercises a custom graph-type actor (5 nodes, 6 edges) with parallel fan-out to security/performance/style reviewer nodes and result synthesis using mocked LLM providers. The action is read-only. Runs within the standard `nox -s integration_tests` session using mocked LLM providers. ## Expected Behavior The integration test validates graph actor execution with mocked LLM responses. Parallel fan-out, specialized sub-actors, and result synthesis are verified. No files are modified (read-only action). ## Acceptance Criteria - [x] Robot Framework test suite in `robot/` directory (standard integration tests) - [x] Test registers a custom graph-type actor with YAML topology - [x] Test uses integration-appropriate mocking (mocked LLM providers) - [x] Test verifies parallel fan-out to 3 reviewer nodes - [x] Test verifies unified review report synthesis - [x] Test verifies no file modifications (read-only) - [x] Test passes via `nox -s integration_tests` - [x] Coverage >=97% maintained ## Subtasks - [x] Write Robot Framework integration test suite for workflow example 11 - [x] Create graph actor YAML fixture - [x] Configure mocked LLM responses for code review - [x] Implement graph actor review workflow - [x] Verify via `nox -s integration_tests` - [x] Verify coverage >=97% via `nox -s coverage_report` - [x] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

freemo added the

labels

2026-03-12 19:39:54 +00:00

brent.edwards was assigned by freemo

2026-03-12 19:39:54 +00:00

freemo added this to the v3.1.0 milestone

2026-03-12 19:39:54 +00:00

freemo added a new dependency

2026-03-12 19:39:54 +00:00

#401 Epic: Milestone Acceptance and Workflow Example Integration Tests

freemo added the

Points

label

2026-03-12 20:32:28 +00:00

brent.edwards referenced this issue from a commit

2026-03-13 05:47:47 +00:00

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)

~~brent.edwards referenced this issue 2026-03-13 05:48:18 +00:00~~

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile) #810

brent.edwards added a new dependency

2026-03-13 05:48:43 +00:00

#810 test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)

brent.edwards added

and removed

labels

2026-03-13 05:48:51 +00:00

brent.edwards commented

2026-03-13 05:49:19 +00:00

Member

Implementation Notes

Design Decisions

Graph Topology: Designed a 5-node, 6-edge graph actor matching the spec for multi-stage code review with parallel fan-out:

dispatcher (entry, agent) → fans out to 3 reviewer nodes
security_reviewer, performance_reviewer, style_reviewer (agent nodes)
synthesizer (exit, agent) → receives all 3 review reports

This creates a classic diamond/fan-out-fan-in pattern with 6 edges: 3 from dispatcher, 3 into synthesizer.

Read-Only Action: The action YAML sets read_only: true — verified via action show output and tested that plan execute handles it gracefully (no crash).

Provider Field Discovery: The actor add CLI resolves through ActorConfiguration.from_blob() which requires provider in the YAML config. The pure-domain ActorConfigSchema does not require it, but the CLI pathway does. Added provider: openai to the graph actor YAML.

JSON Output with Debug Prefix: The action show --format json CLI output can be prefixed with debug log lines (e.g., Starting attempt...), making json.loads() fail. The verify-no-file-modifications subcommand uses --format plain instead and checks for keywords, avoiding the JSON parsing issue.

Test Structure

Subcommand	What it tests
`register-graph-actor`	CLI `actor add --config` + `actor show` for graph actor
`verify-graph-topology`	Domain-level `ActorConfigSchema.from_yaml_file()` — 5 nodes, 6 edges, fan-out/fan-in
`compile-graph-actor`	`compile_actor()` → `CompiledActor` with 5 nodes, 6 edges, metadata
`create-review-action`	CLI `action create` with read-only flag + `action show` verification
`plan-use-review`	CLI `plan use` + `plan status` — plan in strategize/queued
`verify-read-only-guard`	CLI `plan execute` on read-only plan — no crash
`verify-review-synthesis`	Compiled graph: 3 incoming edges to synthesizer, sole exit node
`verify-no-file-modifications`	Action show confirms read_only and graph actor references

Key File Locations

robot/wf11_graph_actor.robot — 8 Robot test cases
robot/helper_wf11_graph_actor.py — 8-subcommand helper (~470 lines)

Verification Results

nox -s integration_tests — All 8 tests pass (77s wall time)
nox -s coverage_report — 98% (≥97% threshold)
nox -s typecheck — 0 errors, 1 pre-existing warning
nox -s lint — All checks passed
nox -s format — No changes needed
nox -s docs — Builds successfully
nox -s build — Wheel built successfully
Pre-existing async resource warnings unchanged

## Implementation Notes ### Design Decisions **Graph Topology**: Designed a 5-node, 6-edge graph actor matching the spec for multi-stage code review with parallel fan-out: - `dispatcher` (entry, agent) → fans out to 3 reviewer nodes - `security_reviewer`, `performance_reviewer`, `style_reviewer` (agent nodes) - `synthesizer` (exit, agent) → receives all 3 review reports This creates a classic diamond/fan-out-fan-in pattern with 6 edges: 3 from dispatcher, 3 into synthesizer. **Read-Only Action**: The action YAML sets `read_only: true` — verified via `action show` output and tested that `plan execute` handles it gracefully (no crash). **Provider Field Discovery**: The `actor add` CLI resolves through `ActorConfiguration.from_blob()` which requires `provider` in the YAML config. The pure-domain `ActorConfigSchema` does not require it, but the CLI pathway does. Added `provider: openai` to the graph actor YAML. **JSON Output with Debug Prefix**: The `action show --format json` CLI output can be prefixed with debug log lines (e.g., `Starting attempt...`), making `json.loads()` fail. The `verify-no-file-modifications` subcommand uses `--format plain` instead and checks for keywords, avoiding the JSON parsing issue. ### Test Structure | Subcommand | What it tests | |---|---| | `register-graph-actor` | CLI `actor add --config` + `actor show` for graph actor | | `verify-graph-topology` | Domain-level `ActorConfigSchema.from_yaml_file()` — 5 nodes, 6 edges, fan-out/fan-in | | `compile-graph-actor` | `compile_actor()` → `CompiledActor` with 5 nodes, 6 edges, metadata | | `create-review-action` | CLI `action create` with read-only flag + `action show` verification | | `plan-use-review` | CLI `plan use` + `plan status` — plan in strategize/queued | | `verify-read-only-guard` | CLI `plan execute` on read-only plan — no crash | | `verify-review-synthesis` | Compiled graph: 3 incoming edges to synthesizer, sole exit node | | `verify-no-file-modifications` | Action show confirms read_only and graph actor references | ### Key File Locations - `robot/wf11_graph_actor.robot` — 8 Robot test cases - `robot/helper_wf11_graph_actor.py` — 8-subcommand helper (~470 lines) ### Verification Results - `nox -s integration_tests` — All 8 tests pass (77s wall time) - `nox -s coverage_report` — 98% (≥97% threshold) - `nox -s typecheck` — 0 errors, 1 pre-existing warning - `nox -s lint` — All checks passed - `nox -s format` — No changes needed - `nox -s docs` — Builds successfully - `nox -s build` — Wheel built successfully - Pre-existing async resource warnings unchanged

freemo modified the milestone from v3.1.0 to v3.2.0

2026-03-16 00:31:56 +00:00

brent.edwards referenced this issue from a pull request that will close it,

2026-03-18 21:57:36 +00:00

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile) #810

brent.edwards referenced this issue from a commit

2026-03-18 22:14:25 +00:00

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)

brent.edwards referenced this issue from a commit

2026-03-18 23:14:27 +00:00

test(integration): workflow example 11 — complex graph actor for multi-stage code review (trusted profile)

brent.edwards referenced this issue