cleveragents/cleveractors-core

Fork 0

test(robot): add end-to-end integration tests for email categorization actor graph #30

New issue

Closed

opened 2026-06-05 17:34:42 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2026-06-05 17:34:42 +00:00

Member

Metadata

Commit Message: test(robot): add end-to-end integration tests for email categorization actor graph
Branch: feature/m1-email-graph-integration-tests

Background and context

This issue adds comprehensive end-to-end integration tests for CleverActors, exercising the
library via Robot Framework (robot/) against a realistic multi-agent use case: an email
categorization pipeline powered by a graph of specialized actors.

The actor graph models an email processing system where inbound emails are categorized and
routed to domain-specific agents:

Email Categorizer Actor — routes incoming emails to the correct downstream agent
Electronic Component Orders Agent — handles purchase orders, part number extraction,
and order verification for electronic components
Personal Email Summarization Agent — summarizes personal correspondence, extracting
key dates, action items, and sentiment
Business Email Agent — processes general business communications, meeting requests,
and corporate announcements
Supplier Communication Agent — manages supplier inquiries, RFQs, delivery updates,
and invoice processing
Client Communication Agent — handles client-facing emails, proposals, support
requests, and follow-ups

BDD unit tests (Behave in features/) serve as a secondary concern — they fill any gaps
in per-module coverage that the integration tests do not naturally reach.

Part of Epic: Package Registry Client — Support Package Registry Standard v1.0.0

Expected behavior

Integration tests (primary — Robot Framework, `robot/`)

Full actor graph compiled via merge_configs with all 6 specialized agents + categorizer
End-to-end flow: raw email input → categorization → specialized agent processing → output
Each agent type tested with realistic email samples (electronic orders, personal emails,
business emails, supplier messages, client correspondence)
Actor composition verified: correct routing, argument passing, error propagation
Integration with the registry subsystem when agents reference tool packages
Graceful degradation when a specialized agent fails (error boundaries, retry logic)
Parallel execution via pabot (nox -s integration_tests)
Real HTTP stubs/services — no mocks in integration tests

Agent graph structure:

                    ┌─────────────────────┐
                    │   Email Categorizer  │
                    └────────┬────────────┘
                             │ routes to:
         ┌───────────────────┼───────────────────┐
         │                   │                    │
    ┌────▼────┐         ┌───▼────┐          ┌───▼────┐
    │Component│         │Personal│          │Business│
    │ Orders  │         │Summary │          │ Email  │
    └─────────┘         └────────┘          └───┬────┘
                                                │
                                    ┌───────────┼───────────┐
                                    │                       │
                               ┌───▼────┐              ┌───▼────┐
                               │Supplier│              │ Client │
                               └────────┘              └────────┘

BDD unit tests (secondary — Behave, `features/`)

Identify any per-module code paths not reached by integration tests
Fill coverage gaps for edge cases, error handlers, and utility functions
Ensure 97%+ coverage is maintained after integration test addition

Acceptance criteria

Integration tests

Robot Framework test suite exercising the full 6-agent email categorization graph
Each actor type tested with ≥3 realistic email scenarios
Categorizer routing tested: misrouted emails produce appropriate errors
Supplier ↔ Business agent interaction (cascading delegation) tested
Client demand → Component Order generation chain tested
Error propagation through multi-level actor graphs verified
All integration tests pass with nox -s integration_tests

BDD unit tests (gap fill)

Coverage analysis: nox -s coverage_report → identify lines missed by integration tests
Behave scenarios added ONLY for uncovered paths (not re-testing integration-covered code)
nox -s unit_tests all-green
Coverage ≥ 97% maintained

Shared

nox all-green
nox -s coverage_report ≥ 97%

Subtasks

Design and document the 6-agent email categorization graph structure
Create robot/email_categorization_graph.robot — main integration suite
Create robot/resources/email_testdata.resource — shared email fixtures
Create robot/component_orders_agent.robot — electronic component order scenarios
Create robot/personal_email_summarization_agent.robot — personal email scenarios
Create robot/business_email_agent.robot — business email scenarios
Create robot/supplier_agent.robot — supplier communication scenarios
Create robot/client_agent.robot — client communication scenarios
Run coverage gap analysis: nox -s coverage_report
Create Behave scenarios ONLY for uncovered paths in features/
Verify nox -s integration_tests all-green
Verify nox -s unit_tests all-green
Verify nox -s coverage_report ≥ 97%
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line matches the Commit Message in Metadata exactly.
The commit is pushed to the branch matching the Branch in Metadata exactly.
The commit is submitted as a PR to master, reviewed, and merged.

## Metadata Commit Message: test(robot): add end-to-end integration tests for email categorization actor graph Branch: feature/m1-email-graph-integration-tests ## Background and context This issue adds comprehensive end-to-end integration tests for CleverActors, exercising the library via Robot Framework (`robot/`) against a realistic multi-agent use case: an email categorization pipeline powered by a graph of specialized actors. The actor graph models an email processing system where inbound emails are categorized and routed to domain-specific agents: - **Email Categorizer Actor** — routes incoming emails to the correct downstream agent - **Electronic Component Orders Agent** — handles purchase orders, part number extraction, and order verification for electronic components - **Personal Email Summarization Agent** — summarizes personal correspondence, extracting key dates, action items, and sentiment - **Business Email Agent** — processes general business communications, meeting requests, and corporate announcements - **Supplier Communication Agent** — manages supplier inquiries, RFQs, delivery updates, and invoice processing - **Client Communication Agent** — handles client-facing emails, proposals, support requests, and follow-ups BDD unit tests (Behave in `features/`) serve as a secondary concern — they fill any gaps in per-module coverage that the integration tests do not naturally reach. Part of Epic: Package Registry Client — Support Package Registry Standard v1.0.0 ## Expected behavior ### Integration tests (primary — Robot Framework, `robot/`) - Full actor graph compiled via `merge_configs` with all 6 specialized agents + categorizer - End-to-end flow: raw email input → categorization → specialized agent processing → output - Each agent type tested with realistic email samples (electronic orders, personal emails, business emails, supplier messages, client correspondence) - Actor composition verified: correct routing, argument passing, error propagation - Integration with the registry subsystem when agents reference tool packages - Graceful degradation when a specialized agent fails (error boundaries, retry logic) - Parallel execution via pabot (`nox -s integration_tests`) - Real HTTP stubs/services — no mocks in integration tests **Agent graph structure:** ``` ┌─────────────────────┐ │ Email Categorizer │ └────────┬────────────┘ │ routes to: ┌───────────────────┼───────────────────┐ │ │ │ ┌────▼────┐ ┌───▼────┐ ┌───▼────┐ │Component│ │Personal│ │Business│ │ Orders │ │Summary │ │ Email │ └─────────┘ └────────┘ └───┬────┘ │ ┌───────────┼───────────┐ │ │ ┌───▼────┐ ┌───▼────┐ │Supplier│ │ Client │ └────────┘ └────────┘ ``` ### BDD unit tests (secondary — Behave, `features/`) - Identify any per-module code paths not reached by integration tests - Fill coverage gaps for edge cases, error handlers, and utility functions - Ensure 97%+ coverage is maintained after integration test addition ## Acceptance criteria ### Integration tests - [ ] Robot Framework test suite exercising the full 6-agent email categorization graph - [ ] Each actor type tested with ≥3 realistic email scenarios - [ ] Categorizer routing tested: misrouted emails produce appropriate errors - [ ] Supplier ↔ Business agent interaction (cascading delegation) tested - [ ] Client demand → Component Order generation chain tested - [ ] Error propagation through multi-level actor graphs verified - [ ] All integration tests pass with `nox -s integration_tests` ### BDD unit tests (gap fill) - [ ] Coverage analysis: `nox -s coverage_report` → identify lines missed by integration tests - [ ] Behave scenarios added ONLY for uncovered paths (not re-testing integration-covered code) - [ ] `nox -s unit_tests` all-green - [ ] Coverage ≥ 97% maintained ### Shared - [ ] `nox` all-green - [ ] `nox -s coverage_report` ≥ 97% ## Subtasks - [ ] Design and document the 6-agent email categorization graph structure - [ ] Create `robot/email_categorization_graph.robot` — main integration suite - [ ] Create `robot/resources/email_testdata.resource` — shared email fixtures - [ ] Create `robot/component_orders_agent.robot` — electronic component order scenarios - [ ] Create `robot/personal_email_summarization_agent.robot` — personal email scenarios - [ ] Create `robot/business_email_agent.robot` — business email scenarios - [ ] Create `robot/supplier_agent.robot` — supplier communication scenarios - [ ] Create `robot/client_agent.robot` — client communication scenarios - [ ] Run coverage gap analysis: `nox -s coverage_report` - [ ] Create Behave scenarios ONLY for uncovered paths in `features/` - [ ] Verify `nox -s integration_tests` all-green - [ ] Verify `nox -s unit_tests` all-green - [ ] Verify `nox -s coverage_report` ≥ 97% - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the first line matches the Commit Message in Metadata exactly. - The commit is pushed to the branch matching the Branch in Metadata exactly. - The commit is submitted as a PR to master, reviewed, and merged.

CoreRasurae added this to the v2.1.0 milestone

2026-06-05 17:34:42 +00:00

~~CoreRasurae referenced this issue 2026-06-05 17:36:35 +00:00~~

Epic: Package Registry Client — Support Package Registry Standard v1.0.0 #22

CoreRasurae added a new dependency

2026-06-05 17:41:49 +00:00

#22 Epic: Package Registry Client — Support Package Registry Standard v1.0.0

CoreRasurae changed title from ~~BDD features and test vectors — canonicalization, parsing, mock registry~~ to feat(registry): implement BDD test features and compliance test vectors

2026-06-05 17:48:12 +00:00

CoreRasurae changed title from ~~feat(registry): implement BDD test features and compliance test vectors~~ to feat(tests): add end-to-end integration tests for email categorization actor graph

2026-06-12 17:57:26 +00:00

CoreRasurae changed title from ~~feat(tests): add end-to-end integration tests for email categorization actor graph~~ to test(tests): add end-to-end integration tests for email categorization actor graph

2026-06-12 17:58:40 +00:00

CoreRasurae changed title from ~~test(tests): add end-to-end integration tests for email categorization actor graph~~ to test(robot): add end-to-end integration tests for email categorization actor graph

2026-06-12 17:59:24 +00:00

CoreRasurae referenced this issue from a commit

2026-06-12 20:25:10 +00:00

test(robot): add end-to-end integration tests for email categorization actor graph

CoreRasurae referenced this issue from a pull request that will close it,

2026-06-12 20:25:31 +00:00

test(robot): add end-to-end integration tests for email categorization actor graph #50

CoreRasurae added a new dependency

2026-06-12 20:33:23 +00:00

#50 test(robot): add end-to-end integration tests for email categorization actor graph

CoreRasurae added the

State