cleveragents/cleveragents-core

Fork 3

test(integration): verify M2 success criteria — actor compiler and tool routing #403

New issue

Closed

opened 2026-02-23 22:17:17 +00:00 by freemo · 1 comment

freemo commented

2026-02-23 22:17:17 +00:00

Owner

Metadata

Commit Message: test(integration): verify M2 success criteria — actor compiler and tool routing
Branch: test/m2-e2e-verification

Description

Robot Framework integration test suite that exercises the complete M2 success criteria verification sequence: actor YAML creation and loading, custom actor action creation, plan execution with custom actors, and validation of tool routing and the validation runner. Uses integration-appropriate mocking (e.g., mocked LLM providers).

Acceptance Criteria

Robot test creates an actor YAML file and loads it via agents actor add --config
Robot test creates an action referencing the custom actor
Robot test runs plan use/execute with the custom actor
Assertions verify actor YAML files parse and validate correctly
Assertions verify actors compile to LangGraph StateGraphs
Assertions verify tool router can resolve external tools
Assertions verify validation runner executes required/informational validations
Assertions verify multi-file generation produces correct ChangeSet
nox passes with coverage >=97%

Verification Commands

# Full M2 verification sequence (from specification)
agents actor add --config ./my_actor.yaml
agents action create --config /tmp/custom_action.yaml
agents plan use local/custom-action local/test-project
agents plan execute <plan_id>
agents plan validations <plan_id>

Subtasks

Write Robot Framework integration test suite covering all acceptance criteria above
Verify all assertions pass against working implementation
Tests (Behave): N/A (this is an integration test issue)
Tests (Robot): The Robot test suite IS this issue's deliverable
Tests (ASV): N/A (no benchmark needed for integration verification)
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `test(integration): verify M2 success criteria — actor compiler and tool routing` - **Branch**: `test/m2-e2e-verification` ## Description Robot Framework integration test suite that exercises the complete M2 success criteria verification sequence: actor YAML creation and loading, custom actor action creation, plan execution with custom actors, and validation of tool routing and the validation runner. Uses integration-appropriate mocking (e.g., mocked LLM providers). ## Acceptance Criteria - [ ] Robot test creates an actor YAML file and loads it via `agents actor add --config` - [ ] Robot test creates an action referencing the custom actor - [ ] Robot test runs plan use/execute with the custom actor - [ ] Assertions verify actor YAML files parse and validate correctly - [ ] Assertions verify actors compile to LangGraph StateGraphs - [ ] Assertions verify tool router can resolve external tools - [ ] Assertions verify validation runner executes required/informational validations - [ ] Assertions verify multi-file generation produces correct ChangeSet - [ ] `nox` passes with coverage >=97% ## Verification Commands ```bash # Full M2 verification sequence (from specification) agents actor add --config ./my_actor.yaml agents action create --config /tmp/custom_action.yaml agents plan use local/custom-action local/test-project agents plan execute <plan_id> agents plan validations <plan_id> ``` ## Subtasks - [ ] Write Robot Framework integration test suite covering all acceptance criteria above - [ ] Verify all assertions pass against working implementation - [ ] Tests (Behave): N/A (this is an integration test issue) - [ ] Tests (Robot): The Robot test suite IS this issue's deliverable - [ ] Tests (ASV): N/A (no benchmark needed for integration verification) - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

freemo added the

labels

2026-02-23 22:17:40 +00:00

freemo added this to the v3.1.0 milestone

2026-02-23 22:17:48 +00:00

freemo added the

labels

2026-02-24 03:35:56 +00:00

brent.edwards was assigned by freemo

2026-02-24 03:37:55 +00:00

freemo referenced this issue

2026-02-24 03:42:29 +00:00

Epic: Skill & Tool Lifecycle #393

~~freemo referenced this issue 2026-02-24 03:42:31 +00:00~~

Epic: Milestone Acceptance and Workflow Example Integration Tests #401

freemo added

and removed

labels

2026-02-24 21:54:29 +00:00

freemo commented

2026-02-24 21:54:47 +00:00

Author

Owner

Parent Epic: #401 (End-to-End Milestone Verification)

~~brent.edwards referenced this issue 2026-02-25 03:07:22 +00:00~~

test(e2e): verify M2 success criteria — actor compiler and tool routing #426

brent.edwards added

and removed

labels

2026-02-25 03:07:34 +00:00

brent.edwards referenced this issue

2026-02-25 07:43:37 +00:00

test: consolidated Brent QA batch — issues #156, #169, #326, #402, #403 #431

freemo added a new dependency

2026-02-25 18:17:49 +00:00

#426 test(e2e): verify M2 success criteria — actor compiler and tool routing

freemo referenced this issue from a pull request that will close it,

2026-02-25 18:18:03 +00:00

test(e2e): verify M2 success criteria — actor compiler and tool routing #426

brent.edwards referenced this issue from a commit

2026-02-25 20:46:20 +00:00

Merge pull request 'test: consolidated Brent QA batch — issues #156, #169, #326, #402, #403' (#431) from develop-brent-4 into master

brent.edwards

2026-02-25 21:06:56 +00:00

closed this issue
added
State
Completed
and removed
State
In Review
labels

freemo changed title from ~~test(e2e): verify M2 success criteria — actor compiler and tool routing~~ to test(integration): verify M2 success criteria — actor compiler and tool routing

2026-03-12 19:22:26 +00:00