cleveragents/cleveragents-core

Fork 3

test(integration): verify M6 success criteria — Firefox-scale autonomous porting #407

New issue

Closed

opened 2026-02-23 22:17:34 +00:00 by freemo · 1 comment

freemo commented

2026-02-23 22:17:34 +00:00

Owner

Metadata

Commit Message: test(integration): verify M6 success criteria — Firefox-scale autonomous porting
Branch: test/m6-e2e-verification

Description

Robot Framework integration test suite that exercises the complete M6 success criteria verification sequence: creating and executing an autonomous porting action on a large project, monitoring hierarchical decomposition, and applying the results. Uses integration-appropriate mocking (e.g., mocked LLM providers).

Acceptance Criteria

Robot test creates a porting action from YAML config
Robot test executes the porting plan on a large project
Robot test monitors hierarchical decomposition via agents plan tree
Robot test applies completed results
Assertions verify hierarchical decomposition creates 4+ levels of subplans
Assertions verify decision correction recomputes only affected subtree
Assertions verify parallel execution scales to 10+ concurrent subplans
Assertions verify a realistic porting task completes autonomously
nox passes with coverage >=97% including large-project suites

Verification Commands

# Full M6 verification sequence (from specification)
agents action create --config /tmp/port_action.yaml
agents plan use local/port-to-typescript local/large-project
agents plan execute <plan_id>
agents plan tree <plan_id>
agents plan apply <plan_id>

Subtasks

Write Robot Framework integration test suite covering all acceptance criteria above
Verify all assertions pass against working implementation
Tests (Behave): N/A (this is an integration test issue)
Tests (Robot): The Robot test suite IS this issue's deliverable
Tests (ASV): N/A (no benchmark needed for integration verification)
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `test(integration): verify M6 success criteria — Firefox-scale autonomous porting` - **Branch**: `test/m6-e2e-verification` ## Description Robot Framework integration test suite that exercises the complete M6 success criteria verification sequence: creating and executing an autonomous porting action on a large project, monitoring hierarchical decomposition, and applying the results. Uses integration-appropriate mocking (e.g., mocked LLM providers). ## Acceptance Criteria - [ ] Robot test creates a porting action from YAML config - [ ] Robot test executes the porting plan on a large project - [ ] Robot test monitors hierarchical decomposition via `agents plan tree` - [ ] Robot test applies completed results - [ ] Assertions verify hierarchical decomposition creates 4+ levels of subplans - [ ] Assertions verify decision correction recomputes only affected subtree - [ ] Assertions verify parallel execution scales to 10+ concurrent subplans - [ ] Assertions verify a realistic porting task completes autonomously - [ ] `nox` passes with coverage >=97% including large-project suites ## Verification Commands ```bash # Full M6 verification sequence (from specification) agents action create --config /tmp/port_action.yaml agents plan use local/port-to-typescript local/large-project agents plan execute <plan_id> agents plan tree <plan_id> agents plan apply <plan_id> ``` ## Subtasks - [ ] Write Robot Framework integration test suite covering all acceptance criteria above - [ ] Verify all assertions pass against working implementation - [ ] Tests (Behave): N/A (this is an integration test issue) - [ ] Tests (Robot): The Robot test suite IS this issue's deliverable - [ ] Tests (ASV): N/A (no benchmark needed for integration verification) - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

freemo added the

labels

2026-02-23 22:17:43 +00:00

freemo added this to the v3.5.0 milestone

2026-02-23 22:17:51 +00:00

freemo added the

labels

2026-02-24 03:35:59 +00:00

brent.edwards was assigned by freemo

2026-02-24 03:37:56 +00:00

freemo referenced this issue

2026-02-24 03:42:30 +00:00

Epic: Server & Autonomy Infrastructure #397

~~freemo referenced this issue 2026-02-24 03:42:31 +00:00~~

Epic: Milestone Acceptance and Workflow Example Integration Tests #401

freemo added

and removed

labels

2026-02-24 21:54:30 +00:00

freemo commented

2026-02-24 21:54:54 +00:00

Author

Owner

Parent Epic: #401 (End-to-End Milestone Verification)

brent.edwards referenced this issue from a commit

2026-02-26 19:28:05 +00:00

test(e2e): verify M6 success criteria — Firefox-scale autonomous porting

brent.edwards referenced this issue from a pull request that will close it,

2026-02-26 19:28:20 +00:00

test(e2e): verify M6 success criteria — Firefox-scale autonomous porting #457

brent.edwards added

and removed

labels

2026-02-26 19:28:27 +00:00

brent.edwards closed this issue

2026-02-27 23:00:15 +00:00

freemo referenced this issue

2026-03-02 01:56:52 +00:00

test(e2e): validate M6 acceptance criteria for v3.5.0 milestone closure #497

freemo added a new dependency

2026-03-02 02:13:38 +00:00

#497 test(e2e): validate M6 acceptance criteria for v3.5.0 milestone closure

~~brent.edwards referenced this issue 2026-03-02 21:34:00 +00:00~~

test(e2e): validate M6 acceptance criteria for v3.5.0 milestone closure #517

freemo changed title from ~~test(e2e): verify M6 success criteria — Firefox-scale autonomous porting~~ to test(integration): verify M6 success criteria — Firefox-scale autonomous porting

2026-03-12 19:22:48 +00:00