cleveragents/cleveragents-core

Fork 3

test(integration): verify M1 success criteria — minimal plan execution flow #402

New issue

Closed

opened 2026-02-23 22:17:12 +00:00 by freemo · 1 comment

freemo commented

2026-02-23 22:17:12 +00:00

Owner

Metadata

Commit Message: test(integration): verify M1 success criteria — minimal plan execution flow
Branch: test/m1-e2e-verification

Description

Robot Framework integration test suite that exercises the complete M1 success criteria verification sequence: action creation from YAML, git resource registration, project creation and resource linking, plan use/execute/diff/apply lifecycle, and post-apply commit verification. Uses integration-appropriate mocking (e.g., mocked LLM providers).

Acceptance Criteria

Robot test creates an action from YAML config via agents action create --config
Robot test registers a git-checkout resource via agents resource add
Robot test creates a project and links the resource
Robot test runs the full plan lifecycle: plan use -> plan execute -> plan diff -> plan apply
Assertions verify Plan and Action records persist to SQLite
Assertions verify ChangeSet is built from tool invocations (not parsed output)
Assertions verify git worktree sandbox creates isolated working directory
Assertions verify sandbox changes do not affect original until Apply
Assertions verify post-apply commit exists in the target repo
nox passes with coverage >=97%

Verification Commands

# Full M1 verification sequence (from specification)
agents action create --config /tmp/test_action.yaml
agents resource add git-checkout local/main-repo --path /path/to/repo --branch main
agents project create local/test-project
agents project link-resource local/test-project local/main-repo
agents plan use local/test-action local/test-project
agents plan execute <plan_id>
agents plan diff <plan_id>
agents plan apply <plan_id>
cd /path/to/repo && git log -1

Subtasks

Write Robot Framework integration test suite covering all acceptance criteria above
Verify all assertions pass against working implementation
Tests (Behave): N/A (this is an integration test issue)
Tests (Robot): The Robot test suite IS this issue's deliverable
Tests (ASV): N/A (no benchmark needed for integration verification)
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `test(integration): verify M1 success criteria — minimal plan execution flow` - **Branch**: `test/m1-e2e-verification` ## Description Robot Framework integration test suite that exercises the complete M1 success criteria verification sequence: action creation from YAML, git resource registration, project creation and resource linking, plan use/execute/diff/apply lifecycle, and post-apply commit verification. Uses integration-appropriate mocking (e.g., mocked LLM providers). ## Acceptance Criteria - [ ] Robot test creates an action from YAML config via `agents action create --config` - [ ] Robot test registers a git-checkout resource via `agents resource add` - [ ] Robot test creates a project and links the resource - [ ] Robot test runs the full plan lifecycle: `plan use` -> `plan execute` -> `plan diff` -> `plan apply` - [ ] Assertions verify Plan and Action records persist to SQLite - [ ] Assertions verify ChangeSet is built from tool invocations (not parsed output) - [ ] Assertions verify git worktree sandbox creates isolated working directory - [ ] Assertions verify sandbox changes do not affect original until Apply - [ ] Assertions verify post-apply commit exists in the target repo - [ ] `nox` passes with coverage >=97% ## Verification Commands ```bash # Full M1 verification sequence (from specification) agents action create --config /tmp/test_action.yaml agents resource add git-checkout local/main-repo --path /path/to/repo --branch main agents project create local/test-project agents project link-resource local/test-project local/main-repo agents plan use local/test-action local/test-project agents plan execute <plan_id> agents plan diff <plan_id> agents plan apply <plan_id> cd /path/to/repo && git log -1 ``` ## Subtasks - [ ] Write Robot Framework integration test suite covering all acceptance criteria above - [ ] Verify all assertions pass against working implementation - [ ] Tests (Behave): N/A (this is an integration test issue) - [ ] Tests (Robot): The Robot test suite IS this issue's deliverable - [ ] Tests (ASV): N/A (no benchmark needed for integration verification) - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

freemo added the

labels

2026-02-23 22:17:39 +00:00

freemo added this to the v3.0.0 milestone

2026-02-23 22:17:47 +00:00

freemo added the

labels

2026-02-24 03:35:55 +00:00

brent.edwards was assigned by freemo

2026-02-24 03:37:55 +00:00

freemo referenced this issue

2026-02-24 03:42:29 +00:00

Epic: Resource Handlers & Apply Pipeline #391

~~freemo referenced this issue 2026-02-24 03:42:31 +00:00~~

Epic: Milestone Acceptance and Workflow Example Integration Tests #401

freemo added

and removed

labels

2026-02-24 21:54:21 +00:00

freemo commented

2026-02-24 21:54:46 +00:00

Author

Owner

Parent Epic: #401 (End-to-End Milestone Verification)

~~brent.edwards referenced this issue 2026-02-25 03:07:18 +00:00~~

test(e2e): verify M1 success criteria — minimal plan execution flow #425

brent.edwards added

and removed

labels

2026-02-25 03:07:33 +00:00

brent.edwards referenced this issue

2026-02-25 07:43:37 +00:00

test: consolidated Brent QA batch — issues #156, #169, #326, #402, #403 #431

freemo added a new dependency

2026-02-25 18:17:49 +00:00

#425 test(e2e): verify M1 success criteria — minimal plan execution flow

freemo referenced this issue from a pull request that will close it,

2026-02-25 18:18:07 +00:00

test(e2e): verify M1 success criteria — minimal plan execution flow #425

brent.edwards referenced this issue from a commit

2026-02-25 20:46:20 +00:00

Merge pull request 'test: consolidated Brent QA batch — issues #156, #169, #326, #402, #403' (#431) from develop-brent-4 into master

brent.edwards

2026-02-25 21:06:53 +00:00

closed this issue
added
State
Completed
and removed
State
In Review
labels

brent.edwards referenced this issue

2026-03-10 00:28:40 +00:00

bug(test): M1–M6 E2E verification suites use mocks instead of exercising the real system #658

freemo changed title from ~~test(e2e): verify M1 success criteria — minimal plan execution flow~~ to test(integration): verify M1 success criteria — minimal plan execution flow

2026-03-12 19:22:19 +00:00