UAT: No E2E acceptance test for v3.6.0 (M7) — robot/e2e/m7_acceptance.robot missing while M1, M2, M5, M6 all have acceptance tests #3328

Open
opened 2026-04-05 09:53:19 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: test/e2e-m7-acceptance
  • Commit Message: test(e2e): add M7 acceptance test suite for v3.6.0 advanced concepts milestone
  • Milestone: None (backlog — see note below)
  • Parent Epic: #398

Backlog note: This issue was discovered during autonomous operation
on milestone v3.6.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Background

The v3.6.0 milestone (M7: Advanced Concepts & Deferred Features) is missing a dedicated E2E acceptance test file. The E2E test directory (robot/e2e/) contains acceptance tests for other milestones:

  • robot/e2e/m1_acceptance.robot — M1 (v3.0.0) acceptance tests
  • robot/e2e/m2_acceptance.robot — M2 (v3.1.0) acceptance tests
  • robot/e2e/m5_acceptance.robot — M5 (v3.4.0) acceptance tests
  • robot/e2e/m6_acceptance.robot — M6 (v3.5.0) acceptance tests

However, there is no robot/e2e/m7_acceptance.robot for M7 (v3.6.0).

The v3.6.0 milestone scope (per docs/specification.md and the milestone description) includes:

  • Advanced context strategies (beyond basic ACMS pipeline)
  • Additional LLM backends and provider integrations
  • Additional resource types (cloud infrastructure, database resources, virtual types)
  • ACP to A2A module rename and symbol standardization
  • Container tool execution
  • Cost/session budgets, safety profiles
  • E2E workflow specification tests

The spec explicitly lists "E2E workflow specification tests" as a v3.6.0 deliverable, yet no M7 acceptance test file exists.

Current Behavior

robot/e2e/m7_acceptance.robot does not exist. The existing workflow E2E tests (robot/e2e/wf16_devcontainer.robot, robot/e2e/wf17_explicit_container.robot, robot/e2e/wf18_container_clone.robot) cover some container scenarios but do not constitute a comprehensive M7 acceptance test that validates all v3.6.0 acceptance criteria.

Steps to reproduce:

  1. Check robot/e2e/ directory — no m7_acceptance.robot file exists
  2. Compare with other milestones — M1, M2, M5, M6 all have dedicated acceptance tests

Expected Behavior

robot/e2e/m7_acceptance.robot should exist with Robot Framework test cases covering all v3.6.0 acceptance criteria, following the same structure and conventions as m1_acceptance.robot, m2_acceptance.robot, m5_acceptance.robot, and m6_acceptance.robot.

Acceptance Criteria

  • robot/e2e/m7_acceptance.robot exists in the repository
  • The file follows the same structural conventions as the other milestone acceptance test files
  • Test cases cover all v3.6.0 acceptance criteria: advanced context strategies, additional LLM backends, additional resource types (cloud, database, virtual), ACP→A2A rename, container tool execution, cost/session budgets, safety profiles, and E2E workflow specification tests
  • All test cases in m7_acceptance.robot pass in CI
  • Coverage remains ≥ 97%

Supporting Information

  • Code location: robot/e2e/ directory — missing m7_acceptance.robot
  • Related files: robot/e2e/m1_acceptance.robot, robot/e2e/m2_acceptance.robot, robot/e2e/m5_acceptance.robot, robot/e2e/m6_acceptance.robot
  • Related workflow tests: robot/e2e/wf16_devcontainer.robot, robot/e2e/wf17_explicit_container.robot, robot/e2e/wf18_container_clone.robot
  • Milestone spec: v3.6.0 milestone description explicitly lists "E2E workflow specification tests" as a deliverable

Subtasks

  • Review existing milestone acceptance test files (m1_acceptance.robot, m2_acceptance.robot, m5_acceptance.robot, m6_acceptance.robot) to understand structure and conventions
  • Review v3.6.0 milestone acceptance criteria from docs/specification.md and milestone description
  • Create robot/e2e/m7_acceptance.robot with test suites covering all v3.6.0 acceptance criteria
  • Add test cases for advanced context strategies (beyond basic ACMS pipeline)
  • Add test cases for additional LLM backends and provider integrations
  • Add test cases for additional resource types (cloud infrastructure, database resources, virtual types)
  • Add test cases for ACP→A2A module rename and symbol standardization
  • Add test cases for container tool execution
  • Add test cases for cost/session budgets and safety profiles
  • Add test cases for E2E workflow specification tests
  • Verify all test cases pass locally via the project task runner
  • Verify coverage ≥ 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • robot/e2e/m7_acceptance.robot exists and all its test cases pass in CI.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(e2e): add M7 acceptance test suite for v3.6.0 advanced concepts milestone), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/e2e-m7-acceptance).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/e2e-m7-acceptance` - **Commit Message**: `test(e2e): add M7 acceptance test suite for v3.6.0 advanced concepts milestone` - **Milestone**: None (backlog — see note below) - **Parent Epic**: #398 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.6.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Background The v3.6.0 milestone (M7: Advanced Concepts & Deferred Features) is missing a dedicated E2E acceptance test file. The E2E test directory (`robot/e2e/`) contains acceptance tests for other milestones: - `robot/e2e/m1_acceptance.robot` — M1 (v3.0.0) acceptance tests - `robot/e2e/m2_acceptance.robot` — M2 (v3.1.0) acceptance tests - `robot/e2e/m5_acceptance.robot` — M5 (v3.4.0) acceptance tests - `robot/e2e/m6_acceptance.robot` — M6 (v3.5.0) acceptance tests However, there is **no** `robot/e2e/m7_acceptance.robot` for M7 (v3.6.0). The v3.6.0 milestone scope (per `docs/specification.md` and the milestone description) includes: - Advanced context strategies (beyond basic ACMS pipeline) - Additional LLM backends and provider integrations - Additional resource types (cloud infrastructure, database resources, virtual types) - ACP to A2A module rename and symbol standardization - Container tool execution - Cost/session budgets, safety profiles - E2E workflow specification tests The spec explicitly lists **"E2E workflow specification tests"** as a v3.6.0 deliverable, yet no M7 acceptance test file exists. ## Current Behavior `robot/e2e/m7_acceptance.robot` does not exist. The existing workflow E2E tests (`robot/e2e/wf16_devcontainer.robot`, `robot/e2e/wf17_explicit_container.robot`, `robot/e2e/wf18_container_clone.robot`) cover some container scenarios but do not constitute a comprehensive M7 acceptance test that validates all v3.6.0 acceptance criteria. **Steps to reproduce:** 1. Check `robot/e2e/` directory — no `m7_acceptance.robot` file exists 2. Compare with other milestones — M1, M2, M5, M6 all have dedicated acceptance tests ## Expected Behavior `robot/e2e/m7_acceptance.robot` should exist with Robot Framework test cases covering all v3.6.0 acceptance criteria, following the same structure and conventions as `m1_acceptance.robot`, `m2_acceptance.robot`, `m5_acceptance.robot`, and `m6_acceptance.robot`. ## Acceptance Criteria - [ ] `robot/e2e/m7_acceptance.robot` exists in the repository - [ ] The file follows the same structural conventions as the other milestone acceptance test files - [ ] Test cases cover all v3.6.0 acceptance criteria: advanced context strategies, additional LLM backends, additional resource types (cloud, database, virtual), ACP→A2A rename, container tool execution, cost/session budgets, safety profiles, and E2E workflow specification tests - [ ] All test cases in `m7_acceptance.robot` pass in CI - [ ] Coverage remains ≥ 97% ## Supporting Information - **Code location**: `robot/e2e/` directory — missing `m7_acceptance.robot` - **Related files**: `robot/e2e/m1_acceptance.robot`, `robot/e2e/m2_acceptance.robot`, `robot/e2e/m5_acceptance.robot`, `robot/e2e/m6_acceptance.robot` - **Related workflow tests**: `robot/e2e/wf16_devcontainer.robot`, `robot/e2e/wf17_explicit_container.robot`, `robot/e2e/wf18_container_clone.robot` - **Milestone spec**: v3.6.0 milestone description explicitly lists "E2E workflow specification tests" as a deliverable ## Subtasks - [ ] Review existing milestone acceptance test files (`m1_acceptance.robot`, `m2_acceptance.robot`, `m5_acceptance.robot`, `m6_acceptance.robot`) to understand structure and conventions - [ ] Review v3.6.0 milestone acceptance criteria from `docs/specification.md` and milestone description - [ ] Create `robot/e2e/m7_acceptance.robot` with test suites covering all v3.6.0 acceptance criteria - [ ] Add test cases for advanced context strategies (beyond basic ACMS pipeline) - [ ] Add test cases for additional LLM backends and provider integrations - [ ] Add test cases for additional resource types (cloud infrastructure, database resources, virtual types) - [ ] Add test cases for ACP→A2A module rename and symbol standardization - [ ] Add test cases for container tool execution - [ ] Add test cases for cost/session budgets and safety profiles - [ ] Add test cases for E2E workflow specification tests - [ ] Verify all test cases pass locally via the project task runner - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - `robot/e2e/m7_acceptance.robot` exists and all its test cases pass in CI. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(e2e): add M7 acceptance test suite for v3.6.0 advanced concepts milestone`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/e2e-m7-acceptance`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator
freemo added this to the v3.6.0 milestone 2026-04-05 10:01:02 +00:00
freemo removed this from the v3.6.0 milestone 2026-04-05 10:05:12 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#398 Epic: Post-MVP Resources
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#3328
No description provided.