bug(test): cli_lifecycle_e2e full plan lifecycle integration test missing #9459

Open
opened 2026-04-14 18:07:48 +00:00 by HAL9000 · 1 comment
Owner

Metadata

  • Commit Message: test(cli): add cli_lifecycle_e2e feature covering full plan use→execute→apply lifecycle
  • Branch: test/cli-lifecycle-e2e-full-plan-lifecycle

Background and Context

The test suite contains many individual BDD tests for each agents plan subcommand in isolation (e.g., plan_cli_coverage.feature, plan_cli_spec_alignment.feature, cli_lifecycle_coverage.feature), but no test verifies the complete plan lifecycle as a single integrated workflow: plan useplan executeplan apply.

The spec (docs/specification.md §Plan Lifecycle, lines ~18296–18316) defines the lifecycle as a connected workflow with specific status transitions at each phase. Testing each command in isolation misses integration bugs that only surface when the commands are chained together. This gap was identified during UAT testing and referenced in UAT task issue #5489.

Current Behavior

No features/cli_lifecycle_e2e.feature file exists. The features/ directory contains many individual plan CLI tests but none that test the full end-to-end lifecycle as a single integrated scenario. Running nox -s tests does not exercise the complete plan useplan executeplan apply workflow in a single scenario.

Expected Behavior

A features/cli_lifecycle_e2e.feature file should exist that tests the complete plan lifecycle:

  1. agents plan use <action> <project> → creates plan in strategize/queued state
  2. agents plan execute <plan_id> → runs strategize + execute phases, transitions to execute/complete
  3. agents plan apply <plan_id> → applies changes, reaches terminal state apply/applied
  4. Status transitions are verified at each step: strategize/queuedexecute/completeapply/applied
  5. JSON output at each step is validated against the spec-required envelope structure (command, status, exit_code, data, timing, messages)
  6. agents plan list shows the plan at each lifecycle stage with the correct phase/state

Acceptance Criteria

  • features/cli_lifecycle_e2e.feature exists with at least 3 end-to-end scenarios
  • Scenario: Full lifecycle plan useplan executeplan apply succeeds end-to-end
  • Scenario: Full lifecycle with --format json at each step returns spec-required envelopes
  • Scenario: Status transitions are correct at each phase (strategize/queuedexecute/completeapply/applied)
  • features/steps/cli_lifecycle_e2e_steps.py exists with all step definitions fully implemented (no placeholder steps)
  • Mock LLM actors (CLEVERAGENTS_TESTING_USE_MOCK_AI=true) are used for deterministic testing
  • All scenarios pass with nox -s tests
  • Coverage remains ≥97%

Supporting Information

  • Spec Reference: docs/specification.md §Plan Lifecycle (lines ~18296–18316)
  • Related UAT Issue: #5489
  • Files to Create:
    • features/cli_lifecycle_e2e.feature — new feature file with full lifecycle scenarios
    • features/steps/cli_lifecycle_e2e_steps.py — step definitions for the new feature

Subtasks

  • Create features/cli_lifecycle_e2e.feature with full lifecycle scenarios
  • Scenario 1: Full lifecycle plan useplan executeplan apply succeeds
  • Scenario 2: Full lifecycle with --format json at each step returns spec-required envelopes
  • Scenario 3: Status transitions are correct at each phase (strategize/queuedexecute/completeapply/applied)
  • Scenario 4: plan list shows plan at each lifecycle stage with correct phase/state
  • Create features/steps/cli_lifecycle_e2e_steps.py with all step definitions fully implemented
  • Use mock LLM actors (CLEVERAGENTS_TESTING_USE_MOCK_AI=true) for deterministic testing
  • Verify all new scenarios pass with nox -s tests
  • Verify coverage remains ≥97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • features/cli_lifecycle_e2e.feature exists with at least 3 end-to-end scenarios
  • Full lifecycle scenario passes: plan useplan executeplan apply
  • JSON output at each step is validated against spec-required envelope structure
  • Status transitions are verified at each phase
  • All scenarios pass with nox -s tests
  • Coverage ≥97%
  • All subtasks above are completed and checked off
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done

Automated by CleverAgents Bot
Agent: new-issue-creator

## Metadata - **Commit Message**: `test(cli): add cli_lifecycle_e2e feature covering full plan use→execute→apply lifecycle` - **Branch**: `test/cli-lifecycle-e2e-full-plan-lifecycle` ## Background and Context The test suite contains many individual BDD tests for each `agents plan` subcommand in isolation (e.g., `plan_cli_coverage.feature`, `plan_cli_spec_alignment.feature`, `cli_lifecycle_coverage.feature`), but no test verifies the complete plan lifecycle as a single integrated workflow: `plan use` → `plan execute` → `plan apply`. The spec (`docs/specification.md` §Plan Lifecycle, lines ~18296–18316) defines the lifecycle as a connected workflow with specific status transitions at each phase. Testing each command in isolation misses integration bugs that only surface when the commands are chained together. This gap was identified during UAT testing and referenced in UAT task issue #5489. ## Current Behavior No `features/cli_lifecycle_e2e.feature` file exists. The `features/` directory contains many individual plan CLI tests but none that test the full end-to-end lifecycle as a single integrated scenario. Running `nox -s tests` does not exercise the complete `plan use` → `plan execute` → `plan apply` workflow in a single scenario. ## Expected Behavior A `features/cli_lifecycle_e2e.feature` file should exist that tests the complete plan lifecycle: 1. `agents plan use <action> <project>` → creates plan in `strategize/queued` state 2. `agents plan execute <plan_id>` → runs strategize + execute phases, transitions to `execute/complete` 3. `agents plan apply <plan_id>` → applies changes, reaches terminal state `apply/applied` 4. Status transitions are verified at each step: `strategize/queued` → `execute/complete` → `apply/applied` 5. JSON output at each step is validated against the spec-required envelope structure (`command`, `status`, `exit_code`, `data`, `timing`, `messages`) 6. `agents plan list` shows the plan at each lifecycle stage with the correct phase/state ## Acceptance Criteria - [ ] `features/cli_lifecycle_e2e.feature` exists with at least 3 end-to-end scenarios - [ ] Scenario: Full lifecycle `plan use` → `plan execute` → `plan apply` succeeds end-to-end - [ ] Scenario: Full lifecycle with `--format json` at each step returns spec-required envelopes - [ ] Scenario: Status transitions are correct at each phase (`strategize/queued` → `execute/complete` → `apply/applied`) - [ ] `features/steps/cli_lifecycle_e2e_steps.py` exists with all step definitions fully implemented (no placeholder steps) - [ ] Mock LLM actors (`CLEVERAGENTS_TESTING_USE_MOCK_AI=true`) are used for deterministic testing - [ ] All scenarios pass with `nox -s tests` - [ ] Coverage remains ≥97% ## Supporting Information - **Spec Reference**: `docs/specification.md` §Plan Lifecycle (lines ~18296–18316) - **Related UAT Issue**: #5489 - **Files to Create**: - `features/cli_lifecycle_e2e.feature` — new feature file with full lifecycle scenarios - `features/steps/cli_lifecycle_e2e_steps.py` — step definitions for the new feature ## Subtasks - [ ] Create `features/cli_lifecycle_e2e.feature` with full lifecycle scenarios - [ ] Scenario 1: Full lifecycle `plan use` → `plan execute` → `plan apply` succeeds - [ ] Scenario 2: Full lifecycle with `--format json` at each step returns spec-required envelopes - [ ] Scenario 3: Status transitions are correct at each phase (`strategize/queued` → `execute/complete` → `apply/applied`) - [ ] Scenario 4: `plan list` shows plan at each lifecycle stage with correct phase/state - [ ] Create `features/steps/cli_lifecycle_e2e_steps.py` with all step definitions fully implemented - [ ] Use mock LLM actors (`CLEVERAGENTS_TESTING_USE_MOCK_AI=true`) for deterministic testing - [ ] Verify all new scenarios pass with `nox -s tests` - [ ] Verify coverage remains ≥97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - [ ] `features/cli_lifecycle_e2e.feature` exists with at least 3 end-to-end scenarios - [ ] Full lifecycle scenario passes: `plan use` → `plan execute` → `plan apply` - [ ] JSON output at each step is validated against spec-required envelope structure - [ ] Status transitions are verified at each phase - [ ] All scenarios pass with `nox -s tests` - [ ] Coverage ≥97% - [ ] All subtasks above are completed and checked off - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done --- **Automated by CleverAgents Bot** Agent: new-issue-creator
HAL9000 added this to the v3.2.0 milestone 2026-04-14 18:12:56 +00:00
Author
Owner

Triage Decision [AUTO-OWNR-3]: Verified as a valid test coverage gap. The cli_lifecycle_e2e full plan lifecycle integration test is missing, leaving the end-to-end plan flow untested. Should Have for v3.2.0 to ensure integration coverage.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Triage Decision [AUTO-OWNR-3]**: Verified as a valid test coverage gap. The `cli_lifecycle_e2e` full plan lifecycle integration test is missing, leaving the end-to-end plan flow untested. `Should Have` for v3.2.0 to ensure integration coverage. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#9459
No description provided.