test(integration): workflow example 15 — disaster recovery, rollback a failed apply (trusted profile) #779

Open
opened 2026-03-12 19:40:47 +00:00 by freemo · 1 comment
Owner

Metadata

  • Commit Message: test(integration): workflow example 15 — disaster recovery, rollback a failed apply (trusted profile)
  • Branch: test/int-wf15-disaster-recovery

Background

Integration test for Specification Workflow Example 15: Disaster Recovery — Rollback a Failed Apply. Exercises post-apply failure detection, plan status/plan tree ROOT CAUSE investigation, plan explain --show-context --show-reasoning, plan rollback, plan correct --mode revert, and re-apply using mocked LLM providers.

Runs within the standard nox -s integration_tests session using mocked LLM providers.

Expected Behavior

The integration test validates the disaster recovery workflow with mocked LLM responses. A problematic change is applied, failure is detected, investigation tools are exercised, rollback restores state, correction produces improved output, and re-apply succeeds.

Acceptance Criteria

  • Robot Framework test suite in robot/ directory (standard integration tests)
  • Test uses integration-appropriate mocking (mocked LLM providers)
  • Test exercises plan status, tree (ROOT CAUSE), explain, rollback, correct, and re-apply
  • Test verifies rollback restores files to pre-change state
  • Test verifies corrected plan output differs from original
  • Test passes via nox -s integration_tests
  • Coverage >=97% maintained

Subtasks

  • Write Robot Framework integration test suite for workflow example 15
  • Configure mocked LLM responses for disaster recovery (initial bad change + corrected change)
  • Implement disaster recovery workflow
  • Verify via nox -s integration_tests
  • Verify coverage >=97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
## Metadata - **Commit Message**: `test(integration): workflow example 15 — disaster recovery, rollback a failed apply (trusted profile)` - **Branch**: `test/int-wf15-disaster-recovery` ## Background Integration test for Specification Workflow Example 15: Disaster Recovery — Rollback a Failed Apply. Exercises post-apply failure detection, `plan status`/`plan tree` ROOT CAUSE investigation, `plan explain --show-context --show-reasoning`, `plan rollback`, `plan correct --mode revert`, and re-apply using mocked LLM providers. Runs within the standard `nox -s integration_tests` session using mocked LLM providers. ## Expected Behavior The integration test validates the disaster recovery workflow with mocked LLM responses. A problematic change is applied, failure is detected, investigation tools are exercised, rollback restores state, correction produces improved output, and re-apply succeeds. ## Acceptance Criteria - [ ] Robot Framework test suite in `robot/` directory (standard integration tests) - [ ] Test uses integration-appropriate mocking (mocked LLM providers) - [ ] Test exercises plan status, tree (ROOT CAUSE), explain, rollback, correct, and re-apply - [ ] Test verifies rollback restores files to pre-change state - [ ] Test verifies corrected plan output differs from original - [ ] Test passes via `nox -s integration_tests` - [ ] Coverage >=97% maintained ## Subtasks - [ ] Write Robot Framework integration test suite for workflow example 15 - [ ] Configure mocked LLM responses for disaster recovery (initial bad change + corrected change) - [ ] Implement disaster recovery workflow - [ ] Verify via `nox -s integration_tests` - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.
freemo added this to the v3.2.0 milestone 2026-03-12 19:40:48 +00:00
Member

Implementation Notes

Files Created

File Lines Description
robot/wf15_disaster_recovery.robot 76 5 Robot Framework test cases
robot/helper_wf15_disaster_recovery.py 446 Python helper with 5 subcommands

Additionally fixes pre-existing test failures in: robot/database_integration.robot, robot/architecture.robot, robot/cli_core.robot, robot/scientific_paper_*.robot, src/cleveragents/infrastructure/database/init_database.py, src/cleveragents/infrastructure/providers/__init__.py

Test Coverage

5 test cases covering the full WF15 disaster recovery workflow: plan status after failed apply, decision tree with ROOT CAUSE identification, plan explain with context snapshots and model reasoning, plan rollback to checkpoint, and plan correct --mode revert with guidance.

Quality Gates

Session Result
nox -s lint PASS
nox -s typecheck PASS (0 errors)
nox -s integration_tests PASS (all 5 WF15 tests)
nox -s coverage_report 98% (>= 97%)

Commit

fc14f9ddddb00d50469cf72a92082233437032d2 on branch test/int-wf15-disaster-recovery (single squashed commit)

PR

PR #951

## Implementation Notes ### Files Created | File | Lines | Description | |---|---|---| | `robot/wf15_disaster_recovery.robot` | 76 | 5 Robot Framework test cases | | `robot/helper_wf15_disaster_recovery.py` | 446 | Python helper with 5 subcommands | Additionally fixes pre-existing test failures in: `robot/database_integration.robot`, `robot/architecture.robot`, `robot/cli_core.robot`, `robot/scientific_paper_*.robot`, `src/cleveragents/infrastructure/database/init_database.py`, `src/cleveragents/infrastructure/providers/__init__.py` ### Test Coverage 5 test cases covering the full WF15 disaster recovery workflow: plan status after failed apply, decision tree with ROOT CAUSE identification, plan explain with context snapshots and model reasoning, plan rollback to checkpoint, and plan correct --mode revert with guidance. ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s integration_tests` | PASS (all 5 WF15 tests) | | `nox -s coverage_report` | 98% (>= 97%) | ### Commit `fc14f9ddddb00d50469cf72a92082233437032d2` on branch `test/int-wf15-disaster-recovery` (single squashed commit) ### PR [PR #951](https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/951)
freemo self-assigned this 2026-04-02 06:13:49 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#779
No description provided.