test(integration): workflow example 7 — CI/CD integration, automated PR review and fix (ci profile) #771

Closed
opened 2026-03-12 19:39:50 +00:00 by freemo · 1 comment
Owner

Metadata

  • Commit Message: test(integration): workflow example 7 — CI/CD integration, automated PR review and fix (ci profile)
  • Branch: test/int-wf07-cicd

Background

Integration test for Specification Workflow Example 7: CI/CD Integration — Automated PR Review and Fix. Exercises the ci automation profile (headless, non-interactive) with JSON output, idempotent registration, and polling-based completion using mocked LLM providers.

Runs within the standard nox -s integration_tests session using mocked LLM providers.

Expected Behavior

The integration test validates the CI-style headless workflow with mocked LLM responses. JSON output is verified as parseable, plan completion is polled, and all validations (lint, typecheck, tests) pass with mocked results.

Acceptance Criteria

  • Robot Framework test suite in robot/ directory (standard integration tests)
  • Test exercises ci-profile headless workflow with JSON output
  • Test uses integration-appropriate mocking (mocked LLM providers)
  • Test verifies JSON output structure
  • Test verifies idempotent resource/project registration
  • Test verifies polling-based plan completion
  • Test passes via nox -s integration_tests
  • Coverage >=97% maintained

Subtasks

  • Write Robot Framework integration test suite for workflow example 7
  • Configure mocked LLM responses for CI review scenario
  • Implement CI-style headless workflow
  • Verify via nox -s integration_tests
  • Verify coverage >=97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
## Metadata - **Commit Message**: `test(integration): workflow example 7 — CI/CD integration, automated PR review and fix (ci profile)` - **Branch**: `test/int-wf07-cicd` ## Background Integration test for Specification Workflow Example 7: CI/CD Integration — Automated PR Review and Fix. Exercises the `ci` automation profile (headless, non-interactive) with JSON output, idempotent registration, and polling-based completion using mocked LLM providers. Runs within the standard `nox -s integration_tests` session using mocked LLM providers. ## Expected Behavior The integration test validates the CI-style headless workflow with mocked LLM responses. JSON output is verified as parseable, plan completion is polled, and all validations (lint, typecheck, tests) pass with mocked results. ## Acceptance Criteria - [x] Robot Framework test suite in `robot/` directory (standard integration tests) - [x] Test exercises ci-profile headless workflow with JSON output - [x] Test uses integration-appropriate mocking (mocked LLM providers) - [x] Test verifies JSON output structure - [x] Test verifies idempotent resource/project registration - [x] Test verifies polling-based plan completion - [x] Test passes via `nox -s integration_tests` - [x] Coverage >=97% maintained ## Subtasks - [x] Write Robot Framework integration test suite for workflow example 7 - [x] Configure mocked LLM responses for CI review scenario - [x] Implement CI-style headless workflow - [x] Verify via `nox -s integration_tests` - [x] Verify coverage >=97% via `nox -s coverage_report` - [x] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.
freemo added this to the v3.6.0 milestone 2026-03-12 19:39:50 +00:00
Member

Implementation Notes

Design Decisions

  • Test Structure: Created robot/wf07_cicd_integration.robot (6 test cases) and robot/helper_wf07_cicd.py (336 lines, 6 subcommands) following the standard integration test pattern with common.resource.
  • Mocked LLM: Integration tests use CLEVERAGENTS_TESTING_USE_MOCK_AI=true per project guidelines.
  • Helper Pattern: Follows the established helper_*.py pattern used by helper_m6_autonomy_acceptance.py — each subcommand is self-contained and prints a sentinel on success.

Test Cases Implemented

  1. WF07 CI Profile Config Set And Get - ConfigService set/get for automation-profile and format
  2. WF07 Idempotent Resource Registration - ResourceRegistryService with in-memory SQLite, idempotent re-registration
  3. WF07 Idempotent Project Registration - NamespacedProjectRepository duplicate handling
  4. WF07 Validation Registration And Attachment - ValidationCommand and ValidationPipeline with mock executor
  5. WF07 CI Plan Lifecycle - PlanLifecycleService with ci profile, action creation and plan lifecycle
  6. WF07 JSON Output Parsing - JSON serialization/deserialization of action and plan CLI dict

Quality Gates

  • Lint: PASSED
  • Typecheck: PASSED (0 errors)
  • Integration tests (WF07 suite): 6/6 PASSED
  • Unit tests: 378 features, 10700 scenarios PASSED
  • Coverage: 98% (>=97% threshold)

Key Code Locations

  • Robot suite: cleveragents-core/robot/wf07_cicd_integration.robot (commit 2bd9c0a0)
  • Helper script: cleveragents-core/robot/helper_wf07_cicd.py (commit 2bd9c0a0)
## Implementation Notes ### Design Decisions - **Test Structure**: Created `robot/wf07_cicd_integration.robot` (6 test cases) and `robot/helper_wf07_cicd.py` (336 lines, 6 subcommands) following the standard integration test pattern with `common.resource`. - **Mocked LLM**: Integration tests use `CLEVERAGENTS_TESTING_USE_MOCK_AI=true` per project guidelines. - **Helper Pattern**: Follows the established `helper_*.py` pattern used by `helper_m6_autonomy_acceptance.py` — each subcommand is self-contained and prints a sentinel on success. ### Test Cases Implemented 1. **WF07 CI Profile Config Set And Get** - ConfigService set/get for automation-profile and format 2. **WF07 Idempotent Resource Registration** - ResourceRegistryService with in-memory SQLite, idempotent re-registration 3. **WF07 Idempotent Project Registration** - NamespacedProjectRepository duplicate handling 4. **WF07 Validation Registration And Attachment** - ValidationCommand and ValidationPipeline with mock executor 5. **WF07 CI Plan Lifecycle** - PlanLifecycleService with ci profile, action creation and plan lifecycle 6. **WF07 JSON Output Parsing** - JSON serialization/deserialization of action and plan CLI dict ### Quality Gates - Lint: PASSED - Typecheck: PASSED (0 errors) - Integration tests (WF07 suite): 6/6 PASSED - Unit tests: 378 features, 10700 scenarios PASSED - Coverage: 98% (>=97% threshold) ### Key Code Locations - Robot suite: `cleveragents-core/robot/wf07_cicd_integration.robot` (commit `2bd9c0a0`) - Helper script: `cleveragents-core/robot/helper_wf07_cicd.py` (commit `2bd9c0a0`)
CoreRasurae added reference test/int-wf07-cicd 2026-03-13 01:46:23 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#771
No description provided.