cleveragents/cleveragents-core

Fork 3

UAT: wf04_multi_project.robot E2E test has empty test body — multi-project dependency update workflow not implemented #5989

New issue

Open

opened 2026-04-09 13:11:45 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 13:11:45 +00:00

Owner

Bug Report

Feature Area: E2E Workflow Specification Tests — WF04 Multi-Project Dependency Update
Milestone: v3.6.0 (M7) — E2E workflow specification tests
Severity: Priority/Backlog — test infrastructure gap, not blocking runtime

What Was Tested

Code-level analysis of robot/e2e/wf04_multi_project.robot against Specification Example 4: Multi-Project Dependency Update.

Expected Behavior (from spec)

Specification Example 4 describes a supervised-profile workflow where:

Three microservices share a common library with a breaking change (v1 to v2)
CleverAgents creates a parent plan targeting all 4 projects
Child subplans are spawned per project
Execution proceeds in dependency order (common-lib first, then services)
Per-project validation runs
Apply proceeds in dependency order

The test file has extensive keyword infrastructure (Create Library Repo, Create Service Repo, Register Resource And Project, Register Validation For Project, Verify WF04 Child Plan Spawning, Verify WF04 Execution Order, Verify WF04 Validation Outcomes, Verify WF04 Apply Order) but the test case body is empty.

Actual Behavior

The test case body (WF04 Multi Project Dependency Update Supervised Profile) contains only:

WF04 Multi Project Dependency Update Supervised Profile
  [Tags]    tdd_issue tdd_issue_4189
  [Timeout]    25 minutes
  [Teardown]    WF04 Test Teardown
  Skip If No LLM Keys
  # Initialise test variable for teardown access.
  Set Test Variable    ${WF04_PLAN_ID}    ${EMPTY}
  # (lines 328-346 are blank — no further steps)

The test initializes a variable and then does nothing. All the verification keywords (Verify WF04 Child Plan Spawning, Verify WF04 Execution Order, etc.) are defined but never called.

Code Location

File: robot/e2e/wf04_multi_project.robot
Lines 315-346: Test case body — only variable initialization, 18+ blank lines where implementation should be
Lines 60-305: Extensive keyword infrastructure that is never invoked

Impact

The WF04 E2E test provides no coverage of the multi-project dependency update workflow. The 4-project setup, child plan spawning, dependency-ordered execution, and validation outcomes are all untested end-to-end.

Definition of Done

The test body should:

Create 4 git repos (common-lib + 3 services) using Create Library Repo and Create Service Repo
Register resources and projects using Register Resource And Project
Register validations using Register Validation For Project
Create multi-project action with invariants
Run plan use targeting all 4 projects with --automation-profile supervised
Run plan execute and wait for child plan spawning
Call Verify WF04 Child Plan Spawning to assert 4 child plans
Call Verify WF04 Execution Order to assert dependency ordering
Call Verify WF04 Validation Outcomes to assert per-project validation
Call Verify WF04 Apply Order to assert apply ordering

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: E2E Workflow Specification Tests — WF04 Multi-Project Dependency Update **Milestone**: v3.6.0 (M7) — E2E workflow specification tests **Severity**: Priority/Backlog — test infrastructure gap, not blocking runtime ### What Was Tested Code-level analysis of `robot/e2e/wf04_multi_project.robot` against Specification Example 4: Multi-Project Dependency Update. ### Expected Behavior (from spec) Specification Example 4 describes a supervised-profile workflow where: 1. Three microservices share a common library with a breaking change (v1 to v2) 2. CleverAgents creates a parent plan targeting all 4 projects 3. Child subplans are spawned per project 4. Execution proceeds in dependency order (common-lib first, then services) 5. Per-project validation runs 6. Apply proceeds in dependency order The test file has extensive keyword infrastructure (Create Library Repo, Create Service Repo, Register Resource And Project, Register Validation For Project, Verify WF04 Child Plan Spawning, Verify WF04 Execution Order, Verify WF04 Validation Outcomes, Verify WF04 Apply Order) but the test case body is empty. ### Actual Behavior The test case body (`WF04 Multi Project Dependency Update Supervised Profile`) contains only: ```robot WF04 Multi Project Dependency Update Supervised Profile [Tags] tdd_issue tdd_issue_4189 [Timeout] 25 minutes [Teardown] WF04 Test Teardown Skip If No LLM Keys # Initialise test variable for teardown access. Set Test Variable ${WF04_PLAN_ID} ${EMPTY} # (lines 328-346 are blank — no further steps) ``` The test initializes a variable and then does nothing. All the verification keywords (Verify WF04 Child Plan Spawning, Verify WF04 Execution Order, etc.) are defined but never called. ### Code Location - File: `robot/e2e/wf04_multi_project.robot` - Lines 315-346: Test case body — only variable initialization, 18+ blank lines where implementation should be - Lines 60-305: Extensive keyword infrastructure that is never invoked ### Impact The WF04 E2E test provides no coverage of the multi-project dependency update workflow. The 4-project setup, child plan spawning, dependency-ordered execution, and validation outcomes are all untested end-to-end. ### Definition of Done The test body should: 1. Create 4 git repos (common-lib + 3 services) using `Create Library Repo` and `Create Service Repo` 2. Register resources and projects using `Register Resource And Project` 3. Register validations using `Register Validation For Project` 4. Create multi-project action with invariants 5. Run `plan use` targeting all 4 projects with `--automation-profile supervised` 6. Run `plan execute` and wait for child plan spawning 7. Call `Verify WF04 Child Plan Spawning` to assert 4 child plans 8. Call `Verify WF04 Execution Order` to assert dependency ordering 9. Call `Verify WF04 Validation Outcomes` to assert per-project validation 10. Call `Verify WF04 Apply Order` to assert apply ordering --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-09 13:26:38 +00:00

[AUTO-WATCHDOG] System Health Report (Cycle 21) #5915

HAL9000 added the

labels

2026-04-09 13:27:58 +00:00

HAL9000 added a new dependency

2026-04-09 13:28:54 +00:00

#5179 EPIC: E2E Workflow Specification Tests — Full Workflow Validation Suite (v3.6.0)

HAL9000 added

and removed

labels

2026-04-09 13:39:20 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 13:39:40 +00:00

HAL9000 referenced this issue

2026-04-09 13:43:41 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 30) #5908

HAL9000 commented

2026-04-09 13:43:55 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 referenced this issue

2026-04-09 13:56:52 +00:00

[AUTO-EPIC] Epic Planning Health Report (Cycle 21) #5883

HAL9000 added

and removed

labels

2026-04-09 13:59:02 +00:00