cleveragents/cleveragents-core

Fork 3

UAT: m2_acceptance.robot E2E test has empty test body — actor compiler integration steps missing #5986

New issue

Open

opened 2026-04-09 12:55:12 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 12:55:12 +00:00

Owner

Bug Report

Feature Area: E2E Workflow Specification Tests — M2 Acceptance
Milestone: v3.6.0 (M7) — E2E workflow specification tests
Severity: Priority/Backlog — test infrastructure gap, not blocking runtime

What Was Tested

Code-level analysis of robot/e2e/m2_acceptance.robot against the M2 milestone acceptance criteria.

Expected Behavior (from spec)

The M2 acceptance test should exercise the complete actor compiler and LLM integration flow:

Create temp git repo with sample project files
Compile actor YAML into a LangGraph graph
Register actor via actor add --config actor.yaml
Create action referencing the custom actor
Create resource and project
Run full plan lifecycle (use, execute strategize, execute, diff, apply) with custom actor

The test file's own documentation states:

Exercises actor YAML compilation, registration via CLI, resource and project setup, action creation referencing a custom actor, and the full plan lifecycle (use, execute strategize, execute, diff, apply) with real LLM API keys.

Actual Behavior

The test case body contains only git repo creation and branch detection steps, then ends abruptly:

M2 Full Actor Compiler And LLM Integration
  [Tags]    E2E tdd_issue tdd_issue_4189 tdd_expected_fail
  Skip If No LLM Keys
  # ---- Step 1: Create temp git repo with sample project files ----
  ${repo_dir}=    Create Temp Git Repo    m2-e2e-repo
  Create Directory    ${repo_dir}${/}src
  Create File    ${repo_dir}${/}src${/}main.py    print("hello world")\n
  ${r_add}=    Run Process    git    add    .    ...
  ${r_commit}=    Run Process    git    commit    ...
  ${branch_result}=    Run Process    git    rev-parse    ...
  ${branch}=    Strip String    ${branch_result.stdout}
  Log    Detected branch: ${branch}
  # (lines 40-48 are blank — no further steps)

The test creates a git repo and detects the branch name, then does nothing. Steps 2-6 (actor YAML compilation, actor registration, action creation, project setup, plan lifecycle) are completely absent.

Code Location

File: robot/e2e/m2_acceptance.robot
Lines 18-48: Test case body — only git repo setup steps, 8+ blank lines where implementation should be

Impact

The M2 acceptance test provides no coverage of the M2 milestone's core acceptance criteria (actor compiler, LangGraph integration, custom actor plan execution). The tdd_expected_fail tag masks this gap.

Definition of Done

The test body should implement all steps documented in the test's own [Documentation] block:

Actor YAML compilation via actor add --config actor.yaml
Resource registration and project creation
Action creation referencing the custom actor
Full plan lifecycle: plan use → plan execute (strategize) → plan execute (execute) → plan diff → plan apply
Verification that the custom actor was used during execution

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: E2E Workflow Specification Tests — M2 Acceptance **Milestone**: v3.6.0 (M7) — E2E workflow specification tests **Severity**: Priority/Backlog — test infrastructure gap, not blocking runtime ### What Was Tested Code-level analysis of `robot/e2e/m2_acceptance.robot` against the M2 milestone acceptance criteria. ### Expected Behavior (from spec) The M2 acceptance test should exercise the complete actor compiler and LLM integration flow: 1. Create temp git repo with sample project files 2. Compile actor YAML into a LangGraph graph 3. Register actor via `actor add --config actor.yaml` 4. Create action referencing the custom actor 5. Create resource and project 6. Run full plan lifecycle (use, execute strategize, execute, diff, apply) with custom actor The test file's own documentation states: > Exercises actor YAML compilation, registration via CLI, resource and project setup, action creation referencing a custom actor, and the full plan lifecycle (use, execute strategize, execute, diff, apply) with real LLM API keys. ### Actual Behavior The test case body contains only git repo creation and branch detection steps, then ends abruptly: ```robot M2 Full Actor Compiler And LLM Integration [Tags] E2E tdd_issue tdd_issue_4189 tdd_expected_fail Skip If No LLM Keys # ---- Step 1: Create temp git repo with sample project files ---- ${repo_dir}= Create Temp Git Repo m2-e2e-repo Create Directory ${repo_dir}${/}src Create File ${repo_dir}${/}src${/}main.py print("hello world")\n ${r_add}= Run Process git add . ... ${r_commit}= Run Process git commit ... ${branch_result}= Run Process git rev-parse ... ${branch}= Strip String ${branch_result.stdout} Log Detected branch: ${branch} # (lines 40-48 are blank — no further steps) ``` The test creates a git repo and detects the branch name, then does nothing. Steps 2-6 (actor YAML compilation, actor registration, action creation, project setup, plan lifecycle) are completely absent. ### Code Location - File: `robot/e2e/m2_acceptance.robot` - Lines 18-48: Test case body — only git repo setup steps, 8+ blank lines where implementation should be ### Impact The M2 acceptance test provides no coverage of the M2 milestone's core acceptance criteria (actor compiler, LangGraph integration, custom actor plan execution). The `tdd_expected_fail` tag masks this gap. ### Definition of Done The test body should implement all steps documented in the test's own `[Documentation]` block: - Actor YAML compilation via `actor add --config actor.yaml` - Resource registration and project creation - Action creation referencing the custom actor - Full plan lifecycle: `plan use` → `plan execute` (strategize) → `plan execute` (execute) → `plan diff` → `plan apply` - Verification that the custom actor was used during execution --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 13:27:58 +00:00

HAL9000 added a new dependency

2026-04-09 13:28:54 +00:00

#5179 EPIC: E2E Workflow Specification Tests — Full Workflow Validation Suite (v3.6.0)

HAL9000 added

and removed

labels

2026-04-09 13:39:21 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 13:39:40 +00:00

HAL9000 referenced this issue

2026-04-09 13:43:41 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 30) #5908

HAL9000 commented

2026-04-09 13:43:57 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 referenced this issue

2026-04-09 13:56:52 +00:00

[AUTO-EPIC] Epic Planning Health Report (Cycle 21) #5883

HAL9000 added

and removed

labels

2026-04-09 13:59:02 +00:00

HAL9000 referenced this issue