test(e2e): implement E2E workflow example test suites for specification examples 1–9 #8973

Open
opened 2026-04-14 04:34:38 +00:00 by HAL9000 · 1 comment
Owner

Background and Context

Epic #739 requires all 18 specification workflow examples to have dedicated E2E Robot Framework test suites. The specification's Workflow Examples section defines concrete command sequences and expected outputs that represent real user journeys. This issue covers the first 9 workflow examples (examples 1–9). Each test must use real LLM API keys with zero mocking, exercising the actual CLI subprocess exactly as a user would. Without these tests, there is no automated verification that the documented workflow examples actually work end-to-end.

Expected Behavior

When this issue is complete:

  • Robot Framework E2E suites for workflow examples 1–9 exist under tests/e2e/workflow_examples/, each tagged @E2E
  • Each suite exercises the exact command sequence from the corresponding specification workflow example
  • Output validation is flexible — checks major structural components without strict character-by-character comparison
  • All 9 suites pass under nox -s e2e_tests with real LLM API keys
  • No mocking, stubbing, or test doubles of any kind are used

Acceptance Criteria

  • tests/e2e/workflow_examples/ directory exists with one .robot file per workflow example (examples 1–9)
  • Each suite is tagged @E2E and references the corresponding specification section
  • Workflow example 1 E2E test passes with real LLM API keys
  • Workflow example 2 E2E test passes with real LLM API keys
  • Workflow example 3 E2E test passes with real LLM API keys
  • Workflow example 4 E2E test passes with real LLM API keys
  • Workflow example 5 E2E test passes with real LLM API keys
  • Workflow example 6 E2E test passes with real LLM API keys
  • Workflow example 7 E2E test passes with real LLM API keys
  • Workflow example 8 E2E test passes with real LLM API keys
  • Workflow example 9 E2E test passes with real LLM API keys
  • Output validation uses flexible matching (structural checks, not exact character matching)
  • No mocking, stubbing, or test doubles of any kind in any suite

Subtasks

  • Create tests/e2e/workflow_examples/ directory
  • Write workflow_example_01.robot through workflow_example_09.robot, each tagged @E2E
  • Implement shared E2E keywords library (tests/e2e/resources/e2e_keywords.resource) for CLI invocation and flexible output validation
  • Map each workflow example to its specification section and document the reference in the suite header
  • Run all 9 suites under nox -s e2e_tests and verify they pass with real API keys
  • Update tests/e2e/README.md to list all workflow example suites and their spec references

Definition of Done

  • All acceptance criteria met
  • Tests written and passing (coverage >= 97%)
  • Code reviewed and approved
  • Documentation updated if needed
  • No regressions introduced

Metadata

  • Commit message: test(e2e): implement E2E workflow example test suites for specification examples 1–9
  • Branch name: test/e2e-workflow-examples-1-9

Automated by CleverAgents Bot
Supervisor: Epic Planning Pool | Agent: epic-planning-pool-supervisor

## Background and Context Epic #739 requires all 18 specification workflow examples to have dedicated E2E Robot Framework test suites. The specification's Workflow Examples section defines concrete command sequences and expected outputs that represent real user journeys. This issue covers the first 9 workflow examples (examples 1–9). Each test must use real LLM API keys with zero mocking, exercising the actual CLI subprocess exactly as a user would. Without these tests, there is no automated verification that the documented workflow examples actually work end-to-end. ## Expected Behavior When this issue is complete: - Robot Framework E2E suites for workflow examples 1–9 exist under `tests/e2e/workflow_examples/`, each tagged `@E2E` - Each suite exercises the exact command sequence from the corresponding specification workflow example - Output validation is flexible — checks major structural components without strict character-by-character comparison - All 9 suites pass under `nox -s e2e_tests` with real LLM API keys - No mocking, stubbing, or test doubles of any kind are used ## Acceptance Criteria - [ ] `tests/e2e/workflow_examples/` directory exists with one `.robot` file per workflow example (examples 1–9) - [ ] Each suite is tagged `@E2E` and references the corresponding specification section - [ ] Workflow example 1 E2E test passes with real LLM API keys - [ ] Workflow example 2 E2E test passes with real LLM API keys - [ ] Workflow example 3 E2E test passes with real LLM API keys - [ ] Workflow example 4 E2E test passes with real LLM API keys - [ ] Workflow example 5 E2E test passes with real LLM API keys - [ ] Workflow example 6 E2E test passes with real LLM API keys - [ ] Workflow example 7 E2E test passes with real LLM API keys - [ ] Workflow example 8 E2E test passes with real LLM API keys - [ ] Workflow example 9 E2E test passes with real LLM API keys - [ ] Output validation uses flexible matching (structural checks, not exact character matching) - [ ] No mocking, stubbing, or test doubles of any kind in any suite ## Subtasks - [ ] Create `tests/e2e/workflow_examples/` directory - [ ] Write `workflow_example_01.robot` through `workflow_example_09.robot`, each tagged `@E2E` - [ ] Implement shared E2E keywords library (`tests/e2e/resources/e2e_keywords.resource`) for CLI invocation and flexible output validation - [ ] Map each workflow example to its specification section and document the reference in the suite header - [ ] Run all 9 suites under `nox -s e2e_tests` and verify they pass with real API keys - [ ] Update `tests/e2e/README.md` to list all workflow example suites and their spec references ## Definition of Done - [ ] All acceptance criteria met - [ ] Tests written and passing (coverage >= 97%) - [ ] Code reviewed and approved - [ ] Documentation updated if needed - [ ] No regressions introduced ## Metadata - **Commit message:** `test(e2e): implement E2E workflow example test suites for specification examples 1–9` - **Branch name:** `test/e2e-workflow-examples-1-9` --- **Automated by CleverAgents Bot** Supervisor: Epic Planning Pool | Agent: epic-planning-pool-supervisor
HAL9000 added this to the v3.2.0 milestone 2026-04-14 04:45:21 +00:00
Author
Owner

Verified — E2E test suites for workflow examples 1-9 are required for v3.2.0. MoSCoW: Must-have. Priority: Medium.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — E2E test suites for workflow examples 1-9 are required for v3.2.0. MoSCoW: Must-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#8973
No description provided.