test(e2e): implement E2E workflow tests for cost/budget enforcement and plugin architecture #8922

Open
opened 2026-04-14 04:03:34 +00:00 by HAL9000 · 3 comments
Owner

Background and Context

Epic #5179 (E2E Workflow Specification Tests — Full Workflow Validation Suite) has implemented E2E tests for project/plan lifecycle (#5259) and A2A local facade (#5260). The final deliverable for this Epic is E2E tests covering the v3.6.0-specific features: cost/budget enforcement and plugin architecture.

These E2E tests validate that the full system — from CLI invocation through to budget enforcement and plugin execution — works correctly as an integrated whole. They are distinct from unit and integration tests in that they exercise the complete application stack.

This issue blocks Epic #5179.

Acceptance Criteria

  • E2E test: plan execution halts when session budget is exceeded, with a clear user-facing error
  • E2E test: agents plan status shows current cost and budget utilization
  • E2E test: agents plugin activate <name> followed by a plan execution that uses the plugin
  • E2E test: agents plugin deactivate <name> prevents the plugin from being used in subsequent plan execution
  • E2E test: safety profile enforcement blocks a plan step that violates the configured safety rules
  • All E2E tests pass in CI with the full application stack

Subtasks

  • Write E2E test: budget exhaustion halts plan execution (Robot Framework or pytest)
  • Write E2E test: agents plan status cost/budget display
  • Write E2E test: plugin activation and use in plan execution
  • Write E2E test: plugin deactivation prevents use in plan execution
  • Write E2E test: safety profile enforcement blocks violating plan step
  • Ensure tests are tagged for the e2e test suite in CI

Definition of Done

  • All acceptance criteria met
  • Tests written and passing (coverage >= 97%)
  • Code reviewed and approved
  • Documentation updated if needed
  • No regressions introduced

Metadata

  • Commit message: test(e2e): implement E2E workflow tests for cost/budget enforcement and plugin architecture
  • Branch name: test/e2e-budget-plugin-workflows

Automated by CleverAgents Bot
Agent: new-issue-creator

## Background and Context Epic #5179 (E2E Workflow Specification Tests — Full Workflow Validation Suite) has implemented E2E tests for project/plan lifecycle (#5259) and A2A local facade (#5260). The final deliverable for this Epic is E2E tests covering the v3.6.0-specific features: cost/budget enforcement and plugin architecture. These E2E tests validate that the full system — from CLI invocation through to budget enforcement and plugin execution — works correctly as an integrated whole. They are distinct from unit and integration tests in that they exercise the complete application stack. This issue blocks Epic #5179. ## Acceptance Criteria - [ ] E2E test: plan execution halts when session budget is exceeded, with a clear user-facing error - [ ] E2E test: `agents plan status` shows current cost and budget utilization - [ ] E2E test: `agents plugin activate <name>` followed by a plan execution that uses the plugin - [ ] E2E test: `agents plugin deactivate <name>` prevents the plugin from being used in subsequent plan execution - [ ] E2E test: safety profile enforcement blocks a plan step that violates the configured safety rules - [ ] All E2E tests pass in CI with the full application stack ## Subtasks - [ ] Write E2E test: budget exhaustion halts plan execution (Robot Framework or pytest) - [ ] Write E2E test: `agents plan status` cost/budget display - [ ] Write E2E test: plugin activation and use in plan execution - [ ] Write E2E test: plugin deactivation prevents use in plan execution - [ ] Write E2E test: safety profile enforcement blocks violating plan step - [ ] Ensure tests are tagged for the `e2e` test suite in CI ## Definition of Done - [ ] All acceptance criteria met - [ ] Tests written and passing (coverage >= 97%) - [ ] Code reviewed and approved - [ ] Documentation updated if needed - [ ] No regressions introduced ## Metadata - **Commit message:** `test(e2e): implement E2E workflow tests for cost/budget enforcement and plugin architecture` - **Branch name:** `test/e2e-budget-plugin-workflows` --- **Automated by CleverAgents Bot** Agent: new-issue-creator
HAL9000 added this to the v3.6.0 milestone 2026-04-14 04:06:39 +00:00
Author
Owner

Triage Decision [AUTO-OWNR-4]

Verified

E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'.

  • Type: Testing
  • MoSCoW: Should Have — E2E test coverage for advanced features
  • Priority: Medium
  • Milestone: v3.6.0

Automated by CleverAgents Bot
Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor

## Triage Decision [AUTO-OWNR-4] **Verified** ✅ E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'. - **Type:** Testing - **MoSCoW:** Should Have — E2E test coverage for advanced features - **Priority:** Medium - **Milestone:** v3.6.0 --- **Automated by CleverAgents Bot** Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor
Author
Owner

Triage Decision [AUTO-OWNR-4]

Verified

E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'.

  • Type: Testing
  • MoSCoW: Should Have — E2E test coverage for advanced features
  • Priority: Medium
  • Milestone: v3.6.0

Automated by CleverAgents Bot
Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor

## Triage Decision [AUTO-OWNR-4] **Verified** ✅ E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'. - **Type:** Testing - **MoSCoW:** Should Have — E2E test coverage for advanced features - **Priority:** Medium - **Milestone:** v3.6.0 --- **Automated by CleverAgents Bot** Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor
Author
Owner

Triage Decision [AUTO-OWNR-4]

Verified

E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'.

  • Type: Testing
  • MoSCoW: Should Have — E2E test coverage for advanced features
  • Priority: Medium
  • Milestone: v3.6.0

Automated by CleverAgents Bot
Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor

## Triage Decision [AUTO-OWNR-4] **Verified** ✅ E2E workflow tests for cost/budget enforcement and plugin architecture are part of v3.6.0 scope: 'E2E workflow specification tests' and 'Cost/session budgets, safety profiles'. - **Type:** Testing - **MoSCoW:** Should Have — E2E test coverage for advanced features - **Priority:** Medium - **Milestone:** v3.6.0 --- **Automated by CleverAgents Bot** Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#8922
No description provided.