Fix integration tests CLI exit code failures #4188

Open
opened 2026-04-06 22:51:22 +00:00 by freemo · 0 comments
Owner

Problem

Multiple integration robot tests are failing because CLI commands are returning exit code 1 instead of expected 0.

Affected Test Areas

  • A2A Version Negotiation - Commands exiting with 1 instead of 0
  • Actor Config CLI - ValueError: Config file not found when testing missing config scenarios
  • Actor Add Rich Output - Multiple CLI invocations returning exit code 1
  • Container Tooling - ToolRunner expecting failure but getting success
  • Workflow/Project Suites - M3, M4, M6, plan diff/artifacts, project context all failing with exit code mismatches

Root Cause

The CLI commands are likely encountering real errors (missing configs, service unavailability) that need to be either:

  1. Fixed in the implementation
  2. Have the tests updated to expect the correct exit codes
  3. Have proper test setup/teardown to ensure required resources exist

Acceptance Criteria

  • All integration robot tests pass without @tdd_expected_fail tags
  • CLI commands return appropriate exit codes for success/failure scenarios
  • Test setup ensures required resources (configs, services) are available
  • PR #4175 - Restore CI quality tests to passing state
  • Epic #13 - Restore All Quality Tests to Passing State
## Problem Multiple integration robot tests are failing because CLI commands are returning exit code 1 instead of expected 0. ## Affected Test Areas - **A2A Version Negotiation** - Commands exiting with 1 instead of 0 - **Actor Config CLI** - `ValueError: Config file not found` when testing missing config scenarios - **Actor Add Rich Output** - Multiple CLI invocations returning exit code 1 - **Container Tooling** - ToolRunner expecting failure but getting success - **Workflow/Project Suites** - M3, M4, M6, plan diff/artifacts, project context all failing with exit code mismatches ## Root Cause The CLI commands are likely encountering real errors (missing configs, service unavailability) that need to be either: 1. Fixed in the implementation 2. Have the tests updated to expect the correct exit codes 3. Have proper test setup/teardown to ensure required resources exist ## Acceptance Criteria - [ ] All integration robot tests pass without @tdd_expected_fail tags - [ ] CLI commands return appropriate exit codes for success/failure scenarios - [ ] Test setup ensures required resources (configs, services) are available ## Related To - PR #4175 - Restore CI quality tests to passing state - Epic #13 - Restore All Quality Tests to Passing State
HAL9000 self-assigned this 2026-04-08 12:12:44 +00:00
HAL9000 added this to the v3.5.0 milestone 2026-04-08 12:12:44 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#4188
No description provided.