- Add NO_COLOR=1 to E2E Suite Setup and Test Environment to prevent
Rich from injecting ANSI escape codes into JSON output, which breaks
Extract JSON From Stdout and JSON assertions in M5/WF14 acceptance tests.
- Add reset_global_state() calls to all M1/M2/M4/M5 E2E verification
helpers to clear Settings singleton, DI container, provider registry,
and SQLAlchemy engine cache between parallel pabot worker runs.
- Propagate NO_COLOR=1 at suite-setup level in E2E and integration
test resource files so all Run Process subprocesses inherit clean
terminal output mode.
Root causes:
1. Missing NO_COLOR=1 caused Rich ANSI codes in CLI JSON output,
making json.loads() failures in robot Extract JSON From Stdout
2. Stale Settings singleton carried provider keys and DB paths between
pabot workers, causing UNIQUE constraint violations and stale config
lookups in context policy and server mode E2E tests.
Related: PR #9912
Three-pronged fix for intermittent pabot-parallel race condition in
M4 validation integration tests:
1. Composable Setup Database Isolation keyword in common.resource
gives each suite a unique CLEVERAGENTS_DATABASE_URL so concurrent
pabot workers never contend on the same SQLite file.
2. Per-suite CLEVERAGENTS_HOME directories prevent shared temp
directory cleanup from racing between workers.
3. Centralised reset_global_state() in robot/helpers_common.py clears
Settings singleton, DI container, provider registry, and engine
cache between chained CLI invocations in helper processes.
Also:
- Setup Test Environment now accepts optional mock_ai and
auto_apply_migrations arguments (default TRUE) for backward
compatibility while allowing suites to opt out.
- Added Suite Teardown to cli_plan_context_commands.robot.
- Fixed _COMMANDS typing in two helpers to eliminate type: ignore.
- Updated docs/development/testing.md to reflect helpers_common
delegation pattern.
- Added timeout=30s to all Run Process calls in
m4_e2e_verification.robot.
Fixes: #563
Three fixes targeting CI integration_tests failures (18 failures on e8aa5ac):
1. Merge duplicate *** Settings *** blocks in 14 robot files into single
blocks. Multiple Settings sections are non-standard RF practice and
may cause resource import failures in certain Robot Framework versions
or CI environments.
2. Replace bare 'python' with ${PYTHON} variable in all Run Process
calls (14 files). Noxfile now passes --variable PYTHON:<venv-path>
to robot so tests use the venv interpreter regardless of PATH. This
fixes '/usr/local/bin/python: No module named cleveragents' on CI.
3. Add comprehensive CI debug output in noxfile.py: file existence
checks for .resource files, PATH/Python resolution, fixture dir
checks, and RF version. This will diagnose any remaining resource
import issues.
Also: remove hardcoded '/app/src' sys.path.insert in
system_prompt_template_rendering.robot (not portable to CI), and
add trailing newline to common.resource.
All 204 tests pass locally (4 excluded: 2 slow, 2 discovery).
HAL9000
Brent E. Edwards
brent.edwards
freemo
