fix(test): resolve race condition in M4 validation integration test
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 25s
CI / build (pull_request) Successful in 26s
CI / typecheck (pull_request) Successful in 1m9s
CI / security (pull_request) Successful in 1m18s
CI / integration_tests (pull_request) Successful in 3m43s
CI / unit_tests (pull_request) Successful in 4m8s
CI / docker (pull_request) Successful in 43s
CI / coverage (pull_request) Successful in 5m35s
CI / benchmark-regression (pull_request) Successful in 32m44s

Three-pronged fix for intermittent pabot-parallel race condition in
M4 validation integration tests:

1. Composable Setup Database Isolation keyword in common.resource
   gives each suite a unique CLEVERAGENTS_DATABASE_URL so concurrent
   pabot workers never contend on the same SQLite file.

2. Per-suite CLEVERAGENTS_HOME directories prevent shared temp
   directory cleanup from racing between workers.

3. Centralised reset_global_state() in robot/helpers_common.py clears
   Settings singleton, DI container, provider registry, and engine
   cache between chained CLI invocations in helper processes.

Also:
- Setup Test Environment now accepts optional mock_ai and
  auto_apply_migrations arguments (default TRUE) for backward
  compatibility while allowing suites to opt out.
- Added Suite Teardown to cli_plan_context_commands.robot.
- Fixed _COMMANDS typing in two helpers to eliminate type: ignore.
- Updated docs/development/testing.md to reflect helpers_common
  delegation pattern.
- Added timeout=30s to all Run Process calls in
  m4_e2e_verification.robot.

Fixes: #563
This commit is contained in:
Brent E. Edwards
2026-03-09 22:28:42 +00:00
committed by Brent E. Edwards
parent 876217d0ca
commit d88dad94f6
11 changed files with 238 additions and 25 deletions
+7
View File
@@ -20,6 +20,13 @@
- Polymorphic handler resolution with ancestor-type fallback
- CLI: `agents resource type list` shows Inherits column; `type show` displays inheritance chain
- Alembic migration `m6_004_resource_type_inherits` adds `inherits` column to `resource_types`
- Fixed intermittent race condition in M4 validation integration tests when
running under pabot. Root cause was three-pronged: shared SQLite DB URL,
shared CLEVERAGENTS_HOME directory, and singleton leaks in chained CLI
helper invocations. Introduced composable `Setup Database Isolation`
keyword in `common.resource`, per-suite temp directories, and centralised
`reset_global_state()` in `robot/helpers_common.py`. Added `timeout=30s`
to all `Run Process` calls in `m4_e2e_verification.robot`. (#563)
- Fixed `agents project show` not finding a project immediately after creation.
Extended the `session.commit()` fix from #589 to also cover `update()` and
`delete()` in `NamespacedProjectRepository`, and updated the class docstring
+52
View File
@@ -191,6 +191,58 @@ robot/
- **Timeouts**: Always add `timeout=` to `Run Process` calls that invoke long-running commands.
- **Slow tests**: Tag tests that require external services (API keys, running servers) with `slow`. These are excluded in CI via `--exclude slow`.
### Parallel Execution Isolation (pabot)
Robot integration tests run in parallel via `pabot` (CPU-count workers by default).
To prevent race conditions, `common.resource` provides per-suite isolation:
1. **Per-suite CLEVERAGENTS_HOME** — Each suite gets a unique temp directory
(`${TEMPDIR}/.cleveragents_<SuiteName>`) so configuration and data files
never collide.
2. **Per-suite database URL**`CLEVERAGENTS_DATABASE_URL` and
`CLEVERAGENTS_TEST_DATABASE_URL` point to SQLite files inside the
per-suite home directory. Without this, all pabot workers would fall
back to the default `sqlite:///cleveragents.db` (relative to CWD),
causing write contention.
3. **Singleton reset in helpers** — Robot helpers that chain multiple CLI
invocations within a single process (e.g. `full_flow()`) must call
`reset_global_state()` between invocations. The reset logic is
centralised in `robot/helpers_common.py`; each helper imports and
delegates to `helpers_common.reset_global_state()`. This function
resets:
- `Settings._instance` (singleton config)
- `reset_container()` (DI container)
- `reset_provider_registry()` (provider registry)
- `MEMORY_ENGINES` (SQLAlchemy engine cache)
4. **Environment variable cleanup**`Cleanup Test Environment` removes
all environment variables set during setup (`CLEVERAGENTS_HOME`,
`CLEVERAGENTS_AUTO_APPLY_MIGRATIONS`, `CLEVERAGENTS_TESTING_USE_MOCK_AI`,
`CLEVERAGENTS_DATABASE_URL`, and `CLEVERAGENTS_TEST_DATABASE_URL`) after
each suite to prevent leakage into subsequent suites.
#### Adding a new Robot helper
When writing a new Robot helper that invokes `CliRunner` or otherwise
exercises the application layer, import the shared reset function from
`helpers_common.py`:
```python
from helpers_common import reset_global_state
# In the __main__ dispatcher:
if __name__ == "__main__":
reset_global_state()
fn = _COMMANDS[sys.argv[1]]
fn()
```
Call `reset_global_state()` once at the start of the `__main__` block and
again between chained CLI invocations within the same process. Do **not**
redefine the reset logic locally — always delegate to `helpers_common`.
### Running Specific Suites
```bash
+2 -1
View File
@@ -5,7 +5,8 @@ Resource ${CURDIR}/common.resource
Library OperatingSystem
Library Process
Library String
Suite Setup Run Keywords Setup Test Environment AND Set Environment Variable CLEVERAGENTS_AUTO_APPLY_MIGRATIONS true
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
Test Timeout 300 seconds
*** Variables ***
+73 -4
View File
@@ -1,5 +1,10 @@
*** Settings ***
Documentation Common resources and keywords for Robot Framework tests
Documentation Common resources and keywords for Robot Framework tests.
...
... Provides per-suite and per-test isolation for parallel execution
... via pabot. Each suite receives a unique CLEVERAGENTS_HOME and
... CLEVERAGENTS_DATABASE_URL so that concurrent workers never
... contend on the same SQLite file or on shared global state.
Library OperatingSystem
Library String
Library Collections
@@ -14,22 +19,86 @@ ${SRC_DIR} ${WORKSPACE}/src/cleveragents
*** Keywords ***
Setup Test Environment
[Documentation] Setup common test environment
[Documentation] Setup common test environment with per-suite isolation.
...
... Creates a unique CLEVERAGENTS_HOME directory so that
... pabot workers never collide on configuration files.
... Does NOT set CLEVERAGENTS_DATABASE_URL — suites that
... run helper scripts via ``Run Process`` should also call
... ``Setup Database Isolation`` (or ``Setup Test Environment
... With Database Isolation``) so that concurrent workers
... never contend on the same SQLite file.
...
... Optional arguments:
... - ``auto_apply_migrations``: Set CLEVERAGENTS_AUTO_APPLY_MIGRATIONS
... (default: ``${TRUE}``).
... - ``mock_ai``: Set CLEVERAGENTS_TESTING_USE_MOCK_AI
... (default: ``${TRUE}``). Pass ``${FALSE}`` in suites
... that intentionally need real AI responses.
[Arguments] ${auto_apply_migrations}=${TRUE} ${mock_ai}=${TRUE}
Log Setting up test environment
${safe_suite}= Replace String ${SUITE NAME} ${SPACE} _
${home}= Set Variable ${TEMPDIR}${/}.cleveragents_${safe_suite}
Run Keyword And Ignore Error Remove Directory ${home} recursive=True
Create Directory ${home}
Set Environment Variable CLEVERAGENTS_HOME ${home}
Run Keyword And Ignore Error Remove Directory ${TEMPDIR}${/}.cleveragents recursive=True
Set Suite Variable ${SUITE_HOME} ${home}
# Prevent migration prompts in helper sub-processes (opt-in via argument)
Run Keyword If ${auto_apply_migrations}
... Set Environment Variable CLEVERAGENTS_AUTO_APPLY_MIGRATIONS true
# Enable mock AI in helper sub-processes (opt-in via argument)
Run Keyword If ${mock_ai}
... Set Environment Variable CLEVERAGENTS_TESTING_USE_MOCK_AI true
# Get the actual Python executable being used
${python_exec}= Evaluate sys.executable sys
Set Suite Variable ${PYTHON} ${python_exec}
# Don't set AGENTS_EXECUTABLE as a single variable - we'll use ${PYTHON} -m cleveragents directly
Setup Database Isolation
[Documentation] Set per-suite unique database URLs for helper-based suites.
...
... Call this keyword AFTER ``Setup Test Environment`` in
... suites that invoke Python helper scripts via ``Run
... Process``. Without this, helper processes fall back to
... the default ``sqlite:///cleveragents.db`` (relative to
... CWD), which causes concurrent pabot workers to contend
... on the same file.
...
... Do NOT call this in suites that run the real
... ``cleveragents`` CLI as a subprocess (e.g.
... ``cli_plan_context_commands.robot``), because those
... commands determine their own database path at runtime.
${home}= Get Environment Variable CLEVERAGENTS_HOME
${db_path}= Set Variable ${home}${/}cleveragents_suite.db
Set Environment Variable CLEVERAGENTS_DATABASE_URL sqlite:///${db_path}
Set Suite Variable ${SUITE_DB_PATH} ${db_path}
${test_db_path}= Set Variable ${home}${/}cleveragents_suite_test.db
Set Environment Variable CLEVERAGENTS_TEST_DATABASE_URL sqlite:///${test_db_path}
Set Suite Variable ${SUITE_TEST_DB_PATH} ${test_db_path}
Setup Test Environment With Database Isolation
[Documentation] Convenience keyword: calls both ``Setup Test Environment``
... and ``Setup Database Isolation``. Use this in suites
... that execute Python helper scripts via ``Run Process``.
Setup Test Environment
Setup Database Isolation
Cleanup Test Environment
[Documentation] Clean up after tests
[Documentation] Clean up after tests.
...
... Removes the per-suite CLEVERAGENTS_HOME directory and
... any associated SQLite files. Uses the ``${SUITE_HOME}``
... suite variable set during setup rather than re-reading
... the environment variable, which could be stale if a test
... modified ``CLEVERAGENTS_HOME``.
Log Cleaning up test environment
Run Keyword And Ignore Error Remove Directory ${SUITE_HOME} recursive=True
# Clean env vars to avoid leaking into subsequent suites
Remove Environment Variable CLEVERAGENTS_HOME
Remove Environment Variable CLEVERAGENTS_AUTO_APPLY_MIGRATIONS
Remove Environment Variable CLEVERAGENTS_TESTING_USE_MOCK_AI
Remove Environment Variable CLEVERAGENTS_DATABASE_URL
Remove Environment Variable CLEVERAGENTS_TEST_DATABASE_URL
File Should Contain Pattern
[Arguments] ${file_path} ${pattern}
+6 -2
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import os
import sys
import tempfile
from collections.abc import Callable
from pathlib import Path
from unittest.mock import MagicMock, patch
@@ -17,6 +18,7 @@ _SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from helpers_common import reset_global_state # noqa: E402
from typer.testing import CliRunner # noqa: E402
from cleveragents.cli.commands.invariant import app as invariant_app # noqa: E402
@@ -33,6 +35,7 @@ from cleveragents.domain.models.core.invariant import ( # noqa: E402
runner = CliRunner()
_PLAN_ULID = "01M3SM0KE00000000000000001"
_DECISION_ULID = "01M3DEC1S10N00000000000001"
_CORRECTION_ULID = "01M3C0RRECT10N000000000001"
@@ -316,7 +319,7 @@ def plan_correct_dry_run() -> None:
# Dispatcher
# ---------------------------------------------------------------------------
_COMMANDS: dict[str, object] = {
_COMMANDS: dict[str, Callable[[], None]] = {
"invariant-add-global": invariant_add_global,
"invariant-add-project": invariant_add_project,
"invariant-list": invariant_list,
@@ -331,5 +334,6 @@ if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
print(f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>")
sys.exit(1)
reset_global_state()
fn = _COMMANDS[sys.argv[1]]
fn() # type: ignore[operator]
fn()
+20 -5
View File
@@ -7,6 +7,7 @@ from __future__ import annotations
import json
import sys
from collections.abc import Callable
from datetime import UTC, datetime
from pathlib import Path
from unittest.mock import MagicMock, patch
@@ -16,6 +17,7 @@ _SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from helpers_common import reset_global_state # noqa: E402
from typer.testing import CliRunner # noqa: E402
from ulid import ULID # noqa: E402
@@ -41,6 +43,7 @@ from cleveragents.domain.models.core.plan import ( # noqa: E402
runner = CliRunner()
_PLAN_ULID = str(ULID())
_DECISION_ULID = str(ULID())
_CORRECTION_ID_1 = str(ULID())
@@ -380,8 +383,16 @@ def fixture_loading() -> None:
def full_flow() -> None:
"""End-to-end: correction + subplan status + failure handler."""
# Step 1: correction revert
"""End-to-end: correction + subplan status + failure handler.
This subcommand chains three logical steps inside a single process.
``reset_global_state()`` is called between steps to ensure the
Settings singleton, DI container, and engine cache do not carry
stale references from one invocation into the next. The initial
reset is handled by the ``__main__`` dispatcher before calling
this function.
"""
# Step 1: correction revert (no reset needed — __main__ already reset)
mock_svc = MagicMock()
mock_svc.get_plan.return_value = _mock_plan()
mock_correction = _mock_correction_service()
@@ -419,6 +430,7 @@ def full_flow() -> None:
sys.exit(1)
# Step 2: subplan status
reset_global_state()
config = SubplanConfig(
execution_mode=ExecutionMode.SEQUENTIAL,
merge_strategy=SubplanMergeStrategy.GIT_THREE_WAY,
@@ -443,7 +455,7 @@ def full_flow() -> None:
print(f"FAIL step 2: exit={r2.exit_code} output={r2.output}")
sys.exit(1)
# Step 3: failure handler
# Step 3: failure handler (pure domain logic — no CLI / DB)
handler = SubplanFailureHandler()
config_ff = SubplanConfig(fail_fast=True)
failed = SubplanStatus(
@@ -461,7 +473,7 @@ def full_flow() -> None:
# Dispatcher
# ---------------------------------------------------------------------------
_COMMANDS: dict[str, object] = {
_COMMANDS: dict[str, Callable[[], None]] = {
"correction-revert": correction_revert,
"correction-append": correction_append,
"correction-dry-run": correction_dry_run,
@@ -476,5 +488,8 @@ if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
print(f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>")
sys.exit(1)
# Reset singletons once at process entry; full_flow() resets again
# between each chained CLI invocation.
reset_global_state()
fn = _COMMANDS[sys.argv[1]]
fn() # type: ignore[operator]
fn()
+3
View File
@@ -34,6 +34,7 @@ _SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from helpers_common import reset_global_state # noqa: E402
from typer.testing import CliRunner # noqa: E402
from cleveragents.cli.commands.plan import app as plan_app # noqa: E402
@@ -64,6 +65,7 @@ from cleveragents.infrastructure.sandbox.merge import ( # noqa: E402
runner = CliRunner()
_ROOT_ULID = "01KHDE6WWS2171PWW3GJEBXZ8R"
_CHILD_A_ULID = "01KHDE6WWS2171PWW3GJEBXZ8A"
_CHILD_B_ULID = "01KHDE6WWS2171PWW3GJEBXZ8B"
@@ -979,6 +981,7 @@ def main() -> int:
if handler is None:
print(f"Unknown command: {command}")
return 1
reset_global_state()
handler()
return 0
+62
View File
@@ -0,0 +1,62 @@
"""Shared utilities for Robot Framework helper scripts.
This module centralises the ``reset_global_state()`` function used by
``helper_m4_e2e_verification.py``,
``helper_m4_correction_subplan_smoke.py``, and
``helper_m3_decision_validation_smoke.py``.
Any helper script invoked via ``Run Process`` from a ``.robot`` suite
can import this module to reset process-wide singletons between
chained CLI invocations.
"""
import contextlib
import sys
def reset_global_state() -> None:
"""Reset process-wide singletons between CLI invocations.
Clears the Settings singleton, DI container, provider registry,
and in-memory SQLAlchemy engine cache so that chained CLI
invocations within the same helper process do not carry stale
state.
Each import is guarded by ``contextlib.suppress(ImportError)``
because this module lives in ``robot/`` and may be loaded in
contexts where not all application modules are on the path
(e.g. a minimal helper that only exercises domain models).
This is an intentional exception to the top-of-file import rule
per CONTRIBUTING.md §Import Guidelines.
"""
# Settings singleton
with contextlib.suppress(ImportError):
from cleveragents.config.settings import Settings
Settings._instance = None
# DI container singleton
with contextlib.suppress(ImportError):
from cleveragents.application.container import reset_container
reset_container()
# Provider registry singleton
with contextlib.suppress(ImportError):
from cleveragents.providers.registry import reset_provider_registry
reset_provider_registry()
# In-memory SQLAlchemy engine cache
with contextlib.suppress(ImportError):
from cleveragents.infrastructure.database.engine_cache import MEMORY_ENGINES
for _url, engine in list(MEMORY_ENGINES.items()):
try:
engine.dispose()
except Exception as exc:
print(
f"[helpers_common] engine.dispose() suppressed: {exc}",
file=sys.stderr,
)
MEMORY_ENGINES.clear()
+1 -1
View File
@@ -1,7 +1,7 @@
*** Settings ***
Documentation M3 decision tree, validation gating, and invariant enforcement E2E smoke tests via CLI
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Setup Setup Test Environment With Database Isolation
Suite Teardown Cleanup Test Environment
*** Variables ***
+1 -1
View File
@@ -1,7 +1,7 @@
*** Settings ***
Documentation M4 correction + subplan lifecycle E2E smoke tests via CLI
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Setup Setup Test Environment With Database Isolation
Suite Teardown Cleanup Test Environment
*** Variables ***
+11 -11
View File
@@ -3,7 +3,7 @@ Documentation M4 end-to-end verification: subplan spawning, parallel executio
... plan tree viewing, three-way merge, conflict surfacing, and
... parent plan subplan status tracking.
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Setup Setup Test Environment With Database Isolation
Suite Teardown Cleanup Test Environment
*** Variables ***
@@ -14,7 +14,7 @@ Plan Spawns Multiple Subplans During Execute
[Documentation] Execute a parent plan that spawns multiple subplans.
... Verifies subplans are created with correct parent_plan_id
... and that the parent's subplan_statuses list is populated.
${result}= Run Process ${PYTHON} ${HELPER} spawn-subplans cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} spawn-subplans cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -24,7 +24,7 @@ View Subplan Tree Via Plan Tree
[Documentation] View the subplan tree for a parent plan.
... Verifies parent-child hierarchy is correctly represented
... including depth, root_plan_id, and subplan_statuses.
${result}= Run Process ${PYTHON} ${HELPER} plan-tree cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} plan-tree cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -34,7 +34,7 @@ Verify Merged Results Via Plan Diff
[Documentation] View merged results via agents plan diff.
... Verifies the diff service returns unified diff output
... after subplan execution and merging.
${result}= Run Process ${PYTHON} ${HELPER} plan-diff cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} plan-diff cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -44,7 +44,7 @@ Parallel Subplan Execution With Max Parallel
[Documentation] Verify parallel subplan execution respects max_parallel.
... Creates a SubplanConfig with PARALLEL mode and max_parallel=3
... and verifies concurrent scheduling constraints.
${result}= Run Process ${PYTHON} ${HELPER} parallel-max cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} parallel-max cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -53,7 +53,7 @@ Parallel Subplan Execution With Max Parallel
Three Way Merge Combines Non Conflicting Changes
[Documentation] Verify three-way merge combines non-conflicting changes
... from parallel subplans using GitMergeStrategy.
${result}= Run Process ${PYTHON} ${HELPER} merge-clean cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} merge-clean cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -62,7 +62,7 @@ Three Way Merge Combines Non Conflicting Changes
Merge Conflicts Are Surfaced Correctly
[Documentation] Verify merge conflicts from parallel subplans are
... detected and reported with conflict markers.
${result}= Run Process ${PYTHON} ${HELPER} merge-conflict cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} merge-conflict cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -72,7 +72,7 @@ Parent Plan Tracks All Subplan Statuses
[Documentation] Verify parent plan tracks all subplan statuses through
... lifecycle transitions including completion, failure,
... and retry attempts.
${result}= Run Process ${PYTHON} ${HELPER} parent-tracking cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} parent-tracking cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -82,7 +82,7 @@ CLI Plan Use Creates Plan With Subplan Config
[Documentation] Invoke agents plan use local/refactor-action local/monorepo
... via the actual CLI (Typer CliRunner) with a mocked lifecycle
... service and verify the plan is created with subplan config.
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-use cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-use cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -92,7 +92,7 @@ CLI Plan Execute Transitions With Subplans
[Documentation] Invoke agents plan execute <plan_id> via the actual CLI
... (Typer CliRunner) with a mocked lifecycle service and
... verify the plan transitions to Execute phase with subplans.
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-execute cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-execute cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
@@ -103,7 +103,7 @@ CLI Plan Tree Displays Subplan Hierarchy
... actual CLI (Typer CliRunner) with mocked Decision objects
... forming a subplan tree. Verify JSON output contains
... subplan_spawn and subplan_parallel_spawn decision types.
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-tree cwd=${WORKSPACE}
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-tree cwd=${WORKSPACE} timeout=30s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0