cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

fix(plan): wire sandbox_root into plan execute pipeline #10096

Closed
brent.edwards wants to merge 4 commits from bugfix/m1-plan-execute-sandbox-root into test/restore-e2e-tests
pull from: bugfix/m1-plan-execute-sandbox-root
merge into: cleveragents:test/restore-e2e-tests
Conversation 13 Commits 4 Files Changed 29
+7576 -3297

4 Commits

Author SHA1 Message Date
brent.edwards d4cbbaa41f fix(e2e): use Extract JSON From Stdout for WF05 decision tree parsing 
WF05 was manually parsing the plan tree JSON output without unwrapping
the CLI envelope, causing the decision_id count to return 0. The envelope
wraps the decision array in a 'data' key.

Updated WF05 to use Extract JSON From Stdout which automatically unwraps
the envelope, ensuring the decision walker operates on the actual decision
array instead of the envelope structure.

Also added a documentation note to Extract JSON From Stdout explaining
which tests need refactoring to use this keyword instead of custom JSON
extraction logic.
 2026-04-26 17:41:39 +00:00
brent.edwards 8bce7f45d0 fix(e2e): restore M5/WF14 tests broken by JSON envelope and stale config 
Two independent root causes were causing 10 non-quota E2E failures.

**M5 acceptance (9 tests + 1 cascade):**
`format_output(..., "json")` wraps all CLI JSON output in the
spec-required envelope `{"command":…,"status":"ok","exit_code":0,
"data":{…},…}`.  The M5 tests were looking for payload fields
(`total_tokens`, `resolved_view`, `acms_config`, `tier_metrics`,
`plan_id`) at the top level of the extracted JSON object, but they
live inside `data`.

Fix: update `Extract JSON From Stdout` in `common_e2e.resource` to
auto-unwrap the envelope when both `exit_code` and `data` keys are
present, so callers receive the payload dict transparently.  Output
from helper scripts that do not use the CLI envelope (e.g. the WF04
snapshot helper) is returned unchanged.

Also includes the already-staged m5_acceptance.robot rework that fixed
the argument-passing bug (args were previously joined into single
space-containing strings instead of passed individually to Run CLI),
removed stale `tdd_expected_fail` tags, and standardised indentation.

**WF14 server mode (1 test):**
`~/.cleveragents/config.toml` is shared across all parallel pabot
workers and across nox sessions (the `CLEVERAGENTS_HOME` env var
isolates the SQLite DB but not the config file path, which is
hardcoded to `Path.home() / ".cleveragents"`).  A previous
`server_stubs.robot` run wrote `server.url = "https://stub.example.com"`
to that file.  The `config set` CLI command could not overwrite the
entry because tomllib parses TOML dotted keys as nested dicts
(`{"server": {"url": "…"}}`), causing `config set` to write a new
flat key alongside the original nested entry rather than replacing it.

Fix: add `WF14 Clean Global Config` — a keyword that uses
Python/tomlkit directly to remove `server.url`, `server.token`, and
`server.namespace` from the global config TOML, handling both nested
(`[server]` table) and flat quoted-key representations.  The keyword
is called from both `WF14 Suite Setup` (so any stale value is removed
before the test's `config set` runs) and `WF14 Suite Teardown` (so
subsequent runs start clean).  Failures are silently ignored — a
missing config file is normal on a fresh environment.

ISSUES CLOSED: #8459
 2026-04-26 17:41:39 +00:00
brent.edwards 84f5d6eb6b fix(plan): wire sandbox_root into plan execute pipeline 
Fixes three sub-bugs that caused LLM-generated files to be silently lost
during plan execute:

Bug 1 — Duplicate sandbox branch on second execute call:
_create_sandbox_for_plan() was called before phase detection, causing a
fatal duplicate-branch error when plan execute was invoked a second time on
a plan already in Execute/complete.  Moved sandbox creation inside the
Execute/QUEUED block so repeated invocations skip it safely.

Bug 2 — Changeset entries discarded after execute:
The lightweight ChangeSet from LLMExecuteActor was never persisted.  Now
serialised to plan.error_details["changeset_entries_json"] by
PlanExecutor._run_execute_with_stub(); PlanApplyService._resolve_changeset()
reconstructs a SpecChangeSet from this metadata when no changeset_store is
wired, so plan diff shows the generated file changes.

Bug 3 — Wrong output format for plan execute --format plain:
execute_plan() used the now-removed _execute_output_dict (envelope format)
instead of _plan_spec_dict, inconsistent with plan use / plan apply.
Switched to _plan_spec_dict; removed the dead _execute_output_dict function.

Also removes tdd_expected_fail from the M1 acceptance E2E test and adds 6
new BDD regression scenarios (@tdd_issue_1313) covering all three fixes.

ISSUES CLOSED: #1313
 2026-04-26 17:41:13 +00:00
brent.edwards d81b5853a8 fix(e2e): restore M5/WF14 tests broken by JSON envelope and stale config 
Two independent root causes were causing 10 non-quota E2E failures.

**M5 acceptance (9 tests + 1 cascade):**
`format_output(..., "json")` wraps all CLI JSON output in the
spec-required envelope `{"command":…,"status":"ok","exit_code":0,
"data":{…},…}`.  The M5 tests were looking for payload fields
(`total_tokens`, `resolved_view`, `acms_config`, `tier_metrics`,
`plan_id`) at the top level of the extracted JSON object, but they
live inside `data`.

Fix: update `Extract JSON From Stdout` in `common_e2e.resource` to
auto-unwrap the envelope when both `exit_code` and `data` keys are
present, so callers receive the payload dict transparently.  Output
from helper scripts that do not use the CLI envelope (e.g. the WF04
snapshot helper) is returned unchanged.

Also includes the already-staged m5_acceptance.robot rework that fixed
the argument-passing bug (args were previously joined into single
space-containing strings instead of passed individually to Run CLI),
removed stale `tdd_expected_fail` tags, and standardised indentation.

**WF14 server mode (1 test):**
`~/.cleveragents/config.toml` is shared across all parallel pabot
workers and across nox sessions (the `CLEVERAGENTS_HOME` env var
isolates the SQLite DB but not the config file path, which is
hardcoded to `Path.home() / ".cleveragents"`).  A previous
`server_stubs.robot` run wrote `server.url = "https://stub.example.com"`
to that file.  The `config set` CLI command could not overwrite the
entry because tomllib parses TOML dotted keys as nested dicts
(`{"server": {"url": "…"}}`), causing `config set` to write a new
flat key alongside the original nested entry rather than replacing it.

Fix: add `WF14 Clean Global Config` — a keyword that uses
Python/tomlkit directly to remove `server.url`, `server.token`, and
`server.namespace` from the global config TOML, handling both nested
(`[server]` table) and flat quoted-key representations.  The keyword
is called from both `WF14 Suite Setup` (so any stale value is removed
before the test's `config set` runs) and `WF14 Suite Teardown` (so
subsequent runs start clean).  Failures are silently ignored — a
missing config file is normal on a fresh environment.

ISSUES CLOSED: #8459
 2026-04-26 17:40:43 +00:00