CHANGELOG.md

@@ -53,125 +53,9 @@
 
 ## [Unreleased]
 
-- **`agents session tell` invokes real LLM orchestrator actor** (#5784): Replaced the
-  M3 echo-stub with real actor invocation via `SessionWorkflow`, routing through
-  `LangChainSessionCaller` → `ToolCallingRuntime.run_tool_loop()`.  The user prompt
-  is sent to the session's bound orchestrator actor (or `--actor` override), the
-  assistant's response is persisted via `SessionService.append_message()`, and token
-  usage is tracked via `SessionService.update_token_usage()`.  Output includes a
-  **Usage** panel (Rich/Plain) or `usage` object (JSON/YAML) with input tokens,
-  output tokens, estimated cost, and duration.  The `--stream` flag produces real
-  LLM streaming output.  A `SessionActorNotConfiguredError` is raised with exit
-  code 1 when no actor is configured.
-
-- Fixed `ReactiveEventBus.emit()` exception handler to log the full exception
-  message (`str(exc)`) and enable traceback forwarding (`exc_info=True`).
-  Previously the handler logged only the exception type name (e.g.
-  "ValueError") with no diagnostic detail, making production debugging
-  impossible. The handler now includes the error message text and full
-  traceback in the structlog warning entry. Removed `@tdd_expected_fail` tag
-  from the TDD test so both scenarios run as normal regression guards. (#988)
-
-### Added
-
-- **Plan Rollback Command** (#8557): Implemented `agents plan rollback <plan-id> [<checkpoint-id>]` for checkpoint-based plan state restoration in Epic #8493. The command restores a plan's sandbox to the state captured at a given checkpoint, discarding all decisions made after that checkpoint. The checkpoint can be specified as an optional positional second argument or via the `--to-checkpoint` named option. Supports `--yes/-y` flag to skip confirmation prompts and `--format/-f` for output format selection (rich/plain/json/yaml). Included with comprehensive BDD test coverage (>= 97%) and spec-aligned output formatting showing rollback summary, changes reverted, impact analysis, and post-rollback state panels.
-
 ### Fixed
-- **Global CLI options ``--data-dir``, ``--config-path``, and ``-v`` now work correctly**
-  (#6785): These spec-required flags were absent from ``main_callback()`` in
-  ``src/cleveragents/cli/main.py``, causing any invocation with these flags to crash
-  with ``NoSuchOption: No such option``. All three options are now implemented per
-  ADR-021 §Global CLI Flags and ADR-024 §Resolution Chain.  ``--data-dir`` sets
-  ``CLEVERAGENTS_DATA_DIR`` before any ``Settings``-reading code runs, ``--config-path``
-  sets ``CLEVERAGENTS_CONFIG_PATH`` for ``ConfigService`` to pick up, and ``-v``
-  (repeatable count) maps to the appropriate ``structlog`` log level.
 
-- **Actor configuration validation incorrectly requires top-level provider field** (#4300): 
-   Actor configuration in V3 is now obtained from the nested configuration 
-   parameter, according to the specification.
-   Removed the legacy V2 fallback support and the tests affected by that 
-   removal. Mocked existing steps to allow remaining V2 features to be 
-   covered/tested.
-
-- **TUI Prompt Symbol Mode Awareness** (#6431): The prompt widget now displays a
-  mode-dependent symbol (`❯` normal, `/` command, `$` shell, `☰` multi-line),
-  implemented via `_PromptSymbolMixin` and `InputMode.MULTILINE`. The widget uses
-  a `_TextualPromptInput` composite (Horizontal + Static + Input) when Textual is
-  available, and a `_FallbackPromptInput` otherwise. Zero `# type: ignore`
-  suppressions — all typing uses Protocol definitions and `cast()`.
-- **Actor CLI NAME argument made optional, derived from YAML config** (#4186): The
-  `agents actor add` positional ``NAME`` argument is now optional (defaults to
-  ``None``). When omitted, the actor name is derived from the ``name`` field in
-  the config file. Raises ``BadParameter`` if neither the argument nor the config
-  ``name`` field is provided. Updated docstring signature to
-  ``agents actor add [--config|-c <FILE>] [<NAME>]`` and added config-only usage
-  examples. Added Behave scenario for the ``BadParameter`` error path
-  (``actor add without NAME and without config name field raises BadParameter``)
-  in ``features/actor_add_name_positional.feature`` with corresponding step
-  definition. Updated step definitions in
-  ``features/steps/actor_add_update_enforcement_steps.py`` and
-  ``features/steps/actor_add_name_positional_steps.py`` to pass ``context.actor_name``
-  as a positional argument for compatibility.
-
-- **Improved parallel test suite isolation** (#4186): Replaced deprecated
-  ``tempfile.mktemp`` with ``tempfile.mkstemp`` in ``features/environment.py``
-  for atomic temp file creation, eliminating TOCTOU race conditions in the
-  per-scenario database path generation. Added ``fcntl.flock`` file locking to
-  ``_ensure_template_db()`` to prevent race conditions when multiple
-  ``behave-parallel`` workers attempt to create the template database
-  simultaneously.
-
-- **Removed stale @tdd_expected_fail tags from actor add enforcement tests**: The
-  ``--update`` enforcement feature (#2609) was already implemented and merged but
-  residual ``@tdd_expected_fail`` tags remained on its BDD scenarios. These tags
-  were cleaned up in ``features/actor_add_update_enforcement.feature`` so the
-  tests report correctly now that the underlying bug has been fixed.
-
-- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
-  step texts to avoid case-sensitive collisions between different step modules that
-  prevented all Behave tests from loading. Renamed steps in
-  ``edge_case_plan_steps.py``, ``plan_executor_coverage_boost_steps.py``,
-  ``plan_explain_steps.py``, ``plan_model_steps.py``, ``project_repository_steps.py``,
-  ``service_retry_wiring_steps.py``, and ``session_model_steps.py``.
-  Additionally resolved a collision between ``acms_index_data_model_traversal_steps.py``
-  and ``security_audit_steps.py`` for ``Then the count should be``, and fixed
-  ``pr_compliance_checklist_steps.py`` project-root resolution (``parents[3]`` →
-  ``parents[2]``). Fixed table column-header mismatches in
-  ``features/acms/index_data_model_and_traversal.feature`` and guarded
-  ``cli_init_yes_flag_steps.py`` cleanup against ``None`` temp_dir. Annotated
-  ``features/architecture.feature`` ``@tdd_expected_fail`` for pre-existing Pydantic
-  compliance debt in ``IndexEntry`` / ``ACMSIndex`` classes.
-
-- **Cross-actor subgraph cycle detection reads actor_ref field** (#1431): Fixed
-  `_detect_subgraph_cycles()`, `_map_node()`, and the `compile_actor()` main loop
-  in `src/cleveragents/actor/compiler.py` to read `actor_ref` from the top-level
-  `NodeDefinition.actor_ref` field instead of `node.config.get("actor_ref", "")`.
-  Because `actor_ref` is a typed, validated Pydantic field (not a key inside the
-  untyped `config` dict), the old code always returned an empty string, causing
-  cross-actor cycle detection to silently fail and leaving the system vulnerable to
-  infinite recursion at runtime. Added Behave regression tests
-  (`features/actor_subgraph_cycle_detection.feature`) and a Robot Framework
-  integration test (`robot/actor_compiler.robot`) to prevent regressions.
-- **Devcontainer auto-discovery wired into `git-checkout`/`fs-directory` handlers** (#4740):
-  `GitCheckoutHandler.discover_children()` and `FsDirectoryHandler.discover_children()` now
-  call `discover_devcontainers()` after scanning for `fs-directory` children. Any
-  `.devcontainer/devcontainer.json` or root-level `.devcontainer.json` found at the resource
-  location is registered as a `devcontainer-instance` child resource with
-  `provisioning_state: discovered`. Named configurations (`.devcontainer/<name>/devcontainer.json`)
-  are also discovered and carry the configuration name in the `config_name` property.
-  This wires the previously-isolated `discover_devcontainers()` function into the production
-  code path, enabling the spec's zero-configuration devcontainer experience.
-
-- **Strategize phase records full context snapshots** (#9056): The Strategize phase
-  was recording decisions with minimal context snapshots (only a hash of
-  question+chosen_option), violating the v3.2.0 acceptance criterion that decisions
-  must include full context snapshots sufficient to replay the decision. Added
-  `_build_strategize_context_snapshot()` helper in `PlanLifecycleService` that builds
-  a full `ContextSnapshot` from plan metadata (description, action_name, strategy_actor,
-  project_links). Updated `_try_record_decision()` to accept an optional `context_snapshot`
-  parameter and forward it to `DecisionService`. Added BDD scenarios verifying
-  `hot_context_hash`, `hot_context_ref`, `actor_state_ref`, and `relevant_resources`
-  are all populated for Strategize-phase decisions.
+- **`cleanup_stale` destroys git worktree branch on re-invoked execute, causing plan apply to find zero artifacts** (#11121): When `plan execute` is re-invoked for a previously-run plan (e.g., after transient error recovery), `_create_sandbox_for_plan()` was unconditionally calling `cleanup_stale()` before creating a sandbox, which destroyed the worktree branch still containing LLM output from the original execution. The fix checks whether the worktree branch already exists before attempting cleanup — if the branch is present it is preserved, allowing a cancelled or errored execute to preserve its artifacts and enabling a subsequent `plan apply` to find the expected changes. Includes BDD test coverage for the `_create_sandbox_for_plan` branch-preserving behavior on re-invoke.
 
 ### Changed
 

CONTRIBUTORS.md

@@ -27,6 +27,7 @@ Below are some of the specific details of various contributions.
 * HAL 9000 has contributed the file edit encoding parameter fix (PR #8258 / issue #7559).
 * HAL 9000 has contributed the architecture-pool-supervisor milestone assignment feature (PR #8188 / issue #7521): added `forgejo_update_pull_request` permission and documented the PR workflow for major spec changes, enabling automatic milestone assignment for specification PRs.
 * HAL 9000 has contributed the git worktree TOCTOU race condition fix (PR #8178 / issue #7507): replaced the unsafe mkdtemp() + rmdir() pattern with a parent-directory approach to eliminate the race window in concurrent git worktree operations.
+ * HAL 9000 has contributed the re-invoked execute branch preservation fix (#11121): updated `_create_sandbox_for_plan()` to skip `cleanup_stale` when the worktree branch already exists, preventing destruction of LLM output artifacts on plan re-invoke and enabling successful `plan apply`. Includes BDD test coverage.
 * HAL 9000 has contributed the git_tools TOCTOU race condition fix (PR #8255 / issue #7619): eliminated the Time-Of-Check-To-Time-Of-Use race in `_get_base_env()` by adding double-checked locking with a module-level `threading.Lock`, preventing concurrent threads from writing conflicting environment snapshots.
 * HAL 9000 has contributed the mandatory PR compliance checklist to `implementation-supervisor.md` (#9824): added an 8-item checklist to the worker prompt body with concrete items covering CHANGELOG.md, CONTRIBUTORS.md, commit footer, CI verification, BDD tests, Epic reference, labels, and milestone assignment to eliminate systemic PR merge blockers.
 * HAL 9000 has contributed the PlanResult.success derivation fix (PR #8214 / issue #7501): replaced the incorrect `error_message is None` heuristic with a dedicated `result_success` column in the plans table, ensuring plans with historical build errors are not incorrectly marked as failed after a successful apply.

features/sandbox_reinvoke_branch_preserve.feature

@@ -0,0 +1,19 @@
+@sandbox-reinvoke-branch-preserve
+Feature: Branch-preserving _create_sandbox_for_plan on re-invoke (#11121)
+
+  This feature addresses issue #11121: when plan execute is re-invoked for the
+  same plan_id, cleanup_stale was destroying the existing worktree branch that
+  carried LLM output artifacts. The fix skips cleanup when the branch already
+  exists, preserving artifacts so plan-apply can find them.
+
+  Scenario: _create_sandbox_for_plan preserves existing branch on re-invoke for srbp-rinv
+    Given a temp git repo with a worktree branch for plan "01TESTREINVOK00000000" for srbp-rinv
+    And a mock service that returns the first project when listing plans for srbp-rinv
+    When I call _create_sandbox_for_plan for plan "01TESTREINVOK00000000" for srbp-rinv
+    Then the branch "cleveragents/plan-01TESTREINVOK00000000" should still exist after re-invoke for srbp-rin
+
+  Scenario: _create_sandbox_for_plan performs cleanup when no branch exists for srbp-clean
+    Given a temp git repo without any worktree branches for plan "01TESTCLEAN00000000" for srbp-clean
+    And a mock service that returns the first project when listing plans for srbp-clean
+    When I call _create_sandbox_for_plan for plan "01TESTCLEAN00000000" for srbp-clean
+    Then a new sandbox is created for srbp-clean

features/steps/sandbox_reinvoke_branch_preserve_steps.py

@@ -0,0 +1,214 @@
+"""Steps for sandbox_reinvoke_branch_preserve.feature (#11121).
+
+Tests that _create_sandbox_for_plan preserves existing worktree branches
+when a plan is re-invoked, so artifacts from the original execution are
+not destroyed and available for subsequent plan-apply.
+"""
+
+from __future__ import annotations
+
+import os
+import subprocess
+import tempfile
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from behave import given, then, when
+
+
+def _git(args: list[str], cwd: str) -> subprocess.CompletedProcess[str]:
+    return subprocess.run(
+        ["git", *args],
+        cwd=cwd,
+        capture_output=True,
+        text=True,
+        check=True,
+        timeout=10,
+    )
+
+
+# ── Given ──────────────────────────────────────────────
+
+
+@given(
+    'a temp git repo with a worktree branch for plan "{plan_id}" for srbp-rinv'
+)
+def step_create_repo_with_existing_branch(context: object, plan_id: str) -> None:
+    d = tempfile.mkdtemp(prefix="srbp-rinv-")
+    _git(["config", "user.name", "T"], d)
+    _git(["config", "user.email", "t@t"], d)
+    _git(["config", "commit.gpgsign", "false"], d)
+    Path(d, "file.py").write_text("# original content\n")
+    _git(["add", "."], d)
+    _git(["commit", "-q", "-m", "init"], d)
+
+    branch = f"cleveragents/plan-{plan_id}"
+    wt_dir = tempfile.mkdtemp(prefix="srbp-rinv-wt-")
+
+    _git(["worktree", "add", "-b", branch, wt_dir, "HEAD"], d)
+    # LLM output artifact added to the worktree branch
+    Path(wt_dir, "file.py").write_text("# fixed by LLM\n")
+    _git(["add", "."], wt_dir)
+    _git(["commit", "-q", "-m", "LLM fix via sandbox"], wt_dir)
+
+    context.srbp_rinv_repo = d
+    context.srbp_rinv_wt_dir = wt_dir
+    context.srbp_rinv_branch = branch
+    context.srbp_rinv_plan_id = plan_id
+
+
+@given("a mock service that returns the first project when listing plans for srbp-rinv")
+def step_mock_service_for_reinvoke(context: object) -> None:
+    """Build mocks so _create_sandbox_for_plan can look up projects and resources."""
+    rid = "res-srbp-rinv-01"
+
+    mock_lr = MagicMock()
+    mock_lr.resource_id = rid
+
+    mock_proj = MagicMock()
+    mock_proj.linked_resources = [mock_lr]
+
+    mock_resource = MagicMock()
+    mock_resource.resource_type_name = "git-checkout"
+    mock_resource.location = context.srbp_rinv_repo
+    mock_resource.resource_id = rid
+
+    mock_plan = MagicMock()
+    mock_plan.project_links = [MagicMock(project_name="test-reinvoke")]
+
+    mock_service = MagicMock()
+    mock_service.get_plan.return_value = mock_plan
+
+    mock_project_repo = MagicMock()
+    mock_project_repo.get.side_effect = lambda n: mock_proj if n == "test-reinvoke" else None
+
+    mock_resource_registry = MagicMock()
+    mock_resource_registry.show_resource.return_value = mock_resource
+
+    mock_container = MagicMock()
+    mock_container.namespaced_project_repo.return_value = mock_project_repo
+    mock_container.resource_registry_service.return_value = mock_resource_registry
+
+    context.srbp_rinv_service = mock_service
+    context.srbp_rinv_container = mock_container
+
+
+@given('a temp git repo without any worktree branches for plan "{plan_id}" for srbp-clean')
+def step_create_clean_repo(context: object, plan_id: str) -> None:
+    d = tempfile.mkdtemp(prefix="srbp-clean-")
+    _git(["config", "user.name", "T"], d)
+    _git(["config", "user.email", "t@t"], d)
+    _git(["config", "commit.gpgsign", "false"], d)
+    Path(d, "file.py").write_text("# original content\n")
+    _git(["add", "."], d)
+    _git(["commit", "-q", "-m", "init"], d)
+
+    context.srbp_clean_repo = d
+    context.srbp_clean_branch = plan_id
+
+
+@given("a mock service that returns the first project when listing plans for srbp-clean")
+def step_mock_service_for_clean(context: object) -> None:
+    rid = "res-srbp-clean-01"
+
+    mock_lr = MagicMock()
+    mock_lr.resource_id = rid
+
+    mock_proj = MagicMock()
+    mock_proj.linked_resources = [mock_lr]
+
+    mock_resource = MagicMock()
+    mock_resource.resource_type_name = "git-checkout"
+    mock_resource.location = context.srbp_clean_repo
+    mock_resource.resource_id = rid
+
+    mock_plan = MagicMock()
+    mock_plan.project_links = [MagicMock(project_name="test-clean")]
+
+    mock_service = MagicMock()
+    mock_service.get_plan.return_value = mock_plan
+
+    mock_project_repo = MagicMock()
+    mock_project_repo.get.side_effect = (
+        lambda n: mock_proj if n == "test-clean" else None
+    )
+
+    mock_resource_registry = MagicMock()
+    mock_resource_registry.show_resource.return_value = mock_resource
+
+    mock_container = MagicMock()
+    mock_container.namespaced_project_repo.return_value = mock_project_repo
+    mock_container.resource_registry_service.return_value = mock_resource_registry
+
+    context.srbp_clean_service = mock_service
+    context.srbp_clean_container = mock_container
+
+
+# ── When ───────────────────────────────────────────────
+
+
+@when(
+    "I call _create_sandbox_for_plan for plan \"{plan_id}\" for srbp-rinv"
+)
+def step_call_create_on_reinvoke(context: object, plan_id: str) -> None:
+    from cleveragents.cli.commands.plan import _create_sandbox_for_plan
+
+    with patch(
+        "cleveragents.application.container.get_container",
+        return_value=context.srbp_rinv_container,
+    ):
+        (
+            context.srbp_rinv_root,
+            context.srbp_rinv_infos,
+        ) = _create_sandbox_for_plan(plan_id, context.srbp_rinv_service)
+
+    for info in context.srbp_rinv_infos:
+        context.add_cleanup(info.sandbox_obj.cleanup)
+
+
+@when("I call _create_sandbox_for_plan for plan \"{plan_id}\" for srbp-clean")
+def step_call_create_on_clean(context: object, plan_id: str) -> None:
+    from cleveragents.cli.commands.plan import _create_sandbox_for_plan
+
+    with patch(
+        "cleveragents.application.container.get_container",
+        return_value=context.srbp_clean_container,
+    ):
+        (
+            context.srbp_clean_root,
+            context.srbp_clean_infos,
+        ) = _create_sandbox_for_plan(plan_id, context.srbp_clean_service)
+
+    for info in context.srbp_clean_infos:
+        context.add_cleanup(info.sandbox_obj.cleanup)
+
+
+# ── Then ───────────────────────────────────────────────
+
+
+@then(
+    'the branch "{branch_name}" should still exist after re-invoke for srbp-rin'
+)
+def step_branch_still_exists(context: object, branch_name: str) -> None:
+    """Verify the existing worktree branch was NOT deleted by _create_sandbox_for_plan."""
+    result = subprocess.run(
+        ["git", "rev-parse", "--verify", f"refs/heads/{branch_name}"],
+        cwd=context.srbp_rinv_repo,
+        capture_output=True,
+        text=True,
+        check=False,
+        timeout=10,
+    )
+    assert result.returncode == 0, (
+        f"Branch {branch_name} was destroyed during re-invoke branch-preservation fix!"
+    )
+
+
+@then("a new sandbox is created for srbp-clean")
+def step_new_sandbox_created(context: object) -> None:
+    """Verify a new sandbox was created when no branch existed."""
+    assert hasattr(context, "srbp_clean_root"), "sandbox_root not set"
+    assert os.path.isdir(context.srbp_clean_root), (
+        f"sandbox_root is not a directory: {context.srbp_clean_root}"
+    )
+    assert len(context.srbp_clean_infos) > 0, "No sandbox infos returned"

src/cleveragents/cli/commands/plan.py

@@ -623,7 +623,12 @@ def _create_sandbox_for_plan(
         When no git resources are found, falls back to a flat directory
         and returns an empty list.
     """
+    import subprocess
+
     from cleveragents.application.container import get_container
+    from cleveragents.infrastructure.sandbox.git_worktree import (
+        _sanitise_branch_name,
+    )
 
     container = get_container()
     plan = service.get_plan(plan_id)
@@ -658,10 +663,32 @@ def _create_sandbox_for_plan(
                     continue  # M1: skip duplicate repos
                 processed_repos.add(repo_abs)
 
-                GitWorktreeSandbox.cleanup_stale(
-                    resource.location,
-                    plan_id,
+                # Only clean up if no branch exists yet.  If the branch
+                # already exists it may be carrying LLM output from a prior
+                # invocation of this same plan_id (or a cancelled execute);
+                # destroying it would cause plan-apply to find zero artifacts
+                # (issue #11121).
+                _branch_name = f"cleveragents/plan-{_sanitise_branch_name(plan_id)}"
+                branch_check_result = subprocess.run(
+                    ["git", "rev-parse", "--verify", f"refs/heads/{_branch_name}"],
+                    cwd=resource.location,
+                    capture_output=True,
+                    text=True,
+                    check=False,
+                    timeout=10,
                 )
+                if branch_check_result.returncode == 0:
+                    logger = structlog.get_logger(__name__ + ".sandbox")
+                    logger.debug(
+                        "Skipping cleanup_stale: branch %s already exists "
+                        "(plan may be re-invoked)",
+                        _branch_name,
+                    )
+                else:
+                    GitWorktreeSandbox.cleanup_stale(
+                        resource.location,
+                        plan_id,
+                    )
                 sandbox = GitWorktreeSandbox(
                     resource_id=resource.resource_id,
                     original_path=resource.location,