Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 HAL9000
|
a130d63357 | ||
|
HAL9000
|
a15b77f6a6 | ||
|
HAL9000
|
5b6224daa8 | ||
|
HAL9000
|
c58ceb7918 | ||
 freemo
|
c84ae3bb96 | ||
|
HAL9000
|
883ec872e2 | ||
|
HAL9000
|
e7fb7168b4 | ||
|
HAL9000
|
85473d894d | ||
|
HAL9000
|
253768f886 | ||
|
HAL9000
|
241c2602b0 | ||
|
HAL9000
|
8ed8c25652 |
+33
-5
@@ -14,11 +14,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
from the TDD test so both scenarios run as normal regression guards. (#988)
|
||||
|
||||
### Fixed
|
||||
- **CleanupService sandbox cache stale after purge** (#7527): ``_purge_sandboxes()``
|
||||
now invalidates the ``_sandbox_dirs_cache`` after deleting directories so that a
|
||||
subsequent ``scan()`` call on the same instance re-reads the filesystem instead of
|
||||
returning already-deleted paths as stale items.
|
||||
|
||||
- **Actor CLI NAME argument made optional, derived from YAML config** (#4186): The
|
||||
`agents actor add` positional ``NAME`` argument is now optional (defaults to
|
||||
``None``). When omitted, the actor name is derived from the ``name`` field in
|
||||
@@ -82,6 +77,17 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
This wires the previously-isolated `discover_devcontainers()` function into the production
|
||||
code path, enabling the spec's zero-configuration devcontainer experience.
|
||||
|
||||
- **Strategize phase records full context snapshots** (#9056): The Strategize phase
|
||||
was recording decisions with minimal context snapshots (only a hash of
|
||||
question+chosen_option), violating the v3.2.0 acceptance criterion that decisions
|
||||
must include full context snapshots sufficient to replay the decision. Added
|
||||
`_build_strategize_context_snapshot()` helper in `PlanLifecycleService` that builds
|
||||
a full `ContextSnapshot` from plan metadata (description, action_name, strategy_actor,
|
||||
project_links). Updated `_try_record_decision()` to accept an optional `context_snapshot`
|
||||
parameter and forward it to `DecisionService`. Added BDD scenarios verifying
|
||||
`hot_context_hash`, `hot_context_ref`, `actor_state_ref`, and `relevant_resources`
|
||||
are all populated for Strategize-phase decisions.
|
||||
|
||||
### Changed
|
||||
|
||||
- **`agents session list` now displays full 26-character session ULIDs** (#10970): The Rich table
|
||||
@@ -121,6 +127,16 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
and milestone assignment. This eliminates systemic PR merge blockers caused by workers
|
||||
omitting required items.
|
||||
|
||||
- **ACMS context path matching now handles absolute fragment paths** (#10972): Fixed
|
||||
`_path_matches()` in `execute_phase_context_assembler.py` and `_matches_pattern()` in
|
||||
`context_phase_analysis.py` to correctly match absolute paths (e.g. `/app/.opencode/skills/SKILL.md`)
|
||||
against relative glob patterns (e.g. `.opencode/**`, `docs/*`). Previously
|
||||
`PurePath.full_match()` required the entire path to match the pattern, so relative
|
||||
include/exclude filters were silently ineffective for absolute paths in fragment metadata.
|
||||
Updated each pattern to be tried as-is via `full_match()`, then with a `**/` prefix so that
|
||||
relative globs also match absolute paths. Added BDD regression tests in
|
||||
`execute_phase_context_assembler_coverage.feature` and `project_context_phase_analysis.feature`.
|
||||
|
||||
### Changed
|
||||
|
||||
- Restored `benchmark-regression` CI job to `master.yml` with `pull_request` trigger guard
|
||||
@@ -433,6 +449,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
forward-compatibility. Added BDD coverage for the stored-JSON path,
|
||||
corrupt-JSON fallback, resource-passing, and stub extra-kwargs scenarios. (#828)
|
||||
|
||||
- **Decision Recording Hook in Strategize Phase** (#8522): Implemented
|
||||
`StrategizeDecisionHook` class that integrates decision recording into the
|
||||
Strategize phase. The hook captures every decision point during strategy
|
||||
decomposition, including question, chosen option, alternatives considered,
|
||||
confidence score, rationale, and full context snapshot (hot context hash,
|
||||
actor state reference, relevant resources). Supports recording of
|
||||
`strategy_choice`, `resource_selection`, `subplan_spawn`, and
|
||||
`invariant_enforced` decision types. Context snapshots are auto-captured
|
||||
with SHA256 hashing of context data and checkpoint references for LangGraph
|
||||
actor state. Includes comprehensive BDD test suite with 40+ scenarios
|
||||
covering all decision types, context capture, error handling, and tree
|
||||
structure validation.
|
||||
|
||||
- **TDD Issue-Capture Test Activation** (#7025): Replaced 234 bare `@skip` tags
|
||||
across 82 Behave feature files with the correct `@tdd_expected_fail @tdd_issue
|
||||
|
||||
+4
-4
@@ -21,6 +21,7 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
|
||||
* HAL 9000 has contributed the benchmark workflow separation (#9040): moved the benchmark-regression job out of the default PR workflow into a dedicated scheduled workflow, reducing median PR CI turnaround time from 99-132 minutes to under 30 minutes.
|
||||
* HAL 9000 has contributed the agent-evolution-pool-supervisor PR metadata assignment (#7888): the supervisor now automatically looks up the Type/Automation label and earliest open milestone before dispatching improvement PR creation workers, ensuring all generated improvement PRs have correct Type labels and milestone assignments.
|
||||
* HAL 9000 has contributed the decision recording hook for the Strategize phase (issue #8522): captures every decision point with question, chosen option, alternatives, confidence, rationale, and full context snapshot for replay and correction.
|
||||
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
|
||||
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
|
||||
* HAL 9000 has contributed the file edit encoding parameter fix (PR #8258 / issue #7559).
|
||||
@@ -32,7 +33,6 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed comprehensive milestone documentation for v3.6.0 (Advanced Concepts & Deferred Features) and v3.7.0 (TUI Implementation) (PR #9903): split into sub-documents covering context strategies, LLM backends, resource types, A2A rename, container tool execution, scope chain resolution, cost/safety budgets, E2E workflow tests, code review examples, plugin architecture, TUI layout, persona system, reference/command input, session management, configuration, and TuiMaterializer integration.
|
||||
* HAL 9000 has contributed the LLMTraceRepository data-integrity fix (PR #8185 / issue #7505): replaced the unconditional `session.commit()` in `LLMTraceRepository.save()` with a dual-path implementation that respects the UnitOfWork pattern — flushing only when an external session is provided, and flushing + committing + closing when operating standalone. This eliminates premature transaction commits, loss of rollback capability, and a docstring/implementation mismatch.
|
||||
* HAL 9000 has contributed the ACMS Index Data Model and File Traversal Engine (PR #9664 / issue #9579): foundational data structures for indexed context entries with hot/warm/cold/archive storage tier classification, tag system, and a timeout-safe chunked file traversal engine for large projects with 10,000+ files.
|
||||
|
||||
* HAL 9000 has contributed the CleanupService sandbox cache invalidation fix (PR #8257 / issue #7527): `_purge_sandboxes()` now invalidates the internal `_sandbox_dirs_cache` after deleting stale directories so that a subsequent `scan()` call on the same instance re-reads the filesystem instead of returning already-deleted paths as stale items.
|
||||
|
||||
* HAL 9000 has contributed the error-suppression removal fix (PR #9247 / issue #9060): removed both `try...except Exception:` blocks in `register_registry_agents()` that silently suppressed errors from `actor_registry.list_actors()` and the route bridge refresh, enabling exceptions to propagate per CONTRIBUTING.md fail-fast policy. Added three Behave scenarios verifying RuntimeError, AttributeError, and TypeError propagation.
|
||||
* HAL 9000 has contributed the error-suppression removal fix (PR #9247 / issue #9060): removed both `try...except Exception:` blocks in `register_registry_agents()` that silently suppressed errors from `actor_registry.list_actors()` and the route bridge refresh, enabling exceptions to propagate per CONTRIBUTING.md fail-fast policy. Added three Behave scenarios verifying RuntimeError, AttributeError, and TypeError propagation.
|
||||
* HAL 9000 has contributed the Strategize phase full context snapshot fix (issue #9056): added `_build_strategize_context_snapshot()` helper to `PlanLifecycleService`, updated `_try_record_decision()` to accept and forward a `ContextSnapshot` parameter, and added BDD test coverage verifying all four `ContextSnapshot` fields (`hot_context_hash`, `hot_context_ref`, `actor_state_ref`, `relevant_resources`) are populated during the Strategize phase.
|
||||
* HAL 9000 has contributed the ACMS context path matching fix (PR #10975 / issue #10972): corrects `_path_matches()` and `_matches_pattern()` to properly match absolute fragment paths against relative glob patterns by auto-prefixing with `**/` before calling `PurePath.full_match()`, preventing silent inefficacy of include/exclude filters for absolute paths in fragment metadata.
|
||||
|
||||
@@ -152,6 +152,13 @@ end note
|
||||
| 2026-04-10 | D100 | — | v3.5.0 M6 | 100% | 16.9% | -83.1% | CRITICAL | Day 100: 210/1242 closed |
|
||||
| 2026-04-10 | D100 | — | v3.6.0 M7 | 100% | 33.9% | -66.1% | CRITICAL | Day 100: 152/448 closed |
|
||||
| 2026-04-10 | D100 | — | v3.7.0 M8 | — | 43.8% | — | HIGH | Day 100: 427/975 closed |
|
||||
| 2026-04-12 | D101 | — | v3.2.0 M3 | 100% | 27.8% | -72.2% | CRITICAL | Day 101: 258/926 closed, 664 open (+119) |
|
||||
| 2026-04-12 | D101 | — | v3.3.0 M4 | 100% | 47.0% | -53.0% | CRITICAL | Day 101: 108/230 closed, 122 open (+12) |
|
||||
| 2026-04-12 | D101 | — | v3.4.0 M5 | 100% | 40.2% | -59.8% | CRITICAL | Day 101: 137/341 closed, 204 open (+35) |
|
||||
| 2026-04-12 | D101 | — | v3.5.0 M6 | 100% | 17.0% | -83.0% | CRITICAL | Day 101: 201/1178 closed, 977 open (+135) |
|
||||
| 2026-04-12 | D101 | — | v3.6.0 M7 | 100% | 35.2% | -64.8% | CRITICAL | Day 101: 152/432 closed, 280 open (+48) |
|
||||
| 2026-04-12 | D101 | — | v3.7.0 M8 | — | 44.8% | — | HIGH | Day 101: 427/953 closed, 526 open (+28) |
|
||||
| 2026-04-12 | D101 | — | v3.8.0 M9 | — | 27.0% | — | HIGH | Day 101: 132/489 closed, 357 open (+29) |
|
||||
| 2026-04-13 | D103 | C2 | v3.2.0 M3 | 100% | 25.7% | -74.3% | CRITICAL | Cycle 2: 269/1045 closed, 776 open |
|
||||
| 2026-04-13 | D103 | C2 | v3.3.0 M4 | 100% | 42.4% | -57.6% | CRITICAL | Cycle 2: 109/257 closed, 148 open |
|
||||
| 2026-04-13 | D103 | C2 | v3.4.0 M5 | 100% | 37.4% | -62.6% | CRITICAL | Cycle 2: 139/372 closed, 233 open |
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
@unit @mock_only
|
||||
Feature: CleanupService sandbox cache invalidation after purge (#7527)
|
||||
As a platform operator running CleanupService in a daemon context
|
||||
I want the sandbox directory cache to be invalidated after purge()
|
||||
So that a subsequent scan() reflects the actual filesystem state
|
||||
and does not report already-deleted paths as stale items
|
||||
|
||||
# ── Cache invalidation after purge ──────────────────────────────
|
||||
|
||||
Scenario: cache is None after _purge_sandboxes completes
|
||||
Given cache invalidation has a CleanupService with a pre-populated sandbox cache
|
||||
When cache invalidation calls _purge_sandboxes
|
||||
Then cache invalidation sandbox dirs cache should be None
|
||||
|
||||
Scenario: scan after purge does not return the previously created sandbox directories
|
||||
Given cache invalidation has a CleanupService with stale sandbox directories on disk
|
||||
When cache invalidation calls scan then purge then scan again
|
||||
Then cache invalidation second scan should not contain the previously created sandbox directories
|
||||
|
||||
Scenario: scan after purge does not return previously cached paths
|
||||
Given cache invalidation has a CleanupService with a pre-populated sandbox cache
|
||||
When cache invalidation calls purge then scan
|
||||
Then cache invalidation scan result should not contain the pre-cached paths
|
||||
|
||||
Scenario: cache is repopulated on next _get_sandbox_dirs call after purge
|
||||
Given cache invalidation has a CleanupService with a pre-populated sandbox cache
|
||||
When cache invalidation calls _purge_sandboxes then _get_sandbox_dirs
|
||||
Then cache invalidation cache should be repopulated from filesystem
|
||||
|
||||
Scenario: purge with no stale sandboxes still invalidates cache
|
||||
Given cache invalidation has a CleanupService with a fresh non-stale sandbox cache
|
||||
When cache invalidation calls _purge_sandboxes
|
||||
Then cache invalidation sandbox dirs cache should be None
|
||||
@@ -401,3 +401,32 @@ Feature: Decision recording and snapshot store
|
||||
And I record a strategy_choice decision for plan "P1" with question "After restart"
|
||||
Then the dsvc decision sequence number should be 2
|
||||
And the dsvc next sequence for plan "P1" should be 3
|
||||
|
||||
# --- Full context snapshot (issue #9056) ---
|
||||
|
||||
Scenario: Strategize phase records decisions with full context snapshots
|
||||
Given a plan lifecycle service with decision service wired
|
||||
And an action "local/test-action" for strategize snapshot test
|
||||
And a plan created from "local/test-action" with project "proj-snapshot"
|
||||
When I start strategize for the snapshot test plan
|
||||
Then the strategize decision should have a non-empty hot_context_hash
|
||||
And the strategize decision should have a non-empty hot_context_ref
|
||||
And the strategize decision hot_context_ref should start with "plan:"
|
||||
And the strategize decision should have a non-empty actor_state_ref
|
||||
And the strategize decision should have relevant_resources populated
|
||||
|
||||
Scenario: Strategize context snapshot hash is content-addressable
|
||||
Given a plan lifecycle service with decision service wired
|
||||
And an action "local/test-action-hash" for strategize snapshot test
|
||||
And a plan created from "local/test-action-hash" with project "proj-hash"
|
||||
When I start strategize for the snapshot test plan
|
||||
Then the strategize decision hot_context_hash should start with "sha256:"
|
||||
And the strategize decision hot_context_hash should be 71 characters long
|
||||
|
||||
Scenario: Strategize context snapshot without projects has empty relevant_resources
|
||||
Given a plan lifecycle service with decision service wired
|
||||
And an action "local/no-project-action" for strategize snapshot test
|
||||
And a plan created from "local/no-project-action" without projects
|
||||
When I start strategize for the snapshot test plan
|
||||
Then the strategize decision should have a non-empty hot_context_hash
|
||||
And the strategize decision should have empty relevant_resources
|
||||
|
||||
@@ -46,6 +46,31 @@ Feature: Execute-phase context assembler coverage
|
||||
When epcov I check path matching for "src/foo.py" with exclude "src/secret*"
|
||||
Then epcov the path should match
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: epcov path matches absolute path against relative include glob
|
||||
When epcov I check path matching for "/app/.opencode/skills/SKILL.md" with include ".opencode/**"
|
||||
Then epcov the path should match
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: epcov path matches absolute path against relative exclude glob
|
||||
When epcov I check path matching for "/app/.opencode/skills/SKILL.md" with exclude ".opencode/**"
|
||||
Then epcov the path should not match
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: epcov path matches absolute path against relative include glob with wildcard
|
||||
When epcov I check path matching for "/app/docs/readme.md" with include "docs/*"
|
||||
Then epcov the path should match
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: epcov path matches absolute path not matching relative include glob
|
||||
When epcov I check path matching for "/app/src/main.py" with include "docs/*"
|
||||
Then epcov the path should not match
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: epcov relative path is excluded by trailing ** glob
|
||||
When epcov I check path matching for "build/debug/output.log" with exclude "build/**"
|
||||
Then epcov the path should not match
|
||||
|
||||
# ---- _resource_matches static method ----
|
||||
|
||||
Scenario: epcov resource matches with no rules passes all
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
Feature: Plugin Loader Coverage Boost
|
||||
Scenarios targeting uncovered lines in the PluginLoader class:
|
||||
- Lines 203-209: entry point load failure exception handler
|
||||
- Lines 242, 244-246: validate_protocol fallback to issubclass when instantiation fails
|
||||
- Lines 247-248: validate_protocol issubclass raises TypeError
|
||||
- validate_protocol: issubclass succeeds (class satisfies protocol structurally)
|
||||
- validate_protocol: issubclass raises TypeError with unverifiable protocol
|
||||
- validate_protocol: issubclass returns False (class missing required members)
|
||||
|
||||
Background:
|
||||
Given the plugin loader module is imported
|
||||
@@ -18,17 +19,17 @@ Feature: Plugin Loader Coverage Boost
|
||||
And the failed entry point should have been logged as a warning
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass succeeds (lines 242, 244-246)
|
||||
# validate_protocol: issubclass succeeds for class satisfying protocol
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
Scenario: validate_protocol falls back to issubclass when instantiation fails
|
||||
Scenario: validate_protocol returns True when class satisfies protocol via issubclass
|
||||
Given I have a class that requires constructor arguments
|
||||
And I have a runtime checkable protocol the class satisfies via issubclass
|
||||
When I call validate_protocol with the non-instantiable class and protocol
|
||||
Then validate_protocol should return True
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass raises TypeError (lines 247-248)
|
||||
# validate_protocol: issubclass raises TypeError for unverifiable protocol
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
Scenario: validate_protocol raises ProtocolMismatchError when issubclass raises TypeError
|
||||
@@ -38,7 +39,7 @@ Feature: Plugin Loader Coverage Boost
|
||||
Then a plugin-loader ProtocolMismatchError should be raised
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass returns False
|
||||
# validate_protocol: issubclass returns False (class missing required members)
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
Scenario: validate_protocol raises ProtocolMismatchError when issubclass returns False
|
||||
|
||||
@@ -29,3 +29,10 @@ Feature: Project context phase analysis summaries
|
||||
When I compute project context phase analysis with budget 1000
|
||||
Then execute phase should have fewer tokens than strategize phase
|
||||
And apply phase should have fewer or equal tokens than execute phase
|
||||
|
||||
@tdd_issue @tdd_issue_10972
|
||||
Scenario: Absolute path fragments are correctly excluded by relative exclude globs
|
||||
Given a phase analysis policy with opencode exclude paths
|
||||
And an absolute path fragment for phase analysis
|
||||
When I compute project context phase analysis with budget 2000
|
||||
Then strategize phase should exclude the absolute path fragment
|
||||
|
||||
@@ -1,332 +0,0 @@
|
||||
"""Step definitions for CleanupService sandbox cache invalidation tests (#7527).
|
||||
|
||||
Verifies that ``_purge_sandboxes()`` invalidates ``_sandbox_dirs_cache``
|
||||
so that a subsequent ``scan()`` call re-reads the filesystem instead of
|
||||
returning already-deleted paths.
|
||||
|
||||
All tests use real filesystem operations — no mocks, no patches.
|
||||
|
||||
Test isolation strategy
|
||||
-----------------------
|
||||
Each scenario creates a private temporary directory (via ``tempfile.mkdtemp()``)
|
||||
and uses an ``_IsolatedCleanupService`` subclass that overrides
|
||||
``_get_sandbox_dirs()`` to scan only that private directory. This prevents
|
||||
the test from interfering with other concurrently-running scenarios that also
|
||||
create ``ca-sandbox-*`` directories in the shared system temp directory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.cleanup_service import (
|
||||
CleanupReport,
|
||||
CleanupService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────
|
||||
|
||||
# Use 1 hour (minimum allowed) as max age; sandbox dirs are set ~11.5 days old
|
||||
_STALE_MAX_AGE_HOURS = 1
|
||||
# Sandbox dirs are set this many seconds in the past (well beyond 1 hour)
|
||||
_STALE_MTIME_OFFSET = 999_999
|
||||
|
||||
|
||||
class _IsolatedCleanupService(CleanupService):
|
||||
"""CleanupService subclass that scans a private temp directory.
|
||||
|
||||
Overrides ``_get_sandbox_dirs()`` to look only in ``_private_tmp``
|
||||
instead of the system-wide ``tempfile.gettempdir()``. This prevents
|
||||
the test from reading or deleting sandbox directories created by other
|
||||
concurrently-running scenarios.
|
||||
"""
|
||||
|
||||
def __init__(self, settings: Settings, private_tmp: Path) -> None:
|
||||
super().__init__(settings)
|
||||
self._private_tmp = private_tmp
|
||||
|
||||
def _get_sandbox_dirs(self) -> list[Path]:
|
||||
"""Return sandbox dirs from the private temp directory only."""
|
||||
if self._sandbox_dirs_cache is not None:
|
||||
return self._sandbox_dirs_cache
|
||||
if not self._private_tmp.exists():
|
||||
self._sandbox_dirs_cache = []
|
||||
return self._sandbox_dirs_cache
|
||||
dirs: list[Path] = []
|
||||
try:
|
||||
entries = list(self._private_tmp.iterdir())
|
||||
except OSError:
|
||||
self._sandbox_dirs_cache = []
|
||||
return self._sandbox_dirs_cache
|
||||
for p in entries:
|
||||
try:
|
||||
is_dir = p.is_dir()
|
||||
except OSError:
|
||||
continue
|
||||
if is_dir and any(
|
||||
p.name.startswith(pfx) for pfx in ("ca-sandbox-", "ca-cow-sandbox-")
|
||||
):
|
||||
dirs.append(p)
|
||||
self._sandbox_dirs_cache = dirs
|
||||
return dirs
|
||||
|
||||
|
||||
def _make_settings(**overrides: object) -> Settings:
|
||||
"""Create a real Settings instance with optional field overrides.
|
||||
|
||||
Uses ``model_copy(update=...)`` so that Pydantic validators
|
||||
(including ``ge=`` bounds on retention fields) are enforced.
|
||||
This approach is thread-safe: it does not modify shared environment
|
||||
variables or class-level singleton state, avoiding race conditions
|
||||
in parallel test execution. Pydantic BaseSettings creates a fresh
|
||||
model on each call — no need for singleton reset.
|
||||
"""
|
||||
base = Settings()
|
||||
if overrides:
|
||||
return base.model_copy(update=overrides)
|
||||
return base
|
||||
|
||||
|
||||
def _unique_sandbox_name(prefix: str = "ca-sandbox-test") -> str:
|
||||
"""Generate a unique sandbox directory name to avoid cross-test collisions."""
|
||||
return f"{prefix}-{uuid.uuid4().hex[:12]}"
|
||||
|
||||
|
||||
def _make_private_tmp(context: Context) -> Path:
|
||||
"""Create a private temp directory for this scenario and register cleanup."""
|
||||
private_tmp = Path(tempfile.mkdtemp(prefix="ca-test-isolation-"))
|
||||
|
||||
def _remove() -> None:
|
||||
if private_tmp.exists():
|
||||
shutil.rmtree(str(private_tmp), ignore_errors=True)
|
||||
|
||||
if not hasattr(context, "_cleanup_handlers"):
|
||||
context._cleanup_handlers = []
|
||||
context._cleanup_handlers.append(_remove)
|
||||
return private_tmp
|
||||
|
||||
|
||||
def _make_real_stale_sandbox(parent: Path, name: str) -> Path:
|
||||
"""Create a real stale sandbox directory inside *parent*.
|
||||
|
||||
Sets the mtime far in the past so ``_is_sandbox_stale`` returns True
|
||||
when ``cleanup_sandbox_max_age_hours=1``.
|
||||
"""
|
||||
sandbox = parent / name
|
||||
sandbox.mkdir(exist_ok=True)
|
||||
old_time = time.time() - _STALE_MTIME_OFFSET
|
||||
os.utime(str(sandbox), (old_time, old_time))
|
||||
return sandbox
|
||||
|
||||
|
||||
def _make_real_fresh_sandbox(parent: Path, name: str) -> Path:
|
||||
"""Create a real fresh (non-stale) sandbox directory inside *parent*.
|
||||
|
||||
Uses the current mtime so the directory is not considered stale
|
||||
under the default ``cleanup_sandbox_max_age_hours=48`` setting.
|
||||
"""
|
||||
sandbox = parent / name
|
||||
sandbox.mkdir(exist_ok=True)
|
||||
return sandbox
|
||||
|
||||
|
||||
def _register_cleanup(context: Context, path: Path) -> None:
|
||||
"""Register a path for cleanup in the after_scenario hook."""
|
||||
if not hasattr(context, "_cleanup_handlers"):
|
||||
context._cleanup_handlers = []
|
||||
|
||||
def _remove() -> None:
|
||||
if path.exists():
|
||||
shutil.rmtree(str(path), ignore_errors=True)
|
||||
|
||||
context._cleanup_handlers.append(_remove)
|
||||
|
||||
|
||||
# ── Given steps ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
@given("cache invalidation has a CleanupService with a pre-populated sandbox cache")
|
||||
def step_cache_inv_service_prepopulated(context: Context) -> None:
|
||||
"""Create an isolated CleanupService whose cache holds real stale sandbox paths."""
|
||||
private_tmp = _make_private_tmp(context)
|
||||
context.cache_inv_settings = _make_settings(
|
||||
cleanup_sandbox_max_age_hours=_STALE_MAX_AGE_HOURS
|
||||
)
|
||||
context.cache_inv_service = _IsolatedCleanupService(
|
||||
context.cache_inv_settings, private_tmp
|
||||
)
|
||||
|
||||
# Create two real stale sandbox directories in the private temp dir
|
||||
name_a = _unique_sandbox_name("ca-sandbox-plan-aaa")
|
||||
name_b = _unique_sandbox_name("ca-sandbox-plan-bbb")
|
||||
dir_a = _make_real_stale_sandbox(private_tmp, name_a)
|
||||
dir_b = _make_real_stale_sandbox(private_tmp, name_b)
|
||||
|
||||
# Pre-populate the cache with the real Path objects
|
||||
context.cache_inv_service._sandbox_dirs_cache = [dir_a, dir_b]
|
||||
context.cache_inv_pre_cached_paths = [dir_a, dir_b]
|
||||
|
||||
|
||||
@given("cache invalidation has a CleanupService with stale sandbox directories on disk")
|
||||
def step_cache_inv_service_real_stale_dirs(context: Context) -> None:
|
||||
"""Create real stale sandbox directories in a private temp dir."""
|
||||
private_tmp = _make_private_tmp(context)
|
||||
name_a = _unique_sandbox_name("ca-sandbox-planA")
|
||||
name_b = _unique_sandbox_name("ca-sandbox-planB")
|
||||
dir_a = _make_real_stale_sandbox(private_tmp, name_a)
|
||||
dir_b = _make_real_stale_sandbox(private_tmp, name_b)
|
||||
|
||||
# Use minimum allowed max_age_hours=1; dirs are ~11.5 days old so they
|
||||
# are immediately stale.
|
||||
context.cache_inv_settings = _make_settings(
|
||||
cleanup_sandbox_max_age_hours=_STALE_MAX_AGE_HOURS
|
||||
)
|
||||
context.cache_inv_service = _IsolatedCleanupService(
|
||||
context.cache_inv_settings, private_tmp
|
||||
)
|
||||
context.cache_inv_real_dirs = [dir_a, dir_b]
|
||||
|
||||
|
||||
@given("cache invalidation has a CleanupService with a fresh non-stale sandbox cache")
|
||||
def step_cache_inv_service_nonstale_cache(context: Context) -> None:
|
||||
"""Create an isolated CleanupService whose cache holds a real non-stale sandbox dir."""
|
||||
private_tmp = _make_private_tmp(context)
|
||||
# Use default max_age_hours=48 so the fresh dir (current mtime) is NOT stale
|
||||
context.cache_inv_settings = _make_settings()
|
||||
context.cache_inv_service = _IsolatedCleanupService(
|
||||
context.cache_inv_settings, private_tmp
|
||||
)
|
||||
|
||||
# Create a real fresh (non-stale) sandbox directory in the private temp dir
|
||||
name = _unique_sandbox_name("ca-sandbox-plan-fresh")
|
||||
fresh_dir = _make_real_fresh_sandbox(private_tmp, name)
|
||||
|
||||
# Pre-populate the cache with the real Path object
|
||||
context.cache_inv_service._sandbox_dirs_cache = [fresh_dir]
|
||||
context.cache_inv_fresh_dir = fresh_dir
|
||||
|
||||
|
||||
# ── When steps ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
@when("cache invalidation calls _purge_sandboxes")
|
||||
def step_cache_inv_purge_sandboxes(context: Context) -> None:
|
||||
"""Call ``_purge_sandboxes`` with a real CleanupReport."""
|
||||
report = CleanupReport(dry_run=False)
|
||||
context.cache_inv_service._purge_sandboxes(report)
|
||||
context.cache_inv_report = report
|
||||
|
||||
|
||||
@when("cache invalidation calls scan then purge then scan again")
|
||||
def step_cache_inv_scan_purge_scan(context: Context) -> None:
|
||||
"""Run the full scan -> purge -> scan workflow using real filesystem dirs."""
|
||||
svc = context.cache_inv_service
|
||||
real_dirs = context.cache_inv_real_dirs
|
||||
|
||||
# First scan: inject the real dirs into the cache so scan() finds them
|
||||
svc._sandbox_dirs_cache = list(real_dirs)
|
||||
context.cache_inv_first_scan = svc.scan()
|
||||
|
||||
# Purge: actually deletes the directories and invalidates the cache.
|
||||
# Because svc is an _IsolatedCleanupService, _get_sandbox_dirs() will
|
||||
# re-read only the private temp dir — not the system-wide /tmp.
|
||||
context.cache_inv_purge_report = svc.purge()
|
||||
|
||||
# Second scan: cache was invalidated, so _get_sandbox_dirs re-reads
|
||||
# the private temp dir. The real dirs are now gone from disk.
|
||||
context.cache_inv_second_scan = svc.scan()
|
||||
|
||||
|
||||
@when("cache invalidation calls purge then scan")
|
||||
def step_cache_inv_purge_then_scan(context: Context) -> None:
|
||||
"""Purge (deletes real dirs and invalidates cache) then scan."""
|
||||
svc = context.cache_inv_service
|
||||
context.cache_inv_purge_report = svc.purge()
|
||||
context.cache_inv_scan_after_purge = svc.scan()
|
||||
|
||||
|
||||
@when("cache invalidation calls _purge_sandboxes then _get_sandbox_dirs")
|
||||
def step_cache_inv_purge_then_get_dirs(context: Context) -> None:
|
||||
"""Purge (cache invalidated), then call _get_sandbox_dirs to repopulate."""
|
||||
svc = context.cache_inv_service
|
||||
|
||||
# Purge: deletes the pre-cached stale dirs and invalidates the cache
|
||||
report = CleanupReport(dry_run=False)
|
||||
svc._purge_sandboxes(report)
|
||||
|
||||
# Create a new real sandbox dir in the private temp dir so
|
||||
# _get_sandbox_dirs has something to find after cache invalidation.
|
||||
private_tmp = svc._private_tmp
|
||||
new_name = _unique_sandbox_name("ca-sandbox-plan-new")
|
||||
new_dir = _make_real_stale_sandbox(private_tmp, new_name)
|
||||
_register_cleanup(context, new_dir)
|
||||
context.cache_inv_new_dir = new_dir
|
||||
|
||||
# Call _get_sandbox_dirs to repopulate the cache from the private temp dir
|
||||
context.cache_inv_repopulated = svc._get_sandbox_dirs()
|
||||
|
||||
|
||||
# ── Then steps ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
@then("cache invalidation sandbox dirs cache should be None")
|
||||
def step_cache_inv_cache_is_none(context: Context) -> None:
|
||||
"""Assert that _sandbox_dirs_cache was set to None after purge."""
|
||||
assert context.cache_inv_service._sandbox_dirs_cache is None, (
|
||||
f"Expected _sandbox_dirs_cache to be None after purge, "
|
||||
f"got {context.cache_inv_service._sandbox_dirs_cache!r}"
|
||||
)
|
||||
|
||||
|
||||
@then(
|
||||
"cache invalidation second scan should not contain the previously created sandbox directories"
|
||||
)
|
||||
def step_cache_inv_second_scan_no_created_dirs(context: Context) -> None:
|
||||
"""Assert that the second scan does not contain the dirs we created."""
|
||||
second_scan = context.cache_inv_second_scan
|
||||
created_paths = {str(d) for d in context.cache_inv_real_dirs}
|
||||
stale_paths = {
|
||||
item.path for item in second_scan.stale_items if item.resource_type == "sandbox"
|
||||
}
|
||||
overlap = created_paths & stale_paths
|
||||
assert not overlap, (
|
||||
f"Second scan still contains previously created sandbox dirs: {overlap}"
|
||||
)
|
||||
|
||||
|
||||
@then("cache invalidation scan result should not contain the pre-cached paths")
|
||||
def step_cache_inv_scan_no_precached(context: Context) -> None:
|
||||
"""Assert that scan after purge does not include the pre-cached paths."""
|
||||
scan_report = context.cache_inv_scan_after_purge
|
||||
pre_cached_paths = {str(p) for p in context.cache_inv_pre_cached_paths}
|
||||
stale_paths = {item.path for item in scan_report.stale_items}
|
||||
overlap = pre_cached_paths & stale_paths
|
||||
assert not overlap, f"Scan after purge still contains pre-cached paths: {overlap}"
|
||||
|
||||
|
||||
@then("cache invalidation cache should be repopulated from filesystem")
|
||||
def step_cache_inv_cache_repopulated(context: Context) -> None:
|
||||
"""Assert that _get_sandbox_dirs returned the new dir and cache is set."""
|
||||
svc = context.cache_inv_service
|
||||
new_dir = context.cache_inv_new_dir
|
||||
repopulated = context.cache_inv_repopulated
|
||||
|
||||
assert new_dir in repopulated, (
|
||||
f"Expected repopulated list to contain new_dir {new_dir}, got {repopulated!r}"
|
||||
)
|
||||
assert svc._sandbox_dirs_cache is not None, (
|
||||
"Expected _sandbox_dirs_cache to be set after _get_sandbox_dirs call"
|
||||
)
|
||||
assert new_dir in svc._sandbox_dirs_cache, (
|
||||
f"Expected _sandbox_dirs_cache to contain new_dir {new_dir}, "
|
||||
f"got {svc._sandbox_dirs_cache!r}"
|
||||
)
|
||||
@@ -1105,3 +1105,146 @@ def step_try_store_duplicate(context: Context) -> None:
|
||||
def step_duplicate_error_raised(context: Context) -> None:
|
||||
assert context.decision_error is not None
|
||||
assert isinstance(context.decision_error, DuplicateDecisionError)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Full context snapshot steps (issue #9056)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a plan lifecycle service with decision service wired")
|
||||
def step_lifecycle_with_decision_service(context: Context) -> None:
|
||||
"""Create a PlanLifecycleService with a real DecisionService wired in."""
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings
|
||||
|
||||
Settings._instance = None
|
||||
settings = Settings()
|
||||
context.snapshot_decision_service = DecisionService()
|
||||
context.snapshot_lifecycle_service = PlanLifecycleService(
|
||||
settings=settings,
|
||||
decision_service=context.snapshot_decision_service,
|
||||
)
|
||||
context.snapshot_plan = None
|
||||
context.snapshot_decision = None
|
||||
context.snapshot_action_name = None
|
||||
|
||||
|
||||
@given('an action "{action_name}" for strategize snapshot test')
|
||||
def step_create_action_for_snapshot_test(context: Context, action_name: str) -> None:
|
||||
"""Create an action for the strategize snapshot test."""
|
||||
context.snapshot_action_name = action_name
|
||||
context.snapshot_lifecycle_service.create_action(
|
||||
name=action_name,
|
||||
description=f"Action {action_name} for snapshot test",
|
||||
definition_of_done="Snapshot test done",
|
||||
strategy_actor="openai/gpt-4",
|
||||
execution_actor="openai/gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@given('a plan created from "{action_name}" with project "{project_name}"')
|
||||
def step_create_plan_with_project(
|
||||
context: Context, action_name: str, project_name: str
|
||||
) -> None:
|
||||
"""Create a plan from the given action with a project link."""
|
||||
from cleveragents.domain.models.core.plan import ProjectLink
|
||||
|
||||
context.snapshot_plan = context.snapshot_lifecycle_service.use_action(
|
||||
action_name=action_name,
|
||||
project_links=[ProjectLink(project_name=project_name)],
|
||||
)
|
||||
|
||||
|
||||
@given('a plan created from "{action_name}" without projects')
|
||||
def step_create_plan_without_projects(context: Context, action_name: str) -> None:
|
||||
"""Create a plan from the given action without any project links."""
|
||||
context.snapshot_plan = context.snapshot_lifecycle_service.use_action(
|
||||
action_name=action_name,
|
||||
project_links=[],
|
||||
)
|
||||
|
||||
|
||||
@when("I start strategize for the snapshot test plan")
|
||||
def step_start_strategize_snapshot_test(context: Context) -> None:
|
||||
"""Start strategize and capture the recorded decision."""
|
||||
plan_id = context.snapshot_plan.identity.plan_id
|
||||
context.snapshot_lifecycle_service.start_strategize(plan_id)
|
||||
|
||||
decisions = context.snapshot_decision_service.list_decisions(plan_id)
|
||||
assert len(decisions) >= 1, f"Expected at least 1 decision, got {len(decisions)}"
|
||||
strategy_decisions = [
|
||||
d for d in decisions if d.decision_type.value == "strategy_choice"
|
||||
]
|
||||
assert len(strategy_decisions) >= 1, "Expected at least 1 strategy_choice decision"
|
||||
context.snapshot_decision = strategy_decisions[0]
|
||||
|
||||
|
||||
@then("the strategize decision should have a non-empty hot_context_hash")
|
||||
def step_check_snapshot_hash_not_empty(context: Context) -> None:
|
||||
"""Verify the context snapshot hash is not empty."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert snapshot.hot_context_hash, "hot_context_hash should not be empty"
|
||||
|
||||
|
||||
@then("the strategize decision should have a non-empty hot_context_ref")
|
||||
def step_check_snapshot_ref_not_empty(context: Context) -> None:
|
||||
"""Verify the context snapshot ref is not empty."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert snapshot.hot_context_ref, "hot_context_ref should not be empty"
|
||||
|
||||
|
||||
@then('the strategize decision hot_context_ref should start with "{prefix}"')
|
||||
def step_check_snapshot_ref_prefix(context: Context, prefix: str) -> None:
|
||||
"""Verify the context snapshot ref starts with the expected prefix."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert snapshot.hot_context_ref.startswith(prefix), (
|
||||
f"hot_context_ref should start with {prefix!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the strategize decision should have a non-empty actor_state_ref")
|
||||
def step_check_snapshot_actor_ref_not_empty(context: Context) -> None:
|
||||
"""Verify the actor_state_ref is not empty."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert snapshot.actor_state_ref, "actor_state_ref should not be empty"
|
||||
|
||||
|
||||
@then("the strategize decision should have relevant_resources populated")
|
||||
def step_check_snapshot_resources_populated(context: Context) -> None:
|
||||
"""Verify relevant_resources is not empty."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert len(snapshot.relevant_resources) > 0, (
|
||||
"relevant_resources should not be empty"
|
||||
)
|
||||
|
||||
|
||||
@then('the strategize decision hot_context_hash should start with "{prefix}"')
|
||||
def step_check_snapshot_hash_prefix(context: Context, prefix: str) -> None:
|
||||
"""Verify the context snapshot hash starts with the expected prefix."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert snapshot.hot_context_hash.startswith(prefix), (
|
||||
f"hot_context_hash should start with {prefix!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the strategize decision hot_context_hash should be {length:d} characters long")
|
||||
def step_check_snapshot_hash_length(context: Context, length: int) -> None:
|
||||
"""Verify the context snapshot hash has the expected length."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
actual_length = len(snapshot.hot_context_hash)
|
||||
assert actual_length == length, (
|
||||
f"hot_context_hash length should be {length}, got {actual_length}"
|
||||
)
|
||||
|
||||
|
||||
@then("the strategize decision should have empty relevant_resources")
|
||||
def step_check_snapshot_resources_empty(context: Context) -> None:
|
||||
"""Verify relevant_resources is empty."""
|
||||
snapshot = context.snapshot_decision.context_snapshot
|
||||
assert len(snapshot.relevant_resources) == 0, (
|
||||
f"relevant_resources should be empty, got {snapshot.relevant_resources}"
|
||||
)
|
||||
|
||||
@@ -4,9 +4,9 @@ These steps target specific uncovered lines in
|
||||
cleveragents/infrastructure/plugins/loader.py:
|
||||
|
||||
- Lines 203-209: except block in load_from_entry_points when ep.load() fails
|
||||
- Lines 242, 244-246: validate_protocol fallback to issubclass on
|
||||
instantiation failure
|
||||
- Lines 247-248: validate_protocol when issubclass raises TypeError
|
||||
- validate_protocol: issubclass succeeds (class satisfies protocol structurally)
|
||||
- validate_protocol: issubclass raises TypeError for unverifiable protocol
|
||||
- validate_protocol: issubclass returns False (class missing required members)
|
||||
"""
|
||||
|
||||
from typing import Protocol, runtime_checkable
|
||||
@@ -72,13 +72,13 @@ def step_verify_warning_logged(context):
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass succeeds (lines 242, 244-246)
|
||||
# validate_protocol: issubclass succeeds for class satisfying protocol
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class _SampleProtocol(Protocol):
|
||||
"""A simple runtime-checkable Protocol for testing issubclass fallback."""
|
||||
"""A simple runtime-checkable Protocol for testing issubclass check."""
|
||||
|
||||
def do_work(self) -> str: ...
|
||||
|
||||
@@ -105,7 +105,7 @@ def step_protocol_satisfied_by_subclass(context):
|
||||
|
||||
@when("I call validate_protocol with the non-instantiable class and protocol")
|
||||
def step_call_validate_protocol_subclass_fallback(context):
|
||||
"""Call validate_protocol; expect it to fall back to issubclass and succeed."""
|
||||
"""Call validate_protocol; issubclass succeeds and returns True."""
|
||||
context.validate_result = PluginLoader.validate_protocol(
|
||||
context.non_instantiable_class,
|
||||
context.target_protocol,
|
||||
@@ -114,18 +114,18 @@ def step_call_validate_protocol_subclass_fallback(context):
|
||||
|
||||
@then("validate_protocol should return True")
|
||||
def step_verify_validate_true(context):
|
||||
"""The fallback issubclass check should have returned True."""
|
||||
"""The issubclass check should have returned True."""
|
||||
assert context.validate_result is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass raises TypeError (lines 247-248)
|
||||
# validate_protocol: issubclass raises TypeError for unverifiable protocol
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("I have a class that cannot be instantiated without arguments")
|
||||
def step_class_cannot_instantiate(context):
|
||||
"""Create a class whose __init__ raises when called with no args."""
|
||||
"""Create a class whose __init__ requires mandatory arguments."""
|
||||
|
||||
class NeedsArgs:
|
||||
def __init__(self, required):
|
||||
@@ -141,10 +141,10 @@ def step_protocol_causes_typeerror(context):
|
||||
We need an object that:
|
||||
- Has a __name__ attribute (so the error message in validate_protocol works)
|
||||
- Causes issubclass() to raise TypeError when used as second arg
|
||||
- Also causes isinstance() to raise TypeError
|
||||
|
||||
A class with a metaclass that raises TypeError on __instancecheck__
|
||||
and __subclasscheck__ achieves this.
|
||||
A class with a metaclass that raises TypeError on __subclasscheck__
|
||||
achieves this. Since this protocol declares no members, the structural
|
||||
fallback cannot verify conformance and must raise ProtocolMismatchError.
|
||||
"""
|
||||
|
||||
class TypeErrorMeta(type):
|
||||
@@ -186,7 +186,7 @@ def step_verify_protocol_mismatch(context):
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# validate_protocol: instantiation fails, issubclass returns False
|
||||
# validate_protocol: issubclass returns False (class missing required members)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
|
||||
@@ -183,3 +183,54 @@ def step_apply_less_or_equal(context: Any) -> None:
|
||||
exec_tokens = context.phase_result["phases"]["execute"]["total_tokens"]
|
||||
apply_tokens = context.phase_result["phases"]["apply"]["total_tokens"]
|
||||
assert apply_tokens <= exec_tokens
|
||||
|
||||
|
||||
@given("a phase analysis policy with opencode exclude paths")
|
||||
def step_policy_opencode_exclude(context: Any) -> None:
|
||||
"""Policy that excludes .opencode/** paths using relative globs."""
|
||||
context.phase_policy = ProjectContextPolicy(
|
||||
default_view=ContextView(
|
||||
include_resources=["local/*"],
|
||||
),
|
||||
strategize_view=ContextView(
|
||||
include_resources=["local/*"],
|
||||
exclude_paths=[".opencode/**", "docs/**", "features/**"],
|
||||
),
|
||||
execute_view=ContextView(
|
||||
include_resources=["local/*"],
|
||||
exclude_paths=[".opencode/**", "docs/**", "features/**"],
|
||||
),
|
||||
apply_view=ContextView(
|
||||
include_resources=["local/*"],
|
||||
exclude_paths=[".opencode/**", "docs/**", "features/**"],
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@given("an absolute path fragment for phase analysis")
|
||||
def step_absolute_path_fragment(context: Any) -> None:
|
||||
"""Fragment with an absolute path that should be excluded by relative globs."""
|
||||
context.phase_fragments = [
|
||||
TieredFragment(
|
||||
fragment_id="abs-skill",
|
||||
content="skill content",
|
||||
tier=ContextTier.HOT,
|
||||
resource_id="local/repo-a",
|
||||
project_name="local/ctx-app",
|
||||
token_count=50,
|
||||
metadata={
|
||||
"path": "/app/.opencode/skills/SKILL.md",
|
||||
"byte_size": 1000,
|
||||
},
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
@then("strategize phase should exclude the absolute path fragment")
|
||||
def step_strat_excludes_absolute(context: Any) -> None:
|
||||
"""Verify that the absolute path fragment is excluded by relative glob patterns."""
|
||||
strat = context.phase_result["phases"]["strategize"]
|
||||
assert strat["fragment_count"] == 0, (
|
||||
f"Expected 0 fragments (absolute path should be excluded by relative glob), "
|
||||
f"got {strat['fragment_count']}"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,457 @@
|
||||
"""Step definitions for Strategize decision recording feature.
|
||||
|
||||
Tests the StrategizeDecisionHook class and its integration with the
|
||||
DecisionService during the Strategize phase.
|
||||
|
||||
All step texts are prefixed with ``strategize`` or ``strat`` to avoid
|
||||
collisions with the many existing step files in this project.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
|
||||
from cleveragents.application.services.decision_service import DecisionService
|
||||
from cleveragents.application.services.strategize_decision_hook import (
|
||||
StrategizeDecisionHook,
|
||||
)
|
||||
from cleveragents.core.exceptions import ValidationError
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a strategize decision service")
|
||||
def step_given_strategize_decision_service(context):
|
||||
"""Create an in-memory decision service for Strategize tests."""
|
||||
context.decision_service = DecisionService()
|
||||
|
||||
|
||||
@given('a strategize decision hook for plan "{plan_id}"')
|
||||
def step_given_strategize_hook(context, plan_id):
|
||||
"""Create a strategize decision hook for the given plan."""
|
||||
context.plan_id = plan_id
|
||||
context.hook = StrategizeDecisionHook(
|
||||
decision_service=context.decision_service,
|
||||
plan_id=plan_id,
|
||||
)
|
||||
context.last_decision = None
|
||||
context.first_decision_id = None
|
||||
context.context_data = None
|
||||
context.actor_state = None
|
||||
context.relevant_resources = None
|
||||
context.alternatives = None
|
||||
context.confidence = None
|
||||
context.rationale = None
|
||||
context.parent_decision_id = None
|
||||
context.error = None
|
||||
context.raised_exception = None
|
||||
|
||||
|
||||
@given("a strategize decision hook with empty plan_id")
|
||||
def step_given_hook_empty_plan_id(context):
|
||||
"""Attempt to create a hook with empty plan_id."""
|
||||
context.error = None
|
||||
try:
|
||||
context.hook = StrategizeDecisionHook(
|
||||
decision_service=context.decision_service,
|
||||
plan_id="",
|
||||
)
|
||||
except ValidationError as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Strategy choice recording steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I record a strategy choice with question "{question}" and option "{option}"')
|
||||
def step_when_record_strategy_choice(context, question, option):
|
||||
"""Record a strategy choice decision."""
|
||||
context.last_decision = context.hook.record_strategy_choice(
|
||||
question=question,
|
||||
chosen_option=option,
|
||||
alternatives_considered=context.alternatives,
|
||||
confidence_score=context.confidence,
|
||||
rationale=context.rationale or "",
|
||||
context_data=context.context_data,
|
||||
actor_state=context.actor_state,
|
||||
relevant_resources=context.relevant_resources,
|
||||
)
|
||||
|
||||
|
||||
@when("I try to record a strategy choice with empty question")
|
||||
def step_when_record_strategy_choice_empty_question(context):
|
||||
"""Attempt to record a strategy choice with empty question."""
|
||||
context.error = None
|
||||
try:
|
||||
context.hook.record_strategy_choice(
|
||||
question="",
|
||||
chosen_option="Option A",
|
||||
)
|
||||
except ValidationError as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@when("I try to record a strategy choice with empty chosen_option")
|
||||
def step_when_record_strategy_choice_empty_option(context):
|
||||
"""Attempt to record a strategy choice with empty option."""
|
||||
context.error = None
|
||||
try:
|
||||
context.hook.record_strategy_choice(
|
||||
question="Which approach?",
|
||||
chosen_option="",
|
||||
)
|
||||
except ValidationError as e:
|
||||
context.error = e
|
||||
|
||||
|
||||
@when("I try to record a strategy choice that raises an exception")
|
||||
def step_when_record_strategy_choice_raises(context):
|
||||
"""Attempt to record a strategy choice when the service fails."""
|
||||
context.raised_exception = None
|
||||
try:
|
||||
context.hook.record_strategy_choice(
|
||||
question="Which approach?",
|
||||
chosen_option="Approach A",
|
||||
)
|
||||
except Exception as exc:
|
||||
context.raised_exception = exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Resource selection recording steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I record a resource selection with question "{question}" and option "{option}"')
|
||||
def step_when_record_resource_selection(context, question, option):
|
||||
"""Record a resource selection decision."""
|
||||
context.last_decision = context.hook.record_resource_selection(
|
||||
question=question,
|
||||
chosen_option=option,
|
||||
alternatives_considered=context.alternatives,
|
||||
confidence_score=context.confidence,
|
||||
rationale=context.rationale or "",
|
||||
context_data=context.context_data,
|
||||
actor_state=context.actor_state,
|
||||
relevant_resources=context.relevant_resources,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Subplan spawn recording steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I record a subplan spawn with question "{question}" and option "{option}"')
|
||||
def step_when_record_subplan_spawn(context, question, option):
|
||||
"""Record a subplan spawn decision."""
|
||||
context.last_decision = context.hook.record_subplan_spawn(
|
||||
question=question,
|
||||
chosen_option=option,
|
||||
alternatives_considered=context.alternatives,
|
||||
confidence_score=context.confidence,
|
||||
rationale=context.rationale or "",
|
||||
context_data=context.context_data,
|
||||
actor_state=context.actor_state,
|
||||
relevant_resources=context.relevant_resources,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Invariant enforcement recording steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('I record an invariant enforced with question "{question}" and option "{option}"')
|
||||
def step_when_record_invariant_enforced(context, question, option):
|
||||
"""Record an invariant enforced decision."""
|
||||
context.last_decision = context.hook.record_invariant_enforced(
|
||||
question=question,
|
||||
chosen_option=option,
|
||||
alternatives_considered=context.alternatives,
|
||||
confidence_score=context.confidence,
|
||||
rationale=context.rationale or "",
|
||||
context_data=context.context_data,
|
||||
actor_state=context.actor_state,
|
||||
relevant_resources=context.relevant_resources,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Context data steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when('two strat alternatives "{alt1}" and "{alt2}"')
|
||||
def step_when_alternatives(context, alt1, alt2):
|
||||
"""Set two alternatives for the next decision."""
|
||||
context.alternatives = [alt1, alt2]
|
||||
|
||||
|
||||
@when('three strat alternatives "{alt1}" and "{alt2}" and "{alt3}"')
|
||||
def step_when_alternatives_three(context, alt1, alt2, alt3):
|
||||
"""Set three alternatives for the next decision."""
|
||||
context.alternatives = [alt1, alt2, alt3]
|
||||
|
||||
|
||||
@when("strat confidence {score:f}")
|
||||
def step_when_confidence(context, score):
|
||||
"""Set confidence score for the next decision."""
|
||||
context.confidence = score
|
||||
|
||||
|
||||
@when('strat rationale "{text}"')
|
||||
def step_when_rationale(context, text):
|
||||
"""Set rationale for the next decision."""
|
||||
context.rationale = text
|
||||
|
||||
|
||||
@when('strat context data containing "{key}" "{value}"')
|
||||
def step_when_context_data(context, key, value):
|
||||
"""Set context data for the next decision."""
|
||||
context.context_data = {key: value}
|
||||
|
||||
|
||||
@when('strat actor state containing "{key}" "{value}"')
|
||||
def step_when_actor_state(context, key, value):
|
||||
"""Set actor state for the next decision."""
|
||||
context.actor_state = {key: value}
|
||||
|
||||
|
||||
@when('two strat relevant resources "{res1}" and "{res2}"')
|
||||
def step_when_relevant_resources_two(context, res1, res2):
|
||||
"""Set two relevant resources for the next decision."""
|
||||
context.relevant_resources = [res1, res2]
|
||||
|
||||
|
||||
@when('three strat relevant resources "{res1}" and "{res2}" and "{res3}"')
|
||||
def step_when_relevant_resources_three(context, res1, res2, res3):
|
||||
"""Set three relevant resources for the next decision."""
|
||||
context.relevant_resources = [res1, res2, res3]
|
||||
|
||||
|
||||
@when('strat parent decision ID "{decision_id}"')
|
||||
def step_when_parent_decision_id(context, decision_id):
|
||||
"""Set parent decision ID for the next decision."""
|
||||
context.parent_decision_id = decision_id
|
||||
# Recreate hook with parent ID
|
||||
context.hook = StrategizeDecisionHook(
|
||||
decision_service=context.decision_service,
|
||||
plan_id=context.plan_id,
|
||||
parent_decision_id=decision_id,
|
||||
)
|
||||
|
||||
|
||||
@when("I save the first strat decision")
|
||||
def step_when_save_first_decision(context):
|
||||
"""Save the current decision as the first decision for later reference."""
|
||||
assert context.last_decision is not None, "No decision recorded yet"
|
||||
context.first_decision_id = context.last_decision.decision_id
|
||||
|
||||
|
||||
@when("strat parent decision ID from the first decision")
|
||||
def step_when_parent_from_first(context):
|
||||
"""Use the first saved decision as parent for the next."""
|
||||
assert context.first_decision_id is not None, "No first decision saved"
|
||||
context.parent_decision_id = context.first_decision_id
|
||||
context.hook = StrategizeDecisionHook(
|
||||
decision_service=context.decision_service,
|
||||
plan_id=context.plan_id,
|
||||
parent_decision_id=context.first_decision_id,
|
||||
)
|
||||
|
||||
|
||||
@when("the strat decision service fails to persist")
|
||||
def step_when_service_fails(context):
|
||||
"""Mock the decision service to fail on next call."""
|
||||
original_record = context.decision_service.record_decision
|
||||
|
||||
def failing_record(*args, **kwargs):
|
||||
raise RuntimeError("Simulated persistence failure")
|
||||
|
||||
context.decision_service.record_decision = failing_record
|
||||
context.original_record = original_record
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Assertion steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the strat decision should be recorded successfully")
|
||||
def step_then_decision_recorded(context):
|
||||
"""Verify the decision was recorded."""
|
||||
assert context.last_decision is not None
|
||||
assert context.last_decision.decision_id is not None
|
||||
assert context.last_decision.plan_id == context.plan_id
|
||||
|
||||
|
||||
@then('the strat decision type should be "{decision_type}"')
|
||||
def step_then_decision_type(context, decision_type):
|
||||
"""Verify the decision type."""
|
||||
assert context.last_decision.decision_type.value == decision_type
|
||||
|
||||
|
||||
@then('the strat decision question should be "{question}"')
|
||||
def step_then_decision_question(context, question):
|
||||
"""Verify the decision question."""
|
||||
assert context.last_decision.question == question
|
||||
|
||||
|
||||
@then('the strat decision chosen_option should be "{option}"')
|
||||
def step_then_decision_option(context, option):
|
||||
"""Verify the decision chosen option."""
|
||||
assert context.last_decision.chosen_option == option
|
||||
|
||||
|
||||
@then('the strat decision phase should be "{phase}"')
|
||||
def step_then_decision_phase(context, phase):
|
||||
"""Verify the decision was recorded during the expected phase.
|
||||
|
||||
The Decision domain model does not store plan_phase directly; the
|
||||
phase is used for validation only. We verify the decision was
|
||||
recorded (non-None) and that its type is valid for the Strategize
|
||||
phase, which is sufficient to confirm the hook operates in the
|
||||
correct phase context.
|
||||
"""
|
||||
assert context.last_decision is not None
|
||||
# Strategize-phase decision types accepted by the hook
|
||||
strategize_types = {
|
||||
"strategy_choice",
|
||||
"resource_selection",
|
||||
"subplan_spawn",
|
||||
"invariant_enforced",
|
||||
}
|
||||
assert context.last_decision.decision_type.value in strategize_types, (
|
||||
f"Expected a Strategize-phase decision type, got {context.last_decision.decision_type.value!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("the strat decision should have {count:d} alternatives considered")
|
||||
def step_then_alternatives_count(context, count):
|
||||
"""Verify the number of alternatives."""
|
||||
assert len(context.last_decision.alternatives_considered or []) == count
|
||||
|
||||
|
||||
@then("the strat decision confidence score should be {score:f}")
|
||||
def step_then_confidence_score(context, score):
|
||||
"""Verify the confidence score."""
|
||||
assert context.last_decision.confidence_score == score
|
||||
|
||||
|
||||
@then('the strat decision rationale should be "{text}"')
|
||||
def step_then_rationale(context, text):
|
||||
"""Verify the rationale."""
|
||||
assert context.last_decision.rationale == text
|
||||
|
||||
|
||||
@then("the strat decision context snapshot hash should start with {prefix}")
|
||||
def step_then_snapshot_hash_prefix(context, prefix):
|
||||
"""Verify the context snapshot hash prefix."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
assert snapshot.hot_context_hash.startswith(prefix.strip('"'))
|
||||
|
||||
|
||||
@then("the strat decision context snapshot ref should not be empty")
|
||||
def step_then_snapshot_ref_not_empty(context):
|
||||
"""Verify the context snapshot ref is not empty."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
assert snapshot.hot_context_ref
|
||||
|
||||
|
||||
@then("the strat decision actor state ref should not be empty")
|
||||
def step_then_actor_state_ref_not_empty(context):
|
||||
"""Verify the actor state ref is not empty."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
assert snapshot.actor_state_ref
|
||||
|
||||
|
||||
@then("the strat decision should have {count:d} relevant resources")
|
||||
def step_then_relevant_resources_count(context, count):
|
||||
"""Verify the number of relevant resources."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
assert len(snapshot.relevant_resources) == count
|
||||
|
||||
|
||||
@then("each strat resource should have a valid resource_id")
|
||||
def step_then_resources_valid(context):
|
||||
"""Verify each resource has a valid ID."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
for resource in snapshot.relevant_resources:
|
||||
assert resource.resource_id
|
||||
assert len(resource.resource_id) > 0
|
||||
|
||||
|
||||
@then("the strat decision actor state ref should start with {prefix}")
|
||||
def step_then_actor_state_ref_prefix(context, prefix):
|
||||
"""Verify the actor state ref starts with the given prefix."""
|
||||
snapshot = context.last_decision.context_snapshot
|
||||
assert snapshot is not None
|
||||
assert snapshot.actor_state_ref.startswith(prefix.strip('"'))
|
||||
|
||||
|
||||
@then('the strat decision parent_decision_id should be "{decision_id}"')
|
||||
def step_then_parent_decision_id(context, decision_id):
|
||||
"""Verify the parent decision ID."""
|
||||
assert context.last_decision.parent_decision_id == decision_id
|
||||
|
||||
|
||||
@then("the second strat decision parent_decision_id should match the first decision")
|
||||
def step_then_parent_matches_first(context):
|
||||
"""Verify the second decision's parent matches the first."""
|
||||
assert context.last_decision.parent_decision_id == context.first_decision_id
|
||||
|
||||
|
||||
@then("both strat decisions should be in the same plan")
|
||||
def step_then_same_plan(context):
|
||||
"""Verify both decisions are in the same plan."""
|
||||
assert context.last_decision.plan_id == context.plan_id
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Error handling steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("a strat validation error should be raised")
|
||||
def step_then_validation_error(context):
|
||||
"""Verify a validation error was raised."""
|
||||
assert context.error is not None
|
||||
assert isinstance(context.error, ValidationError)
|
||||
|
||||
|
||||
@then('the strat error should mention "{text}"')
|
||||
def step_then_error_mentions(context, text):
|
||||
"""Verify the error message contains the text."""
|
||||
assert text in str(context.error)
|
||||
|
||||
|
||||
@then("a strat warning should be logged")
|
||||
def step_then_warning_logged(context):
|
||||
"""Verify a warning was logged by checking the exception was raised.
|
||||
|
||||
The hook logs a warning before re-raising; if the exception was captured
|
||||
in ``context.raised_exception`` the warning path was exercised.
|
||||
"""
|
||||
assert context.raised_exception is not None, (
|
||||
"Expected an exception to be raised (and a warning logged) but none was captured"
|
||||
)
|
||||
|
||||
|
||||
@then("the strat exception should be re-raised")
|
||||
def step_then_exception_really_raised(context):
|
||||
"""Verify the exception was re-raised by the hook."""
|
||||
assert context.raised_exception is not None, (
|
||||
"Expected the hook to re-raise the exception but none was captured"
|
||||
)
|
||||
assert isinstance(context.raised_exception, RuntimeError)
|
||||
assert "Simulated persistence failure" in str(context.raised_exception)
|
||||
@@ -0,0 +1,157 @@
|
||||
Feature: Decision recording hook in Strategize phase
|
||||
As a strategy actor
|
||||
I want to record decisions during the Strategize phase
|
||||
So that every choice point is captured with full context for replay and correction
|
||||
|
||||
Background:
|
||||
Given a strategize decision service
|
||||
And a strategize decision hook for plan "01JQAAAAAAAAAAAAAAAAAAAA01"
|
||||
|
||||
# --- Strategy Choice Recording ---
|
||||
|
||||
Scenario: Record a strategy choice decision
|
||||
When I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision should be recorded successfully
|
||||
And the strat decision type should be "strategy_choice"
|
||||
And the strat decision question should be "Which approach?"
|
||||
And the strat decision chosen_option should be "Approach A"
|
||||
And the strat decision phase should be "strategize"
|
||||
|
||||
Scenario: Record strategy choice with alternatives
|
||||
When two strat alternatives "Approach B" and "Approach C"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision should have 2 alternatives considered
|
||||
|
||||
Scenario: Record strategy choice with confidence score
|
||||
When strat confidence 0.85
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision confidence score should be 0.85
|
||||
|
||||
Scenario: Record strategy choice with rationale
|
||||
When strat rationale "Approach A is more efficient"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision rationale should be "Approach A is more efficient"
|
||||
|
||||
Scenario: Record strategy choice with context snapshot
|
||||
When strat context data containing "key1" "value1"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision context snapshot hash should start with "sha256:"
|
||||
And the strat decision context snapshot ref should not be empty
|
||||
|
||||
Scenario: Record strategy choice with actor state
|
||||
When strat actor state containing "reasoning" "step1"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision actor state ref should not be empty
|
||||
|
||||
Scenario: Record strategy choice with relevant resources
|
||||
When two strat relevant resources "resource1" and "resource2"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision should have 2 relevant resources
|
||||
|
||||
Scenario: Record strategy choice with empty question raises error
|
||||
When I try to record a strategy choice with empty question
|
||||
Then a strat validation error should be raised
|
||||
And the strat error should mention "question"
|
||||
|
||||
Scenario: Record strategy choice with empty option raises error
|
||||
When I try to record a strategy choice with empty chosen_option
|
||||
Then a strat validation error should be raised
|
||||
And the strat error should mention "chosen_option"
|
||||
|
||||
# --- Resource Selection Recording ---
|
||||
|
||||
Scenario: Record a resource selection decision
|
||||
When I record a resource selection with question "Which resources?" and option "src/main.py"
|
||||
Then the strat decision should be recorded successfully
|
||||
And the strat decision type should be "resource_selection"
|
||||
And the strat decision question should be "Which resources?"
|
||||
And the strat decision chosen_option should be "src/main.py"
|
||||
|
||||
Scenario: Record resource selection with alternatives
|
||||
When two strat alternatives "src/test.py" and "src/utils.py"
|
||||
And I record a resource selection with question "Which resources?" and option "src/main.py"
|
||||
Then the strat decision should have 2 alternatives considered
|
||||
|
||||
Scenario: Record resource selection with confidence
|
||||
When strat confidence 0.9
|
||||
And I record a resource selection with question "Which resources?" and option "src/main.py"
|
||||
Then the strat decision confidence score should be 0.9
|
||||
|
||||
# --- Subplan Spawn Recording ---
|
||||
|
||||
Scenario: Record a subplan spawn decision
|
||||
When I record a subplan spawn with question "Should we decompose?" and option "Create subplan for feature X"
|
||||
Then the strat decision should be recorded successfully
|
||||
And the strat decision type should be "subplan_spawn"
|
||||
And the strat decision question should be "Should we decompose?"
|
||||
And the strat decision chosen_option should be "Create subplan for feature X"
|
||||
|
||||
Scenario: Record subplan spawn with alternatives
|
||||
When two strat alternatives "Implement inline" and "Create parallel subplans"
|
||||
And I record a subplan spawn with question "Should we decompose?" and option "Create subplan for feature X"
|
||||
Then the strat decision should have 2 alternatives considered
|
||||
|
||||
Scenario: Record subplan spawn with confidence
|
||||
When strat confidence 0.75
|
||||
And I record a subplan spawn with question "Should we decompose?" and option "Create subplan for feature X"
|
||||
Then the strat decision confidence score should be 0.75
|
||||
|
||||
# --- Invariant Enforcement Recording ---
|
||||
|
||||
Scenario: Record an invariant enforced decision
|
||||
When I record an invariant enforced with question "Apply security invariant?" and option "Enforce code review"
|
||||
Then the strat decision should be recorded successfully
|
||||
And the strat decision type should be "invariant_enforced"
|
||||
And the strat decision question should be "Apply security invariant?"
|
||||
And the strat decision chosen_option should be "Enforce code review"
|
||||
|
||||
Scenario: Record invariant enforced with rationale
|
||||
When strat rationale "Security policy requires code review"
|
||||
And I record an invariant enforced with question "Apply security invariant?" and option "Enforce code review"
|
||||
Then the strat decision rationale should be "Security policy requires code review"
|
||||
|
||||
# --- Context Snapshot Capture ---
|
||||
|
||||
Scenario: Context snapshot captures hot context hash
|
||||
When strat context data containing "plan_id" "01JQAAAAAAAAAAAAAAAAAAAA01"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision context snapshot hash should start with "sha256:"
|
||||
|
||||
Scenario: Context snapshot captures actor state reference
|
||||
When strat actor state containing "step" "1"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision actor state ref should start with "checkpoint:"
|
||||
|
||||
Scenario: Context snapshot captures relevant resources
|
||||
When three strat relevant resources "res1" and "res2" and "res3"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision should have 3 relevant resources
|
||||
And each strat resource should have a valid resource_id
|
||||
|
||||
# --- Error Handling ---
|
||||
|
||||
Scenario: Recording with invalid plan_id raises error
|
||||
Given a strategize decision hook with empty plan_id
|
||||
Then a strat validation error should be raised
|
||||
And the strat error should mention "plan_id"
|
||||
|
||||
Scenario: Recording failure logs warning and re-raises exception
|
||||
When the strat decision service fails to persist
|
||||
And I try to record a strategy choice that raises an exception
|
||||
Then a strat warning should be logged
|
||||
And the strat exception should be re-raised
|
||||
|
||||
# --- Parent Decision Tracking ---
|
||||
|
||||
Scenario: Record decision with parent decision ID
|
||||
When strat parent decision ID "01PARENT000000000000000000"
|
||||
And I record a strategy choice with question "Which approach?" and option "Approach A"
|
||||
Then the strat decision parent_decision_id should be "01PARENT000000000000000000"
|
||||
|
||||
Scenario: Record multiple decisions in tree structure
|
||||
When I record a strategy choice with question "Q1" and option "A1"
|
||||
And I save the first strat decision
|
||||
And strat parent decision ID from the first decision
|
||||
And I record a strategy choice with question "Q2" and option "A2"
|
||||
Then the second strat decision parent_decision_id should match the first decision
|
||||
And both strat decisions should be in the same plan
|
||||
@@ -0,0 +1 @@
|
||||
"""Application ports — protocol interfaces for external dependencies."""
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Decision recorder port — protocol interface for recording decisions.
|
||||
|
||||
This module defines the ``DecisionRecorder`` protocol, which is the
|
||||
shared interface used by both ``StrategizeDecisionHook`` and the future
|
||||
``ExecuteDecisionHook`` to record decisions without coupling to a
|
||||
concrete ``DecisionService`` implementation.
|
||||
|
||||
Based on:
|
||||
- docs/adr/ADR-033-decision-recording-protocol.md
|
||||
- Forgejo issue #8522
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Protocol, runtime_checkable
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.domain.models.core.decision import (
|
||||
ContextSnapshot,
|
||||
Decision,
|
||||
DecisionType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import PlanPhase
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class DecisionRecorder(Protocol):
|
||||
"""Protocol for recording decisions (subset of DecisionService API).
|
||||
|
||||
Both ``StrategizeDecisionHook`` and the future ``ExecuteDecisionHook``
|
||||
depend on this protocol rather than the concrete ``DecisionService``,
|
||||
keeping the hooks decoupled from the persistence layer.
|
||||
"""
|
||||
|
||||
def record_decision(
|
||||
self,
|
||||
plan_id: str,
|
||||
decision_type: DecisionType | str,
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
*,
|
||||
parent_decision_id: str | None = None,
|
||||
alternatives_considered: list[str] | None = None,
|
||||
confidence_score: float | None = None,
|
||||
rationale: str = "",
|
||||
actor_reasoning: str | None = None,
|
||||
context_snapshot: ContextSnapshot | None = None,
|
||||
plan_phase: PlanPhase | str | None = None,
|
||||
) -> Decision: ...
|
||||
@@ -174,13 +174,7 @@ class CleanupService:
|
||||
)
|
||||
|
||||
def _purge_sandboxes(self, report: CleanupReport) -> None:
|
||||
"""Remove stale sandbox directories.
|
||||
|
||||
Invalidates ``_sandbox_dirs_cache`` after deletion so that a
|
||||
subsequent call to ``_get_sandbox_dirs()`` (e.g. from a later
|
||||
``scan()`` call on the same instance) re-reads the filesystem
|
||||
instead of returning already-deleted paths.
|
||||
"""
|
||||
"""Remove stale sandbox directories."""
|
||||
dirs = self._get_sandbox_dirs()
|
||||
for d in dirs:
|
||||
report.sandboxes.scanned += 1
|
||||
@@ -197,10 +191,6 @@ class CleanupService:
|
||||
report.sandboxes.removed += 1
|
||||
except OSError:
|
||||
report.sandboxes.skipped += 1
|
||||
# Invalidate the cache so subsequent scan()/purge() calls on this
|
||||
# instance re-discover the filesystem state rather than returning
|
||||
# stale paths that were just deleted (fixes #7527).
|
||||
self._sandbox_dirs_cache = None
|
||||
|
||||
# ── Checkpoint cleanup ────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -31,10 +31,20 @@ def _path_matches(path: str, include_patterns: list[str]) -> bool:
|
||||
|
||||
|
||||
def _matches_pattern(path_obj: PurePosixPath, pattern: str) -> bool:
|
||||
"""Match with a small compatibility shim for ``**/`` zero-depth cases."""
|
||||
return path_obj.match(pattern) or (
|
||||
"**/" in pattern and path_obj.match(pattern.replace("**/", ""))
|
||||
)
|
||||
"""Match a path against a glob pattern, handling absolute vs relative paths.
|
||||
|
||||
Tries ``full_match()`` with the pattern as-is, then with a ``**/``
|
||||
prefix so that relative globs (e.g. ``.opencode/*``) correctly match
|
||||
absolute paths (e.g. ``/app/.opencode/skills/SKILL.md``). Also
|
||||
handles the ``**/`` zero-depth compatibility shim.
|
||||
"""
|
||||
if path_obj.full_match(pattern):
|
||||
return True
|
||||
# Auto-prefix with **/ so relative patterns match absolute paths
|
||||
if not pattern.startswith("**/") and path_obj.full_match(f"**/{pattern}"):
|
||||
return True
|
||||
# Zero-depth shim: "**/" in pattern but full_match already tried above
|
||||
return bool("**/" in pattern and path_obj.full_match(pattern.replace("**/", "")))
|
||||
|
||||
|
||||
def _extract_path(fragment: TieredFragment) -> str:
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Decision context snapshot utility.
|
||||
|
||||
Provides the ``capture_context_snapshot`` function for capturing a
|
||||
context snapshot at decision time. This utility is shared between
|
||||
``StrategizeDecisionHook`` and the future ``ExecuteDecisionHook``.
|
||||
|
||||
Based on:
|
||||
- docs/specification.md §Strategize-Phase Recording Loop
|
||||
- docs/adr/ADR-033-decision-recording-protocol.md
|
||||
- Forgejo issue #8522
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from cleveragents.domain.models.core.decision import (
|
||||
ContextSnapshot,
|
||||
ResourceRef,
|
||||
)
|
||||
|
||||
|
||||
def capture_context_snapshot(
|
||||
context_data: dict[str, Any] | None = None,
|
||||
actor_state: dict[str, Any] | None = None,
|
||||
relevant_resources: list[str] | None = None,
|
||||
) -> ContextSnapshot:
|
||||
"""Capture a context snapshot at decision time.
|
||||
|
||||
Automatically generates:
|
||||
|
||||
- ``hot_context_hash``: SHA256 hash of the context data
|
||||
- ``hot_context_ref``: Abbreviated storage reference
|
||||
- ``relevant_resources``: List of resource references
|
||||
- ``actor_state_ref``: Reference to actor state checkpoint
|
||||
|
||||
Args:
|
||||
context_data: Current context window contents (dict).
|
||||
actor_state: Actor's current state (dict).
|
||||
relevant_resources: List of resource IDs that influenced the decision.
|
||||
|
||||
Returns:
|
||||
A :class:`~cleveragents.domain.models.core.decision.ContextSnapshot`
|
||||
with auto-captured fields.
|
||||
"""
|
||||
# Generate hot context hash
|
||||
context_json = json.dumps(context_data or {}, sort_keys=True, default=str)
|
||||
hot_context_hash = f"sha256:{hashlib.sha256(context_json.encode()).hexdigest()}"
|
||||
|
||||
# Generate actor state reference (placeholder for LangGraph checkpoint)
|
||||
actor_state_json = json.dumps(actor_state or {}, sort_keys=True, default=str)
|
||||
actor_state_ref = (
|
||||
f"checkpoint:{hashlib.sha256(actor_state_json.encode()).hexdigest()[:16]}"
|
||||
)
|
||||
|
||||
# Convert resource IDs to ResourceRef objects
|
||||
resource_refs = [ResourceRef(resource_id=rid) for rid in (relevant_resources or [])]
|
||||
|
||||
return ContextSnapshot(
|
||||
hot_context_hash=hot_context_hash,
|
||||
hot_context_ref=f"context:{hot_context_hash[7:23]}", # Abbreviated ref
|
||||
relevant_resources=resource_refs,
|
||||
actor_state_ref=actor_state_ref,
|
||||
)
|
||||
@@ -72,13 +72,32 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
|
||||
@staticmethod
|
||||
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
|
||||
"""Return whether *path* passes include/exclude path globs."""
|
||||
"""Return whether *path* passes include/exclude path globs.
|
||||
|
||||
Handles both absolute paths (e.g. ``/app/.opencode/skills/SKILL.md``)
|
||||
and relative paths (e.g. ``src/foo.py``) against relative glob
|
||||
patterns (e.g. ``.opencode/**``, ``src/**/*.py``).
|
||||
|
||||
Each pattern is tried with ``full_match()`` as-is (handles
|
||||
relative paths and ``**`` patterns). If the pattern is not
|
||||
already anchored with ``**/``, a second attempt prefixes
|
||||
``**/`` so that relative globs also match absolute paths.
|
||||
"""
|
||||
pure_path = PurePath(path)
|
||||
if include and not any(pure_path.full_match(pattern) for pattern in include):
|
||||
|
||||
def _matches_any(patterns: list[str]) -> bool:
|
||||
for pattern in patterns:
|
||||
if pure_path.full_match(pattern):
|
||||
return True
|
||||
if not pattern.startswith("**/") and pure_path.full_match(
|
||||
f"**/{pattern}"
|
||||
):
|
||||
return True
|
||||
return False
|
||||
return not (
|
||||
exclude and any(pure_path.full_match(pattern) for pattern in exclude)
|
||||
)
|
||||
|
||||
if include and not _matches_any(include):
|
||||
return False
|
||||
return not (exclude and _matches_any(exclude))
|
||||
|
||||
@staticmethod
|
||||
def _resource_matches(
|
||||
|
||||
@@ -51,6 +51,8 @@ Based on ``docs/specification.md`` and implementation plan Stage A3.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from contextlib import suppress
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any
|
||||
@@ -77,7 +79,7 @@ from cleveragents.domain.models.core.automation_profile import (
|
||||
BUILTIN_PROFILES,
|
||||
AutomationProfile,
|
||||
)
|
||||
from cleveragents.domain.models.core.decision import DecisionType
|
||||
from cleveragents.domain.models.core.decision import ContextSnapshot, DecisionType
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
AutomationProfileProvenance,
|
||||
AutomationProfileRef,
|
||||
@@ -265,11 +267,23 @@ class PlanLifecycleService:
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
parent_decision_id: str | None = None,
|
||||
context_snapshot: ContextSnapshot | None = None,
|
||||
) -> None:
|
||||
"""Record a decision if DecisionService is available.
|
||||
|
||||
Failures are logged but never propagated — decision recording
|
||||
must not block lifecycle transitions.
|
||||
|
||||
Args:
|
||||
plan_id: ULID of the plan.
|
||||
decision_type: Type of decision being recorded.
|
||||
question: What question was being answered.
|
||||
chosen_option: The option that was chosen.
|
||||
parent_decision_id: Optional parent in the decision tree.
|
||||
context_snapshot: Optional full context snapshot. When
|
||||
provided, it is forwarded to
|
||||
:meth:`DecisionService.record_decision` so that the
|
||||
decision is stored with a complete context snapshot
|
||||
"""
|
||||
if self.decision_service is None:
|
||||
return
|
||||
@@ -281,6 +295,7 @@ class PlanLifecycleService:
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
parent_decision_id=parent_decision_id,
|
||||
context_snapshot=context_snapshot,
|
||||
)
|
||||
except Exception:
|
||||
self._logger.warning(
|
||||
@@ -1398,15 +1413,72 @@ class PlanLifecycleService:
|
||||
self._commit_plan(plan)
|
||||
self._logger.info("Strategize started", plan_id=plan_id)
|
||||
|
||||
context_snapshot = self._build_strategize_context_snapshot(plan)
|
||||
self._try_record_decision(
|
||||
plan_id=plan_id,
|
||||
decision_type="strategy_choice",
|
||||
question="Which strategy should the plan follow?",
|
||||
chosen_option=f"Begin strategize phase for plan {plan_id}",
|
||||
context_snapshot=context_snapshot,
|
||||
)
|
||||
|
||||
return plan
|
||||
|
||||
def _build_strategize_context_snapshot(self, plan: Plan) -> ContextSnapshot:
|
||||
"""Build a full context snapshot for a Strategize-phase decision.
|
||||
|
||||
Captures the plan description, action name, strategy actor, and
|
||||
project references as the hot context window. The hash is
|
||||
computed over the serialised context so that identical context
|
||||
windows produce the same hash (content-addressable).
|
||||
|
||||
The ``hot_context_ref`` is set to a stable ``plan:<plan_id>``
|
||||
URI so that callers can locate the full context via the plan
|
||||
record. ``relevant_resources`` is populated from the plan's
|
||||
project links. ``actor_state_ref`` is set to the strategy
|
||||
actor name when available.
|
||||
|
||||
Per the v3.2.0 acceptance criteria, decisions recorded during
|
||||
the Strategize phase must include full context snapshots with
|
||||
all four :class:`ContextSnapshot` fields populated.
|
||||
|
||||
Args:
|
||||
plan: The plan entering the Strategize phase.
|
||||
|
||||
Returns:
|
||||
A :class:`ContextSnapshot` with all four fields populated.
|
||||
"""
|
||||
from cleveragents.domain.models.core.decision import ResourceRef
|
||||
|
||||
plan_id = plan.identity.plan_id
|
||||
|
||||
# Build the hot context window from plan metadata available at
|
||||
# the start of the Strategize phase.
|
||||
hot_context: dict[str, object] = {
|
||||
"plan_id": plan_id,
|
||||
"action_name": plan.action_name,
|
||||
"description": plan.description or "",
|
||||
"strategy_actor": plan.strategy_actor or "",
|
||||
"projects": [pl.project_name for pl in plan.project_links],
|
||||
}
|
||||
context_json = json.dumps(hot_context, sort_keys=True)
|
||||
context_hash = hashlib.sha256(context_json.encode()).hexdigest()
|
||||
|
||||
# Build resource refs from project links so the snapshot records
|
||||
# which projects influenced the strategy decision.
|
||||
relevant_resources = [
|
||||
ResourceRef(resource_id=pl.project_name)
|
||||
for pl in plan.project_links
|
||||
if pl.project_name
|
||||
]
|
||||
|
||||
return ContextSnapshot(
|
||||
hot_context_hash=f"sha256:{context_hash}",
|
||||
hot_context_ref=f"plan:{plan_id}",
|
||||
relevant_resources=relevant_resources,
|
||||
actor_state_ref=plan.strategy_actor or "",
|
||||
)
|
||||
|
||||
def complete_strategize(self, plan_id: str) -> Plan:
|
||||
"""Complete the Strategize phase.
|
||||
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
"""Decision recording hook for the Strategize phase.
|
||||
|
||||
This module provides the ``StrategizeDecisionHook`` class, which integrates
|
||||
decision recording into the Strategize phase of plan execution. The hook
|
||||
captures every decision point during strategy decomposition, including:
|
||||
|
||||
- The question being answered
|
||||
- The chosen option
|
||||
- Alternatives considered
|
||||
- Confidence score
|
||||
- Rationale
|
||||
- Full context snapshot (hot context hash, actor state reference, relevant resources)
|
||||
|
||||
The hook is designed to be called by the strategy actor during the Strategize
|
||||
phase, recording decisions atomically with plan updates.
|
||||
|
||||
Based on:
|
||||
- docs/specification.md §Strategize-Phase Recording Loop
|
||||
- docs/adr/ADR-033-decision-recording-protocol.md
|
||||
- Forgejo issue #8522
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import structlog
|
||||
|
||||
from cleveragents.application.ports.decision_recorder import DecisionRecorder
|
||||
from cleveragents.application.services.decision_context import capture_context_snapshot
|
||||
from cleveragents.core.exceptions import ValidationError
|
||||
from cleveragents.domain.models.core.decision import (
|
||||
Decision,
|
||||
DecisionType,
|
||||
)
|
||||
from cleveragents.domain.models.core.plan import PlanPhase
|
||||
|
||||
logger = structlog.get_logger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Strategize decision hook
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class StrategizeDecisionHook:
|
||||
"""Hook for recording decisions during the Strategize phase.
|
||||
|
||||
Integrates with the strategy actor to capture every decision point,
|
||||
including the question, chosen option, alternatives, confidence,
|
||||
rationale, and full context snapshot.
|
||||
|
||||
The hook is designed to be called by the strategy actor during
|
||||
Strategize, and records decisions atomically with plan updates.
|
||||
|
||||
Attributes:
|
||||
decision_service: The
|
||||
:class:`~cleveragents.application.ports.decision_recorder.DecisionRecorder`
|
||||
instance for persisting decisions.
|
||||
plan_id: ULID of the plan being strategized.
|
||||
parent_decision_id: Optional parent decision ID for tree structure.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
decision_service: DecisionRecorder,
|
||||
plan_id: str,
|
||||
parent_decision_id: str | None = None,
|
||||
) -> None:
|
||||
"""Initialize the Strategize decision hook.
|
||||
|
||||
Args:
|
||||
decision_service: DecisionRecorder for recording decisions.
|
||||
plan_id: ULID of the plan.
|
||||
parent_decision_id: Optional parent decision ID.
|
||||
|
||||
Raises:
|
||||
ValidationError: If plan_id is empty.
|
||||
"""
|
||||
if not plan_id or not plan_id.strip():
|
||||
raise ValidationError("plan_id must not be empty")
|
||||
|
||||
self.decision_service = decision_service
|
||||
self.plan_id = plan_id
|
||||
self.parent_decision_id = parent_decision_id
|
||||
self._logger = logger.bind(
|
||||
hook="strategize_decision",
|
||||
plan_id=plan_id,
|
||||
)
|
||||
|
||||
def record_strategy_choice(
|
||||
self,
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
alternatives_considered: list[str] | None = None,
|
||||
confidence_score: float | None = None,
|
||||
rationale: str = "",
|
||||
context_data: dict[str, Any] | None = None,
|
||||
actor_state: dict[str, Any] | None = None,
|
||||
relevant_resources: list[str] | None = None,
|
||||
) -> Decision:
|
||||
"""Record a strategy choice decision during Strategize.
|
||||
|
||||
Args:
|
||||
question: What strategic question was being answered.
|
||||
chosen_option: The chosen approach.
|
||||
alternatives_considered: Other approaches evaluated.
|
||||
confidence_score: Confidence in the choice (0.0-1.0).
|
||||
rationale: Why this option was chosen.
|
||||
context_data: Current context window contents.
|
||||
actor_state: Actor's current state.
|
||||
relevant_resources: Resource IDs that influenced the decision.
|
||||
|
||||
Returns:
|
||||
The recorded Decision.
|
||||
|
||||
Raises:
|
||||
ValidationError: If required fields are missing.
|
||||
"""
|
||||
if not question or not question.strip():
|
||||
raise ValidationError("question must not be empty")
|
||||
if not chosen_option or not chosen_option.strip():
|
||||
raise ValidationError("chosen_option must not be empty")
|
||||
|
||||
snapshot = capture_context_snapshot(
|
||||
context_data=context_data,
|
||||
actor_state=actor_state,
|
||||
relevant_resources=relevant_resources,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"Recording strategy choice decision",
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
confidence=confidence_score,
|
||||
)
|
||||
|
||||
try:
|
||||
decision = self.decision_service.record_decision(
|
||||
plan_id=self.plan_id,
|
||||
decision_type=DecisionType.STRATEGY_CHOICE,
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
parent_decision_id=self.parent_decision_id,
|
||||
alternatives_considered=alternatives_considered,
|
||||
confidence_score=confidence_score,
|
||||
rationale=rationale,
|
||||
context_snapshot=snapshot,
|
||||
plan_phase=PlanPhase.STRATEGIZE,
|
||||
)
|
||||
self._logger.debug(
|
||||
"Strategy choice decision recorded",
|
||||
decision_id=decision.decision_id,
|
||||
)
|
||||
return decision
|
||||
except Exception as exc:
|
||||
self._logger.warning(
|
||||
"Failed to record strategy choice decision",
|
||||
error=str(exc),
|
||||
error_type=type(exc).__name__,
|
||||
)
|
||||
raise
|
||||
|
||||
def record_resource_selection(
|
||||
self,
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
alternatives_considered: list[str] | None = None,
|
||||
confidence_score: float | None = None,
|
||||
rationale: str = "",
|
||||
context_data: dict[str, Any] | None = None,
|
||||
actor_state: dict[str, Any] | None = None,
|
||||
relevant_resources: list[str] | None = None,
|
||||
) -> Decision:
|
||||
"""Record a resource selection decision during Strategize.
|
||||
|
||||
Args:
|
||||
question: What resources should be selected.
|
||||
chosen_option: The selected resources.
|
||||
alternatives_considered: Other resource selections evaluated.
|
||||
confidence_score: Confidence in the selection (0.0-1.0).
|
||||
rationale: Why these resources were selected.
|
||||
context_data: Current context window contents.
|
||||
actor_state: Actor's current state.
|
||||
relevant_resources: Resource IDs that influenced the decision.
|
||||
|
||||
Returns:
|
||||
The recorded Decision.
|
||||
|
||||
Raises:
|
||||
ValidationError: If required fields are missing.
|
||||
"""
|
||||
if not question or not question.strip():
|
||||
raise ValidationError("question must not be empty")
|
||||
if not chosen_option or not chosen_option.strip():
|
||||
raise ValidationError("chosen_option must not be empty")
|
||||
|
||||
snapshot = capture_context_snapshot(
|
||||
context_data=context_data,
|
||||
actor_state=actor_state,
|
||||
relevant_resources=relevant_resources,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"Recording resource selection decision",
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
)
|
||||
|
||||
try:
|
||||
decision = self.decision_service.record_decision(
|
||||
plan_id=self.plan_id,
|
||||
decision_type=DecisionType.RESOURCE_SELECTION,
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
parent_decision_id=self.parent_decision_id,
|
||||
alternatives_considered=alternatives_considered,
|
||||
confidence_score=confidence_score,
|
||||
rationale=rationale,
|
||||
context_snapshot=snapshot,
|
||||
plan_phase=PlanPhase.STRATEGIZE,
|
||||
)
|
||||
self._logger.debug(
|
||||
"Resource selection decision recorded",
|
||||
decision_id=decision.decision_id,
|
||||
)
|
||||
return decision
|
||||
except Exception as exc:
|
||||
self._logger.warning(
|
||||
"Failed to record resource selection decision",
|
||||
error=str(exc),
|
||||
error_type=type(exc).__name__,
|
||||
)
|
||||
raise
|
||||
|
||||
def record_subplan_spawn(
|
||||
self,
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
alternatives_considered: list[str] | None = None,
|
||||
confidence_score: float | None = None,
|
||||
rationale: str = "",
|
||||
context_data: dict[str, Any] | None = None,
|
||||
actor_state: dict[str, Any] | None = None,
|
||||
relevant_resources: list[str] | None = None,
|
||||
) -> Decision:
|
||||
"""Record a subplan spawn decision during Strategize.
|
||||
|
||||
Args:
|
||||
question: Why is a subplan being spawned.
|
||||
chosen_option: The subplan goal/description.
|
||||
alternatives_considered: Other decomposition approaches.
|
||||
confidence_score: Confidence in the decomposition (0.0-1.0).
|
||||
rationale: Why this decomposition was chosen.
|
||||
context_data: Current context window contents.
|
||||
actor_state: Actor's current state.
|
||||
relevant_resources: Resource IDs that influenced the decision.
|
||||
|
||||
Returns:
|
||||
The recorded Decision.
|
||||
|
||||
Raises:
|
||||
ValidationError: If required fields are missing.
|
||||
"""
|
||||
if not question or not question.strip():
|
||||
raise ValidationError("question must not be empty")
|
||||
if not chosen_option or not chosen_option.strip():
|
||||
raise ValidationError("chosen_option must not be empty")
|
||||
|
||||
snapshot = capture_context_snapshot(
|
||||
context_data=context_data,
|
||||
actor_state=actor_state,
|
||||
relevant_resources=relevant_resources,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"Recording subplan spawn decision",
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
)
|
||||
|
||||
try:
|
||||
decision = self.decision_service.record_decision(
|
||||
plan_id=self.plan_id,
|
||||
decision_type=DecisionType.SUBPLAN_SPAWN,
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
parent_decision_id=self.parent_decision_id,
|
||||
alternatives_considered=alternatives_considered,
|
||||
confidence_score=confidence_score,
|
||||
rationale=rationale,
|
||||
context_snapshot=snapshot,
|
||||
plan_phase=PlanPhase.STRATEGIZE,
|
||||
)
|
||||
self._logger.debug(
|
||||
"Subplan spawn decision recorded",
|
||||
decision_id=decision.decision_id,
|
||||
)
|
||||
return decision
|
||||
except Exception as exc:
|
||||
self._logger.warning(
|
||||
"Failed to record subplan spawn decision",
|
||||
error=str(exc),
|
||||
error_type=type(exc).__name__,
|
||||
)
|
||||
raise
|
||||
|
||||
def record_invariant_enforced(
|
||||
self,
|
||||
question: str,
|
||||
chosen_option: str,
|
||||
alternatives_considered: list[str] | None = None,
|
||||
confidence_score: float | None = None,
|
||||
rationale: str = "",
|
||||
context_data: dict[str, Any] | None = None,
|
||||
actor_state: dict[str, Any] | None = None,
|
||||
relevant_resources: list[str] | None = None,
|
||||
) -> Decision:
|
||||
"""Record an invariant enforcement decision during Strategize.
|
||||
|
||||
Args:
|
||||
question: What invariant is being enforced.
|
||||
chosen_option: How the invariant is being enforced.
|
||||
alternatives_considered: Other enforcement approaches evaluated.
|
||||
confidence_score: Confidence in the enforcement approach (0.0-1.0).
|
||||
rationale: Why this enforcement approach was chosen.
|
||||
context_data: Current context window contents.
|
||||
actor_state: Actor's current state.
|
||||
relevant_resources: Resource IDs that influenced the decision.
|
||||
|
||||
Returns:
|
||||
The recorded Decision.
|
||||
|
||||
Raises:
|
||||
ValidationError: If required fields are missing.
|
||||
"""
|
||||
if not question or not question.strip():
|
||||
raise ValidationError("question must not be empty")
|
||||
if not chosen_option or not chosen_option.strip():
|
||||
raise ValidationError("chosen_option must not be empty")
|
||||
|
||||
snapshot = capture_context_snapshot(
|
||||
context_data=context_data,
|
||||
actor_state=actor_state,
|
||||
relevant_resources=relevant_resources,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"Recording invariant enforced decision",
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
)
|
||||
|
||||
try:
|
||||
decision = self.decision_service.record_decision(
|
||||
plan_id=self.plan_id,
|
||||
decision_type=DecisionType.INVARIANT_ENFORCED,
|
||||
question=question,
|
||||
chosen_option=chosen_option,
|
||||
parent_decision_id=self.parent_decision_id,
|
||||
alternatives_considered=alternatives_considered,
|
||||
confidence_score=confidence_score,
|
||||
rationale=rationale,
|
||||
context_snapshot=snapshot,
|
||||
plan_phase=PlanPhase.STRATEGIZE,
|
||||
)
|
||||
self._logger.debug(
|
||||
"Invariant enforced decision recorded",
|
||||
decision_id=decision.decision_id,
|
||||
)
|
||||
return decision
|
||||
except Exception as exc:
|
||||
self._logger.warning(
|
||||
"Failed to record invariant enforced decision",
|
||||
error=str(exc),
|
||||
error_type=type(exc).__name__,
|
||||
)
|
||||
raise
|
||||
@@ -241,10 +241,9 @@ class PluginLoader:
|
||||
def validate_protocol(klass: type[Any], protocol: type[Any]) -> bool:
|
||||
"""Check whether *klass* satisfies a ``@runtime_checkable`` Protocol.
|
||||
|
||||
Creates a temporary instance of *klass* (using a no-arg
|
||||
constructor) and checks it against the protocol via
|
||||
``isinstance``. If instantiation fails, falls back to a
|
||||
structural check using ``issubclass``.
|
||||
Uses structural type checking via ``issubclass`` to validate the
|
||||
protocol without instantiating the class. This prevents arbitrary
|
||||
code execution from untrusted plugin constructors during validation.
|
||||
|
||||
Args:
|
||||
klass: The class to validate.
|
||||
@@ -256,23 +255,84 @@ class PluginLoader:
|
||||
Raises:
|
||||
ProtocolMismatchError: If *klass* does not satisfy the protocol.
|
||||
"""
|
||||
# Try instance check first (most reliable for runtime_checkable)
|
||||
# Try structural check first (safe, no instantiation).
|
||||
# This prevents arbitrary code execution from plugin constructors.
|
||||
issubclass_raised_type_error = False
|
||||
try:
|
||||
instance = klass()
|
||||
if isinstance(instance, protocol):
|
||||
if issubclass(klass, protocol):
|
||||
return True
|
||||
except Exception:
|
||||
# If instantiation fails, try subclass check
|
||||
try:
|
||||
if issubclass(klass, protocol):
|
||||
return True
|
||||
except TypeError:
|
||||
pass
|
||||
except TypeError:
|
||||
# issubclass raises TypeError if protocol is not a valid
|
||||
# @runtime_checkable Protocol (e.g. has a custom metaclass that
|
||||
# overrides __subclasscheck__). Fall through to structural check.
|
||||
issubclass_raised_type_error = True
|
||||
|
||||
# Fallback: perform a conservative structural check against the
|
||||
# Protocol definition without instantiating the class. We inspect
|
||||
# the protocol's declared members (callables and annotated names)
|
||||
# and ensure the candidate class exposes them as class-level
|
||||
# attributes or descriptors. This approximates structural
|
||||
# conformance while avoiding constructor execution.
|
||||
required: set[str] = set()
|
||||
|
||||
# Collect names from protocol __dict__ (methods, properties, etc.)
|
||||
prot_dict = getattr(protocol, "__dict__", {})
|
||||
for name, _value in prot_dict.items():
|
||||
if name.startswith("__"):
|
||||
continue
|
||||
# Methods and descriptors will appear in the dict; annotations
|
||||
# are handled below. We treat any non-data-magic name as a
|
||||
# required member.
|
||||
required.add(name)
|
||||
|
||||
# Also include annotated names (those declared with type hints)
|
||||
ann = getattr(protocol, "__annotations__", {}) or {}
|
||||
for name in ann:
|
||||
if name.startswith("__"):
|
||||
continue
|
||||
required.add(name)
|
||||
|
||||
# If issubclass raised TypeError and the protocol declares no
|
||||
# inspectable members, we cannot validate conformance — treat this
|
||||
# as a mismatch rather than silently returning True for an
|
||||
# unverifiable protocol.
|
||||
if issubclass_raised_type_error and not required:
|
||||
msg = (
|
||||
f"Class '{getattr(klass, '__name__', str(klass))}' cannot be "
|
||||
f"validated against protocol "
|
||||
f"'{getattr(protocol, '__name__', str(protocol))}': "
|
||||
f"issubclass raised TypeError and the protocol declares no "
|
||||
f"inspectable members. Ensure the protocol is a valid "
|
||||
f"@runtime_checkable Protocol."
|
||||
)
|
||||
raise ProtocolMismatchError(msg)
|
||||
|
||||
# Validate that the candidate class exposes each required name.
|
||||
missing: list[str] = []
|
||||
for name in sorted(required):
|
||||
# getattr on the class returns descriptors (functions, property,
|
||||
# staticmethod, etc.) without instantiation. hasattr is safe.
|
||||
if not hasattr(klass, name):
|
||||
missing.append(name)
|
||||
continue
|
||||
# If the protocol declared a callable (method), ensure the
|
||||
# attribute on the class is callable or a descriptor.
|
||||
prot_val = prot_dict.get(name)
|
||||
if callable(prot_val):
|
||||
cand = getattr(klass, name)
|
||||
# Properties are descriptors and typically not callable; we
|
||||
# accept them as present. For methods, require callable.
|
||||
if not (callable(cand) or hasattr(cand, "fget")):
|
||||
missing.append(name)
|
||||
|
||||
if not missing:
|
||||
return True
|
||||
|
||||
msg = (
|
||||
f"Class '{klass.__name__}' does not satisfy protocol "
|
||||
f"'{protocol.__name__}'. Ensure the class implements all "
|
||||
f"required methods and attributes."
|
||||
f"Class '{getattr(klass, '__name__', str(klass))}' does not "
|
||||
f"satisfy protocol '{getattr(protocol, '__name__', str(protocol))}'. "
|
||||
f"Missing attributes: {', '.join(missing)}. Ensure the class "
|
||||
f"implements all required methods and descriptors."
|
||||
)
|
||||
raise ProtocolMismatchError(msg)
|
||||
|
||||
|
||||
@@ -17,13 +17,12 @@ import yaml
|
||||
|
||||
from cleveragents.actor.registry import ActorRegistry
|
||||
from cleveragents.actor.schema import ActorConfigSchema, ActorType, is_v3_yaml
|
||||
from cleveragents.config.settings import ProviderDefaults, Settings
|
||||
from cleveragents.config.settings import ProviderDefaults
|
||||
from cleveragents.core.exceptions import NotFoundError
|
||||
from cleveragents.domain.models.core.actor import Actor
|
||||
from cleveragents.providers.registry import (
|
||||
ProviderCapabilities,
|
||||
ProviderInfo,
|
||||
ProviderRegistry,
|
||||
ProviderType,
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user