Files
HAL9000 98bc7c6b5d
CI / push-validation (push) Successful in 37s
CI / helm (push) Successful in 37s
CI / benchmark-publish (push) Failing after 44s
CI / build (push) Successful in 1m6s
CI / lint (push) Successful in 1m33s
CI / security (push) Successful in 1m53s
CI / quality (push) Successful in 1m59s
CI / typecheck (push) Successful in 2m0s
CI / integration_tests (push) Successful in 3m58s
CI / e2e_tests (push) Successful in 5m9s
CI / unit_tests (push) Successful in 6m30s
CI / docker (push) Successful in 1m29s
CI / coverage (push) Successful in 11m12s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 37s
CI / coverage (pull_request) Successful in 12m19s
CI / push-validation (pull_request) Successful in 22s
CI / lint (pull_request) Successful in 1m0s
CI / typecheck (pull_request) Successful in 1m19s
CI / quality (pull_request) Successful in 1m16s
CI / security (pull_request) Successful in 1m33s
CI / helm (pull_request) Successful in 27s
CI / e2e_tests (pull_request) Successful in 3m22s
CI / integration_tests (pull_request) Successful in 3m54s
CI / unit_tests (pull_request) Successful in 5m24s
CI / docker (pull_request) Successful in 1m56s
CI / status-check (pull_request) Successful in 3s
docs(spec): align checkpoint trigger names and config key path with implementation
- Update trigger names from 'on_tool_write' and 'on_tool_write_complete' to 'before_tool_execute' and 'after_tool_execute' to match the actual implementation in config_service.py
- Correct config key path from 'core.checkpoints.auto_create_on' to 'checkpoints.auto_create_on' to match the actual configuration structure
- Update TOML configuration examples to use comma-separated string format instead of array format, matching the actual ConfigService implementation
- Update references to trigger names in the CLI Usage section to use the correct names
2026-04-28 08:20:42 +00:00

7.2 KiB
Raw Permalink Blame History

Checkpointing and Rollback

Overview

CleverAgents supports checkpointing and rollback to allow operators to snapshot sandbox state during plan execution and restore it later. This is useful for recovering from mistakes, reverting tool side-effects, and supporting the decision-correction revert flow.

Concepts

Checkpoint

A Checkpoint is an immutable record of sandbox state at a point in time. Each checkpoint stores:

  • checkpoint_id — Unique ULID identifier
  • plan_id — The plan that owns this checkpoint
  • sandbox_ref — A reference to the sandbox state (e.g. git commit hash)
  • decision_id — Optional decision ULID this checkpoint is aligned to
  • checkpoint_type — One of pre_write, post_step, or manual
  • resource_id — Optional resource ULID association
  • filesystem_path — Relative path within the checkpoint directory
  • size_bytes — Size of the checkpoint data in bytes
  • created_at — Timestamp of creation (UTC)
  • metadata — Audit metadata including reason, source tool, and phase

Retention Policy

A CheckpointRetentionPolicy governs how many checkpoints a plan may keep. When auto_prune is enabled and the number of checkpoints exceeds max_checkpoints, the oldest interior checkpoints are automatically removed when a new one is created. The first (earliest) and most recent checkpoints are always preserved during pruning.

Defaults:

Field Default Range
max_checkpoints 50 1100
auto_prune true

Rollback

A rollback restores sandbox state to a previously captured checkpoint. The RollbackResult describes what changed:

  • restored_files_count — Number of files restored
  • changed_paths — Paths that were modified during rollback
  • from_checkpoint_id — The checkpoint that was restored

Guards

The rollback operation enforces two guards:

  1. Plan is applied — Once a plan reaches the applied terminal state, rollback is rejected with a BusinessRuleViolation.
  2. Sandbox is missing — If the sandbox has been cleaned up, rollback is rejected because there is nothing to restore.

Automatic Checkpoint Triggers (v3.8.0+)

The execution engine now supports four automatic checkpoint triggers, all configurable via checkpoints.auto_create_on (default: all enabled):

Trigger When Component
before_tool_execute Before each write-tool execution ToolRunner
after_tool_execute After each write-tool execution ToolRunner
on_subplan_spawn Before first subplan execution attempt SubplanExecutionService
on_error When the Execute phase fails PlanExecutor

Configuration

[checkpoints]
auto_create_on = "before_tool_execute,after_tool_execute,on_subplan_spawn,on_error"

To disable a specific trigger, remove it from the list. To disable all automatic checkpoints:

[checkpoints]
auto_create_on = ""

Wiring

The CheckpointService is injected as an optional parameter into:

  • ToolRunnercheckpoint_service and auto_checkpoint_triggers parameters
  • SubplanExecutionServicecheckpoint_service, auto_checkpoint_triggers, and parent_plan_id parameters
  • PlanExecutor — uses _is_auto_trigger_active() helper to check trigger config

When checkpoint_service is None, all automatic checkpointing is skipped (backward-compatible no-op).

CLI Usage

Create a checkpoint (programmatic)

Checkpoints are created automatically by the execution engine via the four triggers above, or manually via CheckpointService.create_checkpoint(). The CheckpointScope on each tool's capability declaration controls which tools trigger before_tool_execute / after_tool_execute checkpoints.

Rollback to a checkpoint

agents plan rollback [--yes|-y] <plan_id> <checkpoint_id>

The --yes / -y flag skips the interactive confirmation prompt.

JSON output envelope (when using --format json):

{
  "rollback_summary": {
    "plan_id": "...",
    "from_checkpoint_id": "...",
    "restored_files_count": 3
  },
  "changes_reverted": ["path/to/file1.py", "path/to/file2.py"],
  "impact": {
    "files_affected": 3
  },
  "post_rollback_state": {
    "active_checkpoint": "...",
    "plan_id": "..."
  },
  "timing": {
    "elapsed_seconds": 0.042
  },
  "messages": ["Rollback completed successfully."]
}

Error Cases

Error Cause
Rollback blocked: plan is already applied Plan has reached terminal applied state
Rollback blocked: sandbox is missing Sandbox was cleaned up before rollback
Not found: checkpoint Checkpoint ID does not exist
Validation Error Checkpoint does not belong to the specified plan

Correction Service Integration

The CorrectionService accepts an optional checkpoint_service parameter (typed as CheckpointService | None). This is an integration point for a future milestone: once wired, the correction revert flow will delegate sandbox restoration to the checkpoint rollback mechanism, enabling reuse of the same rollback logic for both explicit CLI rollback and decision-correction reverts. The delegation is not yet active in the current implementation; execute_revert() performs BFS impact analysis and marks decisions as reverted but does not invoke checkpoint_service.rollback_to_checkpoint().

Database Schema

The checkpoint_metadata table stores checkpoint metadata:

Column Type Description
checkpoint_id STRING(26) PK ULID primary key
plan_id STRING(26) FK References v3_plans.plan_id
decision_id STRING(26) Optional decision alignment
checkpoint_type TEXT pre_write, post_step, manual
resource_id STRING(26) Optional resource association
sandbox_ref TEXT Git commit hash or patch reference
filesystem_path TEXT Relative path within checkpoint dir
size_bytes INTEGER Size of checkpoint data in bytes
created_at STRING(30) ISO-8601 timestamp
metadata_json TEXT JSON: reason, source_tool, phase

Indexes: idx_checkpoints_plan, ix_checkpoint_metadata_created_at.

Dependency Injection

The CheckpointService is registered in the DI container (container.py) and backed by a CheckpointRepository that uses the session-factory pattern (ADR-007). The CLI rollback command obtains the service from the container via get_container().checkpoint_service().

The UnitOfWorkContext also exposes a checkpoints property for cross-repository atomicity within a single transaction.

Metadata for Auditability

Each checkpoint stores structured metadata to support audit trails:

  • reason — Why the checkpoint was created (e.g. "pre-execution")
  • source_tool — Which tool triggered the checkpoint (e.g. "lint-check")
  • phase — Which plan phase was active (e.g. "execute")
  • extra — Arbitrary key-value pairs for extension