From f1ab5d90dca372f5c6a2cd37ea3d42e0d81c9023 Mon Sep 17 00:00:00 2001 From: Jeffrey Phillips Freeman Date: Sun, 5 Apr 2026 19:33:19 +0000 Subject: [PATCH] docs: update documentation for v3.8.0 unreleased features - Promote [Unreleased] CHANGELOG entries to [3.8.0] (2026-04-05) - Add Shell Danger Detection section to docs/api/tui.md covering ShellDangerLevel, DangerousPattern, ShellSafetyService, and SafetyCheckResult with full API reference and usage examples - Add InvariantService section to docs/api/core.md documenting the new DI-registered singleton, its methods, and emitted events - Update docs/architecture.md Plan Lifecycle section to document invariant reconciliation as a phase transition gate - Update README.md Highlights with shell danger detection, inline permission questions, invariant reconciliation, UKO provenance tracking, and JSON-RPC 2.0 A2A wire format - Create docs/modules/shell-safety.md with full module guide covering purpose, key classes, built-in patterns, custom pattern registration, TUI integration, and testing guidance ISSUES CLOSED: #1003 #997 #1391 #1004 #891 #1501 #1577 #1941 #2334 --- CHANGELOG.md | 4 + README.md | 11 +++ docs/api/core.md | 55 +++++++++++++ docs/api/tui.md | 102 +++++++++++++++++++++++ docs/architecture.md | 4 + docs/modules/shell-safety.md | 154 +++++++++++++++++++++++++++++++++++ 6 files changed, 330 insertions(+) create mode 100644 docs/modules/shell-safety.md diff --git a/CHANGELOG.md b/CHANGELOG.md index b8d8162ae..8f4128976 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +--- + +## [3.8.0] — 2026-04-05 + ### Added - Wired Invariant Reconciliation Actor auto-invocation into diff --git a/README.md b/README.md index 187826fe0..733986eda 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,17 @@ embracing modern Python tooling. PostgreSQL, MySQL, and DuckDB resources - **Estimation lifecycle** — `actor.default.estimation` config key wires an estimation actor into the Strategize-to-Estimate lifecycle hook +- **Shell danger detection** — TUI shell mode (`!` prefix) classifies commands by danger + level (LOW → CRITICAL) and surfaces a warning overlay before executing destructive, + privilege-escalating, or exfiltration-risk commands +- **Inline permission questions** — `PermissionQuestionWidget` renders single-file + permission requests directly in the conversation stream with single-key shortcuts +- **Invariant reconciliation** — `InvariantReconciliationActor` runs automatically at + every plan phase transition; failures block the transition and emit `INVARIANT_VIOLATED` +- **UKO provenance tracking** — every typed triple now carries `sourceResource`, + `validFrom`, and `isCurrent` metadata; a revision chain enables temporal queries +- **JSON-RPC 2.0 A2A wire format** — `A2aRequest`/`A2aResponse` fields renamed to + standard JSON-RPC 2.0 names (`method`, `id`, `result`, `error`) - Fast Typer/Click-based interface with parity for help/version behavior - Behavior-driven coverage via Behave and Robot Framework - Nox automation for linting, typing, testing, docs, builds, and benchmarks diff --git a/docs/api/core.md b/docs/api/core.md index 23d7e0b4b..98788fe32 100644 --- a/docs/api/core.md +++ b/docs/api/core.md @@ -227,3 +227,58 @@ async with AsyncResourceTracker() as tracker: Implements the circuit-breaker pattern to prevent cascading failures when calling external services. + +--- + +## Invariant Service + +**Module:** `cleveragents.application.services.invariant_service` + +The `InvariantService` manages invariant constraints across scopes and is +automatically invoked at every plan phase transition by the +`InvariantReconciliationActor`. + +### `InvariantService` + +```python +from cleveragents.application.services.invariant_service import InvariantService +from cleveragents.domain.models.core.invariant import InvariantScope + +service = InvariantService(event_bus=event_bus) +inv = service.add_invariant( + text="All output files must be UTF-8 encoded", + scope=InvariantScope.PROJECT, + source_name="my-project", +) +``` + +**Constructor parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `event_bus` | `EventBus \| None` | `None` | Optional event bus for emitting `INVARIANT_VIOLATED` / `INVARIANT_ENFORCED` / `INVARIANT_RECONCILED` events | + +**Methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `add_invariant(text, scope, source_name)` | `Invariant` | Add a new invariant; validates and sanitizes text | +| `list_invariants(scope, source_name, effective)` | `list[Invariant]` | Filter invariants; pass `effective=True` to get the merged precedence chain | +| `remove_invariant(invariant_id)` | `Invariant` | Soft-delete an invariant (sets `active=False`) | +| `get_effective_invariants(plan_id, project_name)` | `list[Invariant]` | Return merged invariants using plan > project > global precedence | +| `enforce_invariants(plan_id, invariants, actor_response, violated_invariant_ids)` | `list[InvariantEnforcementRecord]` | Create enforcement records; emits `INVARIANT_VIOLATED` for each violation | + +**Merge precedence:** plan > project > global. Duplicate invariant texts are +de-duplicated by `merge_invariants()`. + +**Events emitted:** + +| Event | When | +|-------|------| +| `INVARIANT_VIOLATED` | An invariant in `violated_invariant_ids` was not satisfied | +| `INVARIANT_ENFORCED` | Each invariant enforcement record is created | +| `INVARIANT_RECONCILED` | Once per `enforce_invariants()` call (batch summary) | + +> **Note:** The `InvariantService` is registered as a Singleton provider in +> the DI container. Obtain it via `container.invariant_service()` rather +> than constructing it directly. diff --git a/docs/api/tui.md b/docs/api/tui.md index 7db1f0acc..97618ce09 100644 --- a/docs/api/tui.md +++ b/docs/api/tui.md @@ -269,6 +269,108 @@ from cleveragents.tui.screens import PermissionsScreen --- +## Shell Danger Detection + +**Module:** `cleveragents.tui.shell_safety` + +The TUI shell mode (`!` prefix) checks commands against a configurable pattern +registry before execution. Commands that match a dangerous pattern surface a +warning overlay; commands at or above the configured `block_level` are blocked +automatically unless a `warn_callback` is provided. + +### `ShellDangerLevel` + +```python +from cleveragents.tui.shell_safety import ShellDangerLevel +``` + +`IntEnum` with four severity levels (ordered least to most severe): + +| Level | Value | Description | +|-------|-------|-------------| +| `LOW` | 1 | Minor risk — generally recoverable (e.g. `chmod 777` on a single file) | +| `MEDIUM` | 2 | Moderate risk — can cause data loss or security exposure (e.g. `sudo rm`, `wget \| sh`) | +| `HIGH` | 3 | High risk — significant, hard-to-reverse damage (e.g. `dd if=`, `mkfs`) | +| `CRITICAL` | 4 | Critical risk — can destroy the system or create a fork bomb (e.g. `rm -rf /`, `:(){ :\|:& };:`) | + +Numeric comparisons work naturally: `level >= ShellDangerLevel.HIGH`. + +--- + +### `DangerousPattern` + +```python +from cleveragents.tui.shell_safety import DangerousPattern + +pattern = DangerousPattern( + name="rm_rf_root", + pattern=r"rm\s+-[rRfF]*\s+/", + level=ShellDangerLevel.CRITICAL, + description="Recursively deletes the root filesystem.", +) +``` + +Immutable frozen dataclass describing a single dangerous shell pattern. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `name` | `str` | Short human-readable identifier | +| `pattern` | `str` | Regex matched against the command text | +| `level` | `ShellDangerLevel` | Severity classification | +| `description` | `str` | Human-readable explanation of the risk | +| `case_sensitive` | `bool` | When `True`, regex is compiled without `re.IGNORECASE` (default: `False`) | + +**`matches(command: str) → bool`** — returns `True` if `command` matches the compiled pattern. + +--- + +### `ShellSafetyService` + +```python +from cleveragents.tui.shell_safety import ShellSafetyService, ShellDangerLevel + +service = ShellSafetyService(block_level=ShellDangerLevel.MEDIUM) +result = service.check_command("rm -rf /") +if not result.allowed: + print(result.warning.message) +``` + +Application service that checks shell commands before execution. + +**Constructor parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `detector` | `DangerousPatternDetector \| None` | `None` | Custom detector; a default one is created when `None` | +| `block_level` | `ShellDangerLevel` | `MEDIUM` | Minimum level that triggers automatic blocking when no callback is provided | +| `warn_callback` | `Callable[[DangerousCommandWarning], bool] \| None` | `None` | Optional callback invoked for dangerous commands; return `True` to allow, `False` to block | +| `extra_patterns` | `list[DangerousPattern] \| None` | `None` | Additional patterns registered on top of the defaults | + +**Methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `check_command(command)` | `SafetyCheckResult` | Check a command and return a structured result | +| `is_safe(command)` | `bool` | Convenience wrapper — returns `True` if the command passes | + +**`SafetyCheckResult`** attributes: + +| Attribute | Type | Description | +|-----------|------|-------------| +| `command` | `str` | The command that was checked | +| `warning` | `DangerousCommandWarning \| None` | Warning if a dangerous pattern matched, otherwise `None` | +| `allowed` | `bool` | `True` if the command is allowed to proceed | + +**Built-in pattern categories:** + +- Destructive filesystem operations (`rm -rf`, `shred`, `wipe`) +- Privilege escalation (`sudo`, `su`, `chmod 777 /`) +- Network exfiltration (`curl \| sh`, `wget \| bash`, `nc` reverse shells) +- Fork bombs and resource exhaustion +- Disk-level writes (`dd if=`, `mkfs`) + +--- + ## TUI State Persistence The TUI persists minimal state to `~/.config/cleveragents/tui-state.yaml`: diff --git a/docs/architecture.md b/docs/architecture.md index 126cef70c..fd240f5bc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -118,6 +118,10 @@ Phase transitions are gated by: - **Validation pipeline** — rule-based checks with severity levels - **Definition-of-Done** — explicit criteria evaluated before Apply - **Advisory locking** — prevents concurrent modification of the same plan +- **Invariant reconciliation** — `InvariantReconciliationActor` runs automatically + at each phase transition (`start_strategize`, `execute_plan`, `apply_plan`); + failures raise `ReconciliationBlockedError` and emit `INVARIANT_VIOLATED` events, + blocking the transition until invariants are satisfied --- diff --git a/docs/modules/shell-safety.md b/docs/modules/shell-safety.md new file mode 100644 index 000000000..e50f0a953 --- /dev/null +++ b/docs/modules/shell-safety.md @@ -0,0 +1,154 @@ +# Shell Safety Module + +**Package:** `cleveragents.tui.shell_safety` + +The shell safety module protects users from accidentally executing dangerous +commands in the TUI shell mode (`!` prefix). It provides a configurable +pattern registry, a danger-level classification system, and an application +service that integrates with the TUI warning overlay. + +--- + +## Purpose + +When a user types `!` in the TUI, the `ShellSafetyService` checks +the command against a registry of known-dangerous patterns before the +subprocess is spawned. Commands that match a pattern at or above the +configured `block_level` are blocked; commands below that level surface a +warning overlay that the user must acknowledge before proceeding. + +--- + +## Key Classes + +### `ShellDangerLevel` + +Four-level `IntEnum` severity scale: + +``` +LOW (1) < MEDIUM (2) < HIGH (3) < CRITICAL (4) +``` + +Numeric comparisons work naturally: `level >= ShellDangerLevel.HIGH`. + +### `DangerousPattern` + +Immutable frozen dataclass describing a single dangerous pattern: + +```python +DangerousPattern( + name="fork_bomb", + pattern=r":\(\)\s*\{", + level=ShellDangerLevel.CRITICAL, + description="Fork bomb — exhausts process table and crashes the system.", +) +``` + +### `DangerousPatternDetector` + +Maintains the registry of `DangerousPattern` objects and provides +`check_first(command) → DangerousCommandWarning | None` to return the +highest-severity matching pattern. + +### `ShellSafetyService` + +High-level application service used by the TUI: + +```python +from cleveragents.tui.shell_safety import ShellSafetyService, ShellDangerLevel + +# Default: block MEDIUM and above automatically +service = ShellSafetyService() + +# Custom: only block CRITICAL; show overlay for HIGH and below +service = ShellSafetyService(block_level=ShellDangerLevel.CRITICAL) + +# Custom: delegate all decisions to a callback +def my_callback(warning): + # Return True to allow, False to block + return warning.danger_level < ShellDangerLevel.HIGH + +service = ShellSafetyService(warn_callback=my_callback) + +result = service.check_command("rm -rf /") +if not result.allowed: + print(f"Blocked: {result.warning.description}") +``` + +### `DangerousCommandWarning` + +Value object returned when a dangerous pattern is matched: + +| Attribute | Type | Description | +|-----------|------|-------------| +| `command` | `str` | The original command text | +| `pattern` | `DangerousPattern` | The matched pattern | +| `danger_level` | `ShellDangerLevel` | Convenience alias for `pattern.level` | +| `message` | `str` | Human-readable warning message | + +--- + +## Built-in Pattern Categories + +| Category | Example patterns | Default level | +|----------|-----------------|---------------| +| Recursive deletion | `rm -rf /`, `rm -rf ~` | CRITICAL | +| Disk-level writes | `dd if=`, `mkfs`, `shred` | CRITICAL | +| Fork bombs | `:(){ :|:& };:` | CRITICAL | +| Privilege escalation | `sudo su`, `chmod 777 /` | HIGH | +| Network exfiltration | `curl \| sh`, `wget \| bash` | HIGH | +| Reverse shells | `nc -e /bin/sh`, `bash -i >& /dev/tcp` | HIGH | +| Dangerous redirects | `> /dev/sda`, `> /etc/passwd` | HIGH | +| Credential exposure | `env \| curl`, `cat ~/.ssh/id_rsa \| nc` | MEDIUM | + +--- + +## Adding Custom Patterns + +```python +from cleveragents.tui.shell_safety import ( + DangerousPattern, + ShellDangerLevel, + ShellSafetyService, +) + +custom = DangerousPattern( + name="drop_database", + pattern=r"DROP\s+DATABASE", + level=ShellDangerLevel.HIGH, + description="Drops an entire database — irreversible.", + case_sensitive=False, +) + +service = ShellSafetyService(extra_patterns=[custom]) +``` + +--- + +## Integration with the TUI + +The TUI wires `ShellSafetyService` into the shell input handler. When +`check_command()` returns `allowed=False`, the TUI displays a +`DangerousCommandWarningOverlay` widget with the warning message and danger +level. The user must explicitly confirm before the command is executed. + +For commands that are blocked outright (above `block_level` with no +callback), the overlay displays a non-dismissable error and the command is +never executed. + +--- + +## Testing + +The shell safety module is fully covered by BDD scenarios in +`features/shell_safety.feature`. All pattern matching is deterministic and +does not require a running TUI, making it straightforward to test in +isolation: + +```python +from cleveragents.tui.shell_safety import ShellSafetyService + +service = ShellSafetyService() +assert not service.is_safe("rm -rf /") +assert service.is_safe("ls -la") +```