docs: update documentation for v3.8.0 unreleased features
CI / unit_tests (push) Waiting to run
CI / benchmark-publish (push) Waiting to run
CI / build (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / helm (push) Waiting to run
CI / status-check (push) Blocked by required conditions
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / security (push) Waiting to run
CI / quality (push) Waiting to run
CI / integration_tests (push) Waiting to run
CI / e2e_tests (push) Waiting to run
CI / coverage (push) Blocked by required conditions
CI / benchmark-regression (push) Blocked by required conditions
CI / unit_tests (push) Waiting to run
CI / benchmark-publish (push) Waiting to run
CI / build (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / helm (push) Waiting to run
CI / status-check (push) Blocked by required conditions
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / security (push) Waiting to run
CI / quality (push) Waiting to run
CI / integration_tests (push) Waiting to run
CI / e2e_tests (push) Waiting to run
CI / coverage (push) Blocked by required conditions
CI / benchmark-regression (push) Blocked by required conditions
- Promote [Unreleased] CHANGELOG entries to [3.8.0] (2026-04-05) - Add Shell Danger Detection section to docs/api/tui.md covering ShellDangerLevel, DangerousPattern, ShellSafetyService, and SafetyCheckResult with full API reference and usage examples - Add InvariantService section to docs/api/core.md documenting the new DI-registered singleton, its methods, and emitted events - Update docs/architecture.md Plan Lifecycle section to document invariant reconciliation as a phase transition gate - Update README.md Highlights with shell danger detection, inline permission questions, invariant reconciliation, UKO provenance tracking, and JSON-RPC 2.0 A2A wire format - Create docs/modules/shell-safety.md with full module guide covering purpose, key classes, built-in patterns, custom pattern registration, TUI integration, and testing guidance ISSUES CLOSED: #1003 #997 #1391 #1004 #891 #1501 #1577 #1941 #2334
This commit is contained in:
@@ -5,6 +5,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
---
|
||||
|
||||
## [3.8.0] — 2026-04-05
|
||||
|
||||
### Added
|
||||
|
||||
- Wired Invariant Reconciliation Actor auto-invocation into
|
||||
|
||||
@@ -30,6 +30,17 @@ embracing modern Python tooling.
|
||||
PostgreSQL, MySQL, and DuckDB resources
|
||||
- **Estimation lifecycle** — `actor.default.estimation` config key wires an estimation
|
||||
actor into the Strategize-to-Estimate lifecycle hook
|
||||
- **Shell danger detection** — TUI shell mode (`!` prefix) classifies commands by danger
|
||||
level (LOW → CRITICAL) and surfaces a warning overlay before executing destructive,
|
||||
privilege-escalating, or exfiltration-risk commands
|
||||
- **Inline permission questions** — `PermissionQuestionWidget` renders single-file
|
||||
permission requests directly in the conversation stream with single-key shortcuts
|
||||
- **Invariant reconciliation** — `InvariantReconciliationActor` runs automatically at
|
||||
every plan phase transition; failures block the transition and emit `INVARIANT_VIOLATED`
|
||||
- **UKO provenance tracking** — every typed triple now carries `sourceResource`,
|
||||
`validFrom`, and `isCurrent` metadata; a revision chain enables temporal queries
|
||||
- **JSON-RPC 2.0 A2A wire format** — `A2aRequest`/`A2aResponse` fields renamed to
|
||||
standard JSON-RPC 2.0 names (`method`, `id`, `result`, `error`)
|
||||
- Fast Typer/Click-based interface with parity for help/version behavior
|
||||
- Behavior-driven coverage via Behave and Robot Framework
|
||||
- Nox automation for linting, typing, testing, docs, builds, and benchmarks
|
||||
|
||||
@@ -227,3 +227,58 @@ async with AsyncResourceTracker() as tracker:
|
||||
|
||||
Implements the circuit-breaker pattern to prevent cascading failures when
|
||||
calling external services.
|
||||
|
||||
---
|
||||
|
||||
## Invariant Service
|
||||
|
||||
**Module:** `cleveragents.application.services.invariant_service`
|
||||
|
||||
The `InvariantService` manages invariant constraints across scopes and is
|
||||
automatically invoked at every plan phase transition by the
|
||||
`InvariantReconciliationActor`.
|
||||
|
||||
### `InvariantService`
|
||||
|
||||
```python
|
||||
from cleveragents.application.services.invariant_service import InvariantService
|
||||
from cleveragents.domain.models.core.invariant import InvariantScope
|
||||
|
||||
service = InvariantService(event_bus=event_bus)
|
||||
inv = service.add_invariant(
|
||||
text="All output files must be UTF-8 encoded",
|
||||
scope=InvariantScope.PROJECT,
|
||||
source_name="my-project",
|
||||
)
|
||||
```
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `event_bus` | `EventBus \| None` | `None` | Optional event bus for emitting `INVARIANT_VIOLATED` / `INVARIANT_ENFORCED` / `INVARIANT_RECONCILED` events |
|
||||
|
||||
**Methods:**
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `add_invariant(text, scope, source_name)` | `Invariant` | Add a new invariant; validates and sanitizes text |
|
||||
| `list_invariants(scope, source_name, effective)` | `list[Invariant]` | Filter invariants; pass `effective=True` to get the merged precedence chain |
|
||||
| `remove_invariant(invariant_id)` | `Invariant` | Soft-delete an invariant (sets `active=False`) |
|
||||
| `get_effective_invariants(plan_id, project_name)` | `list[Invariant]` | Return merged invariants using plan > project > global precedence |
|
||||
| `enforce_invariants(plan_id, invariants, actor_response, violated_invariant_ids)` | `list[InvariantEnforcementRecord]` | Create enforcement records; emits `INVARIANT_VIOLATED` for each violation |
|
||||
|
||||
**Merge precedence:** plan > project > global. Duplicate invariant texts are
|
||||
de-duplicated by `merge_invariants()`.
|
||||
|
||||
**Events emitted:**
|
||||
|
||||
| Event | When |
|
||||
|-------|------|
|
||||
| `INVARIANT_VIOLATED` | An invariant in `violated_invariant_ids` was not satisfied |
|
||||
| `INVARIANT_ENFORCED` | Each invariant enforcement record is created |
|
||||
| `INVARIANT_RECONCILED` | Once per `enforce_invariants()` call (batch summary) |
|
||||
|
||||
> **Note:** The `InvariantService` is registered as a Singleton provider in
|
||||
> the DI container. Obtain it via `container.invariant_service()` rather
|
||||
> than constructing it directly.
|
||||
|
||||
+102
@@ -269,6 +269,108 @@ from cleveragents.tui.screens import PermissionsScreen
|
||||
|
||||
---
|
||||
|
||||
## Shell Danger Detection
|
||||
|
||||
**Module:** `cleveragents.tui.shell_safety`
|
||||
|
||||
The TUI shell mode (`!` prefix) checks commands against a configurable pattern
|
||||
registry before execution. Commands that match a dangerous pattern surface a
|
||||
warning overlay; commands at or above the configured `block_level` are blocked
|
||||
automatically unless a `warn_callback` is provided.
|
||||
|
||||
### `ShellDangerLevel`
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import ShellDangerLevel
|
||||
```
|
||||
|
||||
`IntEnum` with four severity levels (ordered least to most severe):
|
||||
|
||||
| Level | Value | Description |
|
||||
|-------|-------|-------------|
|
||||
| `LOW` | 1 | Minor risk — generally recoverable (e.g. `chmod 777` on a single file) |
|
||||
| `MEDIUM` | 2 | Moderate risk — can cause data loss or security exposure (e.g. `sudo rm`, `wget \| sh`) |
|
||||
| `HIGH` | 3 | High risk — significant, hard-to-reverse damage (e.g. `dd if=`, `mkfs`) |
|
||||
| `CRITICAL` | 4 | Critical risk — can destroy the system or create a fork bomb (e.g. `rm -rf /`, `:(){ :\|:& };:`) |
|
||||
|
||||
Numeric comparisons work naturally: `level >= ShellDangerLevel.HIGH`.
|
||||
|
||||
---
|
||||
|
||||
### `DangerousPattern`
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import DangerousPattern
|
||||
|
||||
pattern = DangerousPattern(
|
||||
name="rm_rf_root",
|
||||
pattern=r"rm\s+-[rRfF]*\s+/",
|
||||
level=ShellDangerLevel.CRITICAL,
|
||||
description="Recursively deletes the root filesystem.",
|
||||
)
|
||||
```
|
||||
|
||||
Immutable frozen dataclass describing a single dangerous shell pattern.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `name` | `str` | Short human-readable identifier |
|
||||
| `pattern` | `str` | Regex matched against the command text |
|
||||
| `level` | `ShellDangerLevel` | Severity classification |
|
||||
| `description` | `str` | Human-readable explanation of the risk |
|
||||
| `case_sensitive` | `bool` | When `True`, regex is compiled without `re.IGNORECASE` (default: `False`) |
|
||||
|
||||
**`matches(command: str) → bool`** — returns `True` if `command` matches the compiled pattern.
|
||||
|
||||
---
|
||||
|
||||
### `ShellSafetyService`
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import ShellSafetyService, ShellDangerLevel
|
||||
|
||||
service = ShellSafetyService(block_level=ShellDangerLevel.MEDIUM)
|
||||
result = service.check_command("rm -rf /")
|
||||
if not result.allowed:
|
||||
print(result.warning.message)
|
||||
```
|
||||
|
||||
Application service that checks shell commands before execution.
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `detector` | `DangerousPatternDetector \| None` | `None` | Custom detector; a default one is created when `None` |
|
||||
| `block_level` | `ShellDangerLevel` | `MEDIUM` | Minimum level that triggers automatic blocking when no callback is provided |
|
||||
| `warn_callback` | `Callable[[DangerousCommandWarning], bool] \| None` | `None` | Optional callback invoked for dangerous commands; return `True` to allow, `False` to block |
|
||||
| `extra_patterns` | `list[DangerousPattern] \| None` | `None` | Additional patterns registered on top of the defaults |
|
||||
|
||||
**Methods:**
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `check_command(command)` | `SafetyCheckResult` | Check a command and return a structured result |
|
||||
| `is_safe(command)` | `bool` | Convenience wrapper — returns `True` if the command passes |
|
||||
|
||||
**`SafetyCheckResult`** attributes:
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `command` | `str` | The command that was checked |
|
||||
| `warning` | `DangerousCommandWarning \| None` | Warning if a dangerous pattern matched, otherwise `None` |
|
||||
| `allowed` | `bool` | `True` if the command is allowed to proceed |
|
||||
|
||||
**Built-in pattern categories:**
|
||||
|
||||
- Destructive filesystem operations (`rm -rf`, `shred`, `wipe`)
|
||||
- Privilege escalation (`sudo`, `su`, `chmod 777 /`)
|
||||
- Network exfiltration (`curl \| sh`, `wget \| bash`, `nc` reverse shells)
|
||||
- Fork bombs and resource exhaustion
|
||||
- Disk-level writes (`dd if=`, `mkfs`)
|
||||
|
||||
---
|
||||
|
||||
## TUI State Persistence
|
||||
|
||||
The TUI persists minimal state to `~/.config/cleveragents/tui-state.yaml`:
|
||||
|
||||
@@ -118,6 +118,10 @@ Phase transitions are gated by:
|
||||
- **Validation pipeline** — rule-based checks with severity levels
|
||||
- **Definition-of-Done** — explicit criteria evaluated before Apply
|
||||
- **Advisory locking** — prevents concurrent modification of the same plan
|
||||
- **Invariant reconciliation** — `InvariantReconciliationActor` runs automatically
|
||||
at each phase transition (`start_strategize`, `execute_plan`, `apply_plan`);
|
||||
failures raise `ReconciliationBlockedError` and emit `INVARIANT_VIOLATED` events,
|
||||
blocking the transition until invariants are satisfied
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
# Shell Safety Module
|
||||
|
||||
**Package:** `cleveragents.tui.shell_safety`
|
||||
|
||||
The shell safety module protects users from accidentally executing dangerous
|
||||
commands in the TUI shell mode (`!` prefix). It provides a configurable
|
||||
pattern registry, a danger-level classification system, and an application
|
||||
service that integrates with the TUI warning overlay.
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
When a user types `!<command>` in the TUI, the `ShellSafetyService` checks
|
||||
the command against a registry of known-dangerous patterns before the
|
||||
subprocess is spawned. Commands that match a pattern at or above the
|
||||
configured `block_level` are blocked; commands below that level surface a
|
||||
warning overlay that the user must acknowledge before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## Key Classes
|
||||
|
||||
### `ShellDangerLevel`
|
||||
|
||||
Four-level `IntEnum` severity scale:
|
||||
|
||||
```
|
||||
LOW (1) < MEDIUM (2) < HIGH (3) < CRITICAL (4)
|
||||
```
|
||||
|
||||
Numeric comparisons work naturally: `level >= ShellDangerLevel.HIGH`.
|
||||
|
||||
### `DangerousPattern`
|
||||
|
||||
Immutable frozen dataclass describing a single dangerous pattern:
|
||||
|
||||
```python
|
||||
DangerousPattern(
|
||||
name="fork_bomb",
|
||||
pattern=r":\(\)\s*\{",
|
||||
level=ShellDangerLevel.CRITICAL,
|
||||
description="Fork bomb — exhausts process table and crashes the system.",
|
||||
)
|
||||
```
|
||||
|
||||
### `DangerousPatternDetector`
|
||||
|
||||
Maintains the registry of `DangerousPattern` objects and provides
|
||||
`check_first(command) → DangerousCommandWarning | None` to return the
|
||||
highest-severity matching pattern.
|
||||
|
||||
### `ShellSafetyService`
|
||||
|
||||
High-level application service used by the TUI:
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import ShellSafetyService, ShellDangerLevel
|
||||
|
||||
# Default: block MEDIUM and above automatically
|
||||
service = ShellSafetyService()
|
||||
|
||||
# Custom: only block CRITICAL; show overlay for HIGH and below
|
||||
service = ShellSafetyService(block_level=ShellDangerLevel.CRITICAL)
|
||||
|
||||
# Custom: delegate all decisions to a callback
|
||||
def my_callback(warning):
|
||||
# Return True to allow, False to block
|
||||
return warning.danger_level < ShellDangerLevel.HIGH
|
||||
|
||||
service = ShellSafetyService(warn_callback=my_callback)
|
||||
|
||||
result = service.check_command("rm -rf /")
|
||||
if not result.allowed:
|
||||
print(f"Blocked: {result.warning.description}")
|
||||
```
|
||||
|
||||
### `DangerousCommandWarning`
|
||||
|
||||
Value object returned when a dangerous pattern is matched:
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `command` | `str` | The original command text |
|
||||
| `pattern` | `DangerousPattern` | The matched pattern |
|
||||
| `danger_level` | `ShellDangerLevel` | Convenience alias for `pattern.level` |
|
||||
| `message` | `str` | Human-readable warning message |
|
||||
|
||||
---
|
||||
|
||||
## Built-in Pattern Categories
|
||||
|
||||
| Category | Example patterns | Default level |
|
||||
|----------|-----------------|---------------|
|
||||
| Recursive deletion | `rm -rf /`, `rm -rf ~` | CRITICAL |
|
||||
| Disk-level writes | `dd if=`, `mkfs`, `shred` | CRITICAL |
|
||||
| Fork bombs | `:(){ :|:& };:` | CRITICAL |
|
||||
| Privilege escalation | `sudo su`, `chmod 777 /` | HIGH |
|
||||
| Network exfiltration | `curl \| sh`, `wget \| bash` | HIGH |
|
||||
| Reverse shells | `nc -e /bin/sh`, `bash -i >& /dev/tcp` | HIGH |
|
||||
| Dangerous redirects | `> /dev/sda`, `> /etc/passwd` | HIGH |
|
||||
| Credential exposure | `env \| curl`, `cat ~/.ssh/id_rsa \| nc` | MEDIUM |
|
||||
|
||||
---
|
||||
|
||||
## Adding Custom Patterns
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import (
|
||||
DangerousPattern,
|
||||
ShellDangerLevel,
|
||||
ShellSafetyService,
|
||||
)
|
||||
|
||||
custom = DangerousPattern(
|
||||
name="drop_database",
|
||||
pattern=r"DROP\s+DATABASE",
|
||||
level=ShellDangerLevel.HIGH,
|
||||
description="Drops an entire database — irreversible.",
|
||||
case_sensitive=False,
|
||||
)
|
||||
|
||||
service = ShellSafetyService(extra_patterns=[custom])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration with the TUI
|
||||
|
||||
The TUI wires `ShellSafetyService` into the shell input handler. When
|
||||
`check_command()` returns `allowed=False`, the TUI displays a
|
||||
`DangerousCommandWarningOverlay` widget with the warning message and danger
|
||||
level. The user must explicitly confirm before the command is executed.
|
||||
|
||||
For commands that are blocked outright (above `block_level` with no
|
||||
callback), the overlay displays a non-dismissable error and the command is
|
||||
never executed.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
The shell safety module is fully covered by BDD scenarios in
|
||||
`features/shell_safety.feature`. All pattern matching is deterministic and
|
||||
does not require a running TUI, making it straightforward to test in
|
||||
isolation:
|
||||
|
||||
```python
|
||||
from cleveragents.tui.shell_safety import ShellSafetyService
|
||||
|
||||
service = ShellSafetyService()
|
||||
assert not service.is_safe("rm -rf /")
|
||||
assert service.is_safe("ls -la")
|
||||
```
|
||||
Reference in New Issue
Block a user