freemo/cleveragents-core
0
0
Fork 0
forked from cleveragents/cleveragents-core
Code Pull Requests Activity
1,191 Commits 352 Branches 3 Tags
feature/m9-entity-sync
Commit Graph

495 Commits

Author SHA1 Message Date
freemo bb1dc17b08 feat(server): entity sync (_cleveragents/sync/*) 
Implement entity synchronization between client and server using
the _cleveragents/sync/* A2A extension methods per the specification.

Sync models (src/cleveragents/a2a/sync_models.py):
- Pydantic v2 models for pull/push/status requests and responses
- VectorClock with increment, merge, happens-before, and concurrency
  detection for distributed conflict detection
- SyncEntitySnapshot, SyncConflict, SyncState, SyncQueueEntry models
- SyncEntityType enum: actor, skill, action, plan, session
- ConflictResolution enum: last_writer_wins, manual, server_wins,
  client_wins

SyncService (src/cleveragents/application/services/sync_service.py):
- pull(): download server namespace entities to local cache with
  incremental sync (since timestamps) and force-overwrite option
- push(): push local entity definitions to server namespace with
  configurable conflict resolution strategy
- status(): compare local and server entity versions, detect drift,
  report local-ahead, server-ahead, and concurrent conflicts
- resolve_conflict(): manually resolve detected conflicts with
  winner selection (local or server)
- Offline queue: enqueue_offline() and process_offline_queue() for
  retry on reconnect, with max-retries enforcement
- The local/ namespace is never synced per specification

A2A facade (src/cleveragents/a2a/facade.py):
- Replaced sync stub handlers with real _handle_sync_pull,
  _handle_sync_push, _handle_sync_status routing to SyncService
- Added sync_service property and TYPE_CHECKING import
- Facade returns stubs when no SyncService is registered

Tests:
- Behave BDD: features/entity_sync.feature (65 scenarios, 247 steps)
  covering models, vector clocks, pull/push/status, conflict resolution,
  offline queue, facade integration, constructor validation, edge cases
- Robot Framework: robot/entity_sync.robot (8 integration tests)

ISSUES CLOSED: #866
 2026-03-23 22:50:56 +00:00
freemo 5f0a545113 feat(server): ASGI endpoint via uvicorn 
Implement FastAPI-based ASGI application served by uvicorn for
the CleverAgents server mode. Add health check endpoint, A2A
JSON-RPC routing, configurable host:port binding, and graceful
shutdown handling. Server launches via `agents server start`.

- Add FastAPI ASGI app factory at infrastructure/server/asgi_app.py
  with /.well-known/agent.json (Agent Card), /health (liveness),
  and /a2a (A2A JSON-RPC 2.0 dispatch via A2aLocalFacade)
- Add ServerLifecycle class at infrastructure/server/server_lifecycle.py
  wrapping uvicorn.Server with SIGTERM/SIGINT graceful shutdown
- Add `agents server start` CLI command with --host, --port, --log-level
  options, resolving defaults from Settings
- Add fastapi>=0.115.0 dependency to pyproject.toml (uvicorn already present)
- Add Behave BDD tests (features/server_lifecycle.feature, 20 scenarios)
- Add Robot Framework integration tests (robot/server_lifecycle.robot)
- Update CHANGELOG.md and vulture_whitelist.py

ISSUES CLOSED: #862
 2026-03-22 23:50:04 +00:00
brent.edwards 34c6972a05 feat(tool): implement BuiltinAdapter class and MCP automatic resource slot creation (#964) 
## Summary

Implement `BuiltinAdapter` class and MCP automatic resource slot creation. Two main changes:

### 1. BuiltinAdapter Class (`tool/builtins/adapter.py`)
Formal adapter implementing the tool adapter lifecycle pattern for built-in tools:
- `discover()` — returns all built-in tool descriptors (file, git, subplan tools)
- `register(registry)` — registers all tools in a ToolRegistry, returns registered names
- `activate()`/`deactivate()` — no-ops (built-in tools are always available)
- Wraps existing `ALL_FILE_TOOLS`, `ALL_GIT_TOOLS`, `ALL_SUBPLAN_TOOLS` lists
- Backward compatible — existing registration functions still work

### 2. MCP Resource Slot Inference (`mcp/adapter.py`)
New `infer_resource_slots()` static method on `MCPToolAdapter`:
- Scans MCP tool parameter schemas for file/directory/repo patterns
- Creates `ResourceSlot` objects with appropriate type, access mode, binding mode
- Mapping: `file_path`→file(rw), `directory`→directory(ro), `repo_path`→git-checkout(rw)
- Slots stored in `source_metadata["resource_slots"]` on registered `ToolSpec` objects

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,817 scenarios) |
| `nox -s integration_tests` | PASS (6 new tests) |
| `nox -s coverage_report` | 97% (>= 97%) |

Closes #882

Reviewed-on: cleveragents/cleveragents-core#964
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
 2026-03-21 05:30:01 +00:00
brent.edwards b88bc0ec1b feat(perf): large project scaling tests (#984) 
## Summary

Add large project scaling benchmarks and tests at production scale (10K–100K files).

### New ASV Benchmarks

**IndexingScalingSuite** (`large_project_scaling_bench.py`):
- `time_walk_and_index` at 1K/10K/50K/100K files
- `time_incremental_refresh` (1% modified files)
- `track_indexed_file_count`, `track_tokens_per_second`

**ContextAssemblyScalingSuite** (`context_assembly_scaling_bench.py`):
- `time_full_pipeline` at 100/1K/5K/10K fragments
- `time_tiered_strategy`, `time_recency_strategy`
- `track_assembled_tokens`, `track_fragments_per_second`

**ExecutionThroughputSuite** (`execution_throughput_bench.py`):
- `time_sequential_plans` at 10/50/100 plans
- `time_executor_construction`, `time_decision_tree_scaling`

### Scale Fixture Updates

- Added `xlarge` (50K files) and `xxlarge` (100K files) profiles to `scale_metadata.json`
- Added 50K/100K thresholds to `baseline_thresholds.json`
- Added `context_assembly` and `execution_throughput` threshold sections

### Tests & Documentation

- 15 Behave scenarios validating profiles, thresholds, monotonicity, memory budgets
- 6 Robot integration tests including live 1K-file indexing throughput check
- `docs/reference/scaling_baselines.md` documenting all baseline metrics

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,910 scenarios) |
| `nox -s integration_tests` | PASS (1,526 tests) |
| `nox -s coverage_report` | 97% (>= 97%) |

Closes #859

Reviewed-on: cleveragents/cleveragents-core#984
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
 2026-03-21 04:46:45 +00:00
brent.edwards 8a87262f86 feat(sandbox): implement overlay filesystem sandbox strategy (#994) 
## Summary

Implement the overlay filesystem sandbox strategy with OverlayFS support and userspace fallback.

### Implementation

**`OverlaySandbox`** (`infrastructure/sandbox/overlay.py`, 497 lines):
- Detects OverlayFS availability at runtime via `/proc/filesystems` + `os.geteuid() == 0` check
- **Real OverlayFS mode** (requires root): creates upper/work/merged dirs, mounts overlay filesystem, captures writes in upper layer
- **Userspace fallback** (default in CI/containers): `shutil.copytree` the original into merged dir, tracks changes via `filecmp` diff on commit
- `create()`: sets up directory structure, mounts if available
- `commit()`: copies changed/added files from overlay to original, removes deleted files
- `rollback()`: unmounts (or removes) merged, recreates from scratch
- `cleanup()`: unmounts, removes all temp dirs, idempotent

### Domain Model Updates
- Added `OVERLAY = "overlay"` to `SandboxStrategy` enum in both `resource_type.py` and `resource.py`
- Added `STRATEGY_OVERLAY` to `SandboxFactory`, registered for `fs-mount`, `fs-directory`, `fs-file` resources

### Tests
- **22 Behave scenarios**: full lifecycle (create/commit/rollback/cleanup), status transitions, path traversal guard, fallback detection, error handling
- **6 Robot integration tests**: end-to-end overlay sandbox operations

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,917 scenarios) |
| `nox -s coverage_report` | 97% (>= 97%) |

Closes #880

Reviewed-on: cleveragents/cleveragents-core#994
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-21 03:14:40 +00:00
brent.edwards 6531440431 feat(actor): implement estimation actor type (#962) 
## Summary

Implement the estimation actor as a functional actor type. The estimation actor provides cost, time, and resource estimates for plan operations, running after Strategize completes (before Execute). Estimation is informational only and optional — plans work without an estimation actor configured.

### Changes

**New files:**
- `src/cleveragents/domain/models/core/estimation.py` — `EstimationResult` frozen Pydantic model with fields for cost (USD), tokens, steps, child plans, time, risk level/factors, summary
- `features/estimation_actor.feature` — 12 Behave test scenarios (model validation, serialization, stub actor, plan integration, optional behavior)
- `features/steps/estimation_actor_steps.py` — Step definitions
- `robot/estimation_actor.robot` — 6 Robot integration tests
- `robot/helper_estimation_actor.py` — Robot test helper

**Modified files:**
- `domain/models/acms/tiers.py` — Added `ESTIMATOR` to `ActorRole` enum
- `domain/models/core/plan.py` — Added `estimation_result: EstimationResult | None` field, estimation data in `as_cli_dict()`
- `application/services/plan_executor.py` — Added `EstimationStubActor` class
- `application/services/plan_lifecycle_service.py` — Added `_run_estimation()` method, invoked in `execute_plan()`
- `cli/commands/plan.py` — Display estimation results in plan status
- `vulture_whitelist.py` — Added 12 new public symbols

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,818 scenarios) |
| `nox -s integration_tests` | PASS (1,512 tests) |
| `nox -s coverage_report` | 98% (>= 97%) |

Closes #890

Reviewed-on: cleveragents/cleveragents-core#962
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-21 01:40:58 +00:00
brent.edwards 95d3e09925 feat(cli): final CLI polish and UX consistency pass (#1018) 
## Summary

Final CLI polish and UX consistency pass: shared constants, centralized error formatting, shell completion, and standardized help text.

### New Modules

- **`cli/constants.py`** (70 lines): Exit codes (`EXIT_SUCCESS`=0 through `EXIT_CONFLICT`=4), format defaults (`FORMAT_TEXT`, `FORMAT_JSON`, `FORMAT_TABLE`)
- **`cli/errors.py`** (105 lines): `cli_error()` with hint support, `cli_warning()`, `cli_not_found()` with resource-type-aware hint

### CLI Changes

- Shell `completion` command generating scripts for bash/zsh/fish/powershell
- Standardized help text across command modules
- Error functions exported from `cli/__init__.py`

### Tests

- **17 Behave scenarios**: Exit codes, error formatting, cli_not_found, format constants, help text, completion
- **15 Robot integration tests**: All subcommands respond to --help, invalid commands return non-zero, completion generation works

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,912 scenarios) |
| `nox -s coverage_report` | 97% (>= 97%) |

Closes #861

Reviewed-on: cleveragents/cleveragents-core#1018
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
 2026-03-21 01:14:18 +00:00
brent.edwards 00897be24a feat(ci): CI/CD pipeline definitions (#983) 
## Summary

Complete CI/CD pipeline definitions: release workflow, caching, status-check consolidation, and documentation.

### Changes

**New: Release pipeline** (`.forgejo/workflows/release.yml`):
- Triggered on `v*` tags
- 3 jobs: `build-wheel` → `build-docker` → `create-release`
- Builds wheel via `nox -s build`, Docker image via multi-stage Dockerfile
- Creates Forgejo release via API with wheel artifact attached
- Configurable registry push via `REGISTRY_*` secrets

**Updated: CI pipeline** (`.forgejo/workflows/ci.yml`):
- Added `actions/cache@v3` for `~/.cache/uv` on all 8 primary jobs (keyed on `pyproject.toml` hash)
- Added `status-check` consolidation job depending on all required checks — single gate for branch protection

**Updated: CONTRIBUTING.md**:
- New CI/CD section: pipeline overview, job table, required merge checks, release process, secrets documentation

**Tests**:
- 13 new Behave scenarios validating workflow YAML structure, tag triggers, job definitions, dependencies
- 9 new Robot tests validating file existence, content, nox session references

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,819 scenarios) |
| `nox -s coverage_report` | 97.9% (>= 97%) |

Closes #858

Reviewed-on: cleveragents/cleveragents-core#983
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
 2026-03-21 00:29:49 +00:00
brent.edwards fa3f2d6365 test: add TDD bug-capture test for #932 — plan apply missing --yes flag (#958) 
## Summary

TDD expected-fail tests proving bug #932 exists: the `plan apply` (`lifecycle-apply`) command does not accept the `--yes`/`-y` flag required by the specification. The flag should skip the confirmation prompt before applying plan changes.

### Tests Added

**Behave scenarios** (`features/tdd_plan_apply_yes_flag.feature`):
- `lifecycle-apply --yes` should be accepted (tests `--yes` long flag)
- `lifecycle-apply -y` should be accepted (tests `-y` short flag)

Tags: `@tdd_expected_fail @tdd_bug @tdd_bug_932`

**Robot Framework tests** (`robot/tdd_plan_apply_yes_flag.robot`):
- `check-yes-long` — invokes CLI with `--yes`, asserts no "No such option" error
- `check-yes-short` — invokes CLI with `-y`, asserts no "No such option" error

### How the Bug Is Proven

The `lifecycle_apply_plan` function (`cleveragents.cli.commands.plan`) defines only `plan_id` and `--format` parameters — no `--yes`/`-y`. When tests invoke `lifecycle-apply --yes`, Typer/Click rejects it with `"No such option: --yes"` and exit code 2. The assertion that this error is absent **fails**, confirming bug #932. The `@tdd_expected_fail` tag inverts this to a pass.

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,808 scenarios) |
| `nox -s integration_tests` | PASS (1,508 tests) |
| `nox -s coverage_report` | 98% (>= 97%) |

Closes #950

Reviewed-on: cleveragents/cleveragents-core#958
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
 2026-03-21 00:13:17 +00:00
brent.edwards 8d108cb5d1 feat(tool): add tool-level execution environment preferences (#970) 
## Summary

Add tool-level execution environment preferences with four modes: `required`, `preferred`, `specific`, and `none`.

### Changes

**New model** (`domain/models/core/execution_environment_preference.py`):
- `EnvironmentPreferenceMode` enum: REQUIRED, PREFERRED, SPECIFIC, NONE
- `ExecutionEnvironmentPreference` frozen Pydantic model with `mode` and `target_resource` fields
- Model validator ensures `target_resource` required iff mode is SPECIFIC

**ToolSpec & Tool model updates:**
- Added `execution_environment: ExecutionEnvironmentPreference` field to both `ToolSpec` (runtime) and `Tool` (domain)
- `Tool.from_config()` parses `execution_environment` from YAML config dicts

**ToolRunner integration** (`tool/runner.py`):
- Before calling `_env_resolver.resolve()`, checks `spec.execution_environment.mode`:
  - REQUIRED: raises `ContainerUnavailableError` if resolved env is not CONTAINER
  - PREFERRED: tries container, gracefully falls back to host
  - SPECIFIC: overrides `tool_env` with `target_resource`
  - NONE: current behavior (caller-supplied tool_env)

### Tests

- **27 Behave scenarios** covering model validation, serialization, preference routing, YAML parsing, error cases
- **12 Robot integration tests** covering CLI tool preference display and end-to-end routing

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,833 scenarios) |
| `nox -s integration_tests` | PASS (12 new tests) |

Closes #879

Reviewed-on: cleveragents/cleveragents-core#970
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-20 23:51:46 +00:00
brent.edwards 4f950a1600 feat(cli): repo indexing CLI functional (#982) 
## Summary

Add `agents repo index` and `agents repo status` CLI commands, wiring the existing `RepoIndexingService` backend to the CLI layer.

### Commands

- **`agents repo index <resource_name>`** — Triggers full or incremental indexing of a repository resource. Options: `--full` (force full re-index), `--format text|json`
- **`agents repo status <resource_name>`** — Displays indexing metadata: status, file count, token estimate, primary language, last indexed timestamp. Options: `--format text|json`

### Implementation

- New CLI module `src/cleveragents/cli/commands/repo.py` (238 lines)
- Registered as `repo` subcommand group in `cli/main.py`
- Resolves resource by name via `ResourceRegistryService`
- Calls `RepoIndexingService.index_resource()` / `refresh_index()` / `get_index_status()`
- Both text (Rich panel) and JSON output formats

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,819 scenarios) |
| `nox -s coverage_report` | 98% (>= 97%) |
| Integration tests | 5/5 PASS |

Closes #856

Reviewed-on: cleveragents/cleveragents-core#982
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-20 22:28:32 +00:00
freemo 051ee7c290 test(coverage): add Behave BDD tests to improve coverage across 52 source files 
Added 52 new .feature files and corresponding _steps.py files targeting
previously uncovered code paths in the following areas:

- TUI layer: app, commands, persona (state/schema/registry), widgets,
  input (shell_exec, reference_parser)
- Application services: plan lifecycle/service/executor, session,
  project, repo indexing, correction, checkpoint, actor, llm_actors,
  strategy coordinator, resource file watcher, service retry wiring
- CLI commands: session, resource, repl, plan, db, automation_profile
- Domain models: retry_policy, resource_type, cost_budget,
  docker_compose_analyzer, detail_level, _sql_string_aware,
  _postgresql_helpers
- Core: circuit_breaker, retry_service_patterns
- Infrastructure: repositories, transaction_sandbox, strategy_registry,
  plugins/loader, container
- Config: settings
- Agents: plan_generation, context_analysis, auto_debug
- A2A: facade

All new tests follow the Behave/Gherkin BDD standard. Resolved step
definition collisions with unique prefixes. Fixed Alembic fileConfig
logger disabling issue (disable_existing_loggers=False).

ISSUES CLOSED: #1068
 2026-03-20 21:22:10 +00:00
brent.edwards 202c9bfe75 test(acms): TDD failing tests for context tier runtime logic (bug #821) (#1058) 
## Summary

TDD expected-fail tests proving bug #821 exists: `ContextTierService` has data models for hot/warm/cold tiers but **no runtime logic** for automatic promotion, demotion, or eviction.

- **Promotion on access**: Accessing a cold-tier fragment repeatedly via `get()` does NOT auto-promote it — `get()` updates `access_count`/`last_accessed` but never calls `promote()`
- **Demotion on staleness**: No staleness enforcement method exists — tried `enforce_staleness()`, `apply_tier_policy()`, `tick()`, etc. — none are implemented
- **Eviction on budget overflow**: `store()` does NOT enforce `TierBudget.max_tokens_hot` — the hot tier grows without bound

### Files Added
| File | Purpose |
|------|---------|
| `features/tdd_context_tier_runtime.feature` | 3 Behave scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_821 @mock_only` |
| `features/steps/tdd_context_tier_runtime_steps.py` | Type-annotated step definitions exercising real `ContextTierService` |
| `robot/tdd_context_tier_runtime.robot` | 3 Robot Framework integration tests tagged `tdd_expected_fail` |
| `robot/helper_tdd_context_tier_runtime.py` | Helper script for Robot tests with 3 subcommands |

### Verification
- `nox -s lint` — passed
- `nox -s typecheck` — passed (0 errors)
- `nox -s unit_tests -- features/tdd_context_tier_runtime.feature` — **3 scenarios passed** (all assertions fail as expected, `@tdd_expected_fail` inverts to CI pass)

ISSUES CLOSED: #840

Reviewed-on: cleveragents/cleveragents-core#1058
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-19 22:24:41 +00:00
brent.edwards 60af2cae0c fix(cli): make plan explain accept plan_id for plan-level explanation (#1057) 
## Summary

Fixes `plan explain` to accept both decision IDs and plan IDs as the positional argument, matching the M3 acceptance test usage pattern `plan explain <plan_id>`.

### Problem
`explain_decision_cmd` only accepted a decision ULID. When the M3 acceptance test passed a plan ID, `svc.get_decision(plan_id)` returned `None`, causing "Decision not found" error with exit code 1.

### Fix
1. Renamed parameter `decision_id` → `identifier`
2. Tries `svc.get_decision(identifier)` first (backward compat)
3. Falls back to `svc.list_decisions(identifier)` treating it as a plan_id, explaining the root decision
4. Clear error if neither resolves

### Quality Gates
| Session | Result |
|---------|--------|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` (explain features) | 46/46 PASS |

Closes #968

Reviewed-on: cleveragents/cleveragents-core#1057
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
 2026-03-19 22:10:23 +00:00
hamza.khyari e2f90ffcd5 feat(resource): add deferred physical resource types 
Add 11 deferred physical resource types covering the git object taxonomy
(git, git-remote, git-branch, git-tag, git-commit, git-tree, git-tree-entry,
git-stash, git-submodule) and filesystem link types (fs-symlink, fs-hardlink).

- YAML configs under examples/resource-types/
- Bootstrap registration via _resource_registry_physical.py module
- Auto-discovery rules with bounded scan_depth (1-2) for git object graph
- fs-directory scan_depth=1 (immediate children only)
- git-branch scan_depth=1 (single HEAD), git-tree scan_depth=2 (capped)
- Updated fs-directory child_types/parent_types/auto_discovery
- Updated git-checkout child_types to include git
- Fixed git-tag child_types to include git-commit (DAG consistency)
- Behave tests, Robot tests, ASV benchmarks
- Documentation in docs/reference/resource_types_builtin.md

ISSUES CLOSED: #330
 2026-03-19 14:03:17 +00:00
hurui200320 554d6889cc fix(cli): add --skill flag to actor run command (#971) 
## Summary

Add the missing `--skill` repeatable flag to `actor run` and `actor-run` CLI commands, aligning the implementation with the specification (CLI Synopsis line 277). The flag enables ad-hoc skill injection at runtime without modifying YAML configuration.

Closes #887

## Changes

### DI Container
- **`container.py`**: Added `_build_skill_service()` factory and `skill_service` Singleton provider, following the established `_build_*` pattern. Falls back to in-memory `SkillService()` when the database is unavailable. Exception handling narrowed to `(ImportError, OperationalError, DatabaseError, OSError)` with `exc_info=True` for traceability.

### CLI Layer
- **`actor.py`**: Added `--skill` Typer option (`list[str] | None`, repeatable, `metavar="NAME"`). Help text notes that skills only augment tool-bearing agents. Wrapped constructor in the existing `try/except` block so `CleverAgentsException` from skill resolution is properly caught.
- **`actor_run.py`**: Same `--skill` option with `metavar="NAME"`. Exception handler catches `CleverAgentsException` (matching master — not broadened to `CleverAgentsError`).
- **`skill.py`**: Removed module-level `_service` cache. `_get_skill_service()` now always delegates to `get_container().skill_service()` so that `reset_container()` correctly invalidates the cached instance. `_reset_skill_service()` now overrides the container's provider via `providers.Object()`. Removed dead `validate_skill_names()` function.

### Runtime Layer
- **`application.py`** (438 lines, down from 625): `ReactiveCleverAgentsApp` gains `skill_names` parameter with automatic deduplication via `dict.fromkeys`. `_resolve_skills()` obtains `SkillService` from the DI container (no CLI layer import). Separate `except KeyError` and `except ValueError` produce distinct error messages (`"not found in registry"` vs `"resolution failed: {exc}"`). Skill tools are only injected into agents that already have tools (`if self._resolved_skill_tools and tools:`), preventing LLM agents from being converted to pass-through `SimpleToolAgent` instances. When skill tools are skipped for tool-less agents, `logger.debug` emits a diagnostic message. `_sanitize_skill_name()` validates skill name format with tightened regex: `^[\w.-]{1,127}/[\w.-]{1,127}$` with `re.ASCII` flag. Zero-tool skill warning now uses `logger.warning` (not `print(stderr)`), ensuring structured log output and proper log-level filtering.
- **`graph_executor.py`** (334 lines): Extracted graph execution logic. Type annotations improved.

### Tests
- 24+ Behave scenarios across feature files covering: single/multiple/unknown skill flags, skill+context combined, duplicate deduplication, skill resolution, ValueError path, zero-tool resolution, error handling, tool merging, default behavior, overrides, LLM agent guard, `_sanitize_skill_name` edge cases (empty string, too-long name, ANSI escape codes, disallowed characters), `_build_skill_service` happy+fallback paths, `_get_skill_service` container delegation.
- CLI "unknown skill" tests for **both** `actor.py` and `actor_run.py` exercise the real error chain (mock only `get_container()`, not the entire `ReactiveCleverAgentsApp`), testing `_resolve_skills()` → `CleverAgentsException` → `except CleverAgentsException` → exit code 2 end-to-end.
- Combined skill+context tests assert `ContextManager` was instantiated and `exists()` was called in dedicated **Then** steps.
- `@coverage` tags added to all new scenarios.
- **Robot Framework smoke tests** added (`robot/skill_actor_run.robot` + `robot/helper_skill_actor_run.py`): unknown-skill error path and valid-skill acceptance path.

### Changelog
- Added entry under `## Unreleased` in `CHANGELOG.md`.

## Review Fixes Applied (Brent Edwards, Rounds 1 & 2)

| # | Finding | Resolution |
|---|---------|------------|
| **P1-1** | `print(stderr)` for zero-tool skill warning | **Fixed** — replaced with `logger.warning("Skill '%s' resolved to zero tools", name)`, removed unused `import sys` |
| **P2-2** | Skill tools silently skipped for tool-less agents | **Fixed** — added `logger.debug` when skipped; updated `--skill` help text to note "only augments tool-bearing agents" |
| **P2-3** | `container.py` at 739 lines | **Acknowledged** — pre-existing growth (+59 lines for `_build_skill_service`); extracting factories is a separate refactoring task |
| **P2-4↑** | `CleverAgentsException` → `CleverAgentsError` broadens catch scope | **Fixed** — reverted `actor_run.py` to `except CleverAgentsException` matching master |
| **P3-5** | No Robot Framework smoke test for `--skill` | **Fixed** — added `skill_actor_run.robot` with 2 test cases (unknown-skill error, valid-skill acceptance) |
| **P3-6** | `GraphExecutor._follow_chained_edges` static-calling-static | **Acknowledged** — cosmetic pattern that doesn't affect correctness; can address in a follow-up |

## Known Limitations / Deferred Items

| Item | Reason |
|------|--------|
| `actor.py` at 679 lines (500-line guideline) | Pre-existing (670 on master), +9 lines for `--skill`. Refactoring the shared `_execute()` closure is a separate task. |
| `container.py` at 739 lines (500-line guideline) | Was 680 lines on master, +59 lines for `_build_skill_service()` and `skill_service` provider. Refactoring into sub-modules is a separate task. |
| Code duplication between `actor.py` and `actor_run.py` `run()` | ~47 lines identical code. Coupled with the line-count issue above — both require extracting shared execution logic into a helper module. |
| `SimpleToolAgent` only executes `tools[0]` | Deferred to #974. Pre-existing architectural limitation, not introduced by this PR. |
| `GraphExecutor._follow_chained_edges` static-calling-static pattern | Cosmetic, doesn't affect behavior. |

## Quality Gates
- `nox -s lint`: ✅ PASS
- `nox -s typecheck`: ✅ PASS (0 errors)
- `nox -s unit_tests`: ✅ PASS (11,130 scenarios, 0 failures)
- `nox -s integration_tests`: ✅ PASS (1,559 tests, 0 failures)
- `nox -s coverage_report`: ✅ 97% (meets threshold)
- Branch rebased onto latest `master` (`ab1fd19b`)

Reviewed-on: cleveragents/cleveragents-core#971
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 09:30:35 +00:00
hurui200320 2434253c1a feat(cli): implement full output rendering framework (#812) 
## Summary

Implement the 6 missing element handle types (Tree, Text, Code, Diff, Separator, ActionHint) and ensure all 6 materialization strategies (rich, color, table, plain, json, yaml) support all 10 element types. This completes the output rendering framework per the specification (§25417-27276).

Closes #550

## Changes

### Source (restructured into smaller modules)

#### `handles/` package (was `handles.py` — 1062 lines → 5 files, all ≤500 lines)
- **`_models.py`**: All Pydantic data models, constants (MAX_TREE_DEPTH, MAX_ELEMENTS_PER_SESSION, MAX_TABLE_ROWS), exceptions (ElementClosedError), event classes, ElementSnapshot union. P2-1: `DiffLine.type` renamed to `line_type` with backward-compatible alias.
- **`_base.py`**: Generic `ElementHandle[E]` base class with thread-safe `element_copy()` (lock-protected).
- **`_panel_table.py`**: PanelHandle, TableHandle, StatusHandle.
- **`_concrete.py`**: ProgressHandle (P1-1: validates `total` non-negative), TreeHandle, TextHandle (P3-2: close() delegates to super()), CodeHandle, DiffHandle, SeparatorHandle, ActionHintHandle. P3-8: `increment(delta=0)` now rejected.
- **`__init__.py`**: Re-exports all public symbols.

#### `_renderers.py` — plain renderers, sanitization, and shared helpers
- Plain render functions for all 10 element types.
- Terminal escape sanitization (`strip_terminal_escapes`).
- P2-3: `_sort_table_rows` now consults `ColumnDef.col_type` for numeric sorting.
- P2-6: `compute_column_widths()` shared helper extracted (DRY fix).

#### `_color_renderers.py` — ANSI colour renderers
- Colour-coded render functions for all 10 element types.
- P2-2: TextBlock now gets color treatment (`_render_text_color`) instead of falling through to plain.

#### `_boxdraw.py` — box-drawing renderers
- P2-5: Upgraded from ASCII `+-|` to Unicode box-drawing `╭─╮│╰╯` with rounded corners per spec §26821.

#### `_ids.py` — ID generation helpers
- P3-1: Session and handle IDs now use separate counters, making IDs monotonic within their namespace.

#### `materializers.py` — strategy protocol and 6 concrete strategies
- P1-2: `_snapshot_to_dict` now includes `timing` field in JSON/YAML output per spec §27022.
- P2-4: `_column_def_to_dict` always includes all fields unconditionally for stable JSON schemas.

#### `selection.py` — materializer selection with fallback
- P1-4: `NO_COLOR` environment variable now respected (https://no-color.org/). When set, all visual formats fall back to plain. Precedence: explicit flag > NO_COLOR > terminal capability fallback.

#### `session.py`
- P1-1: `session.progress()` factory validates `total >= 0`.
- P1-2: `snapshot()` includes `timing` when available.

#### `__init__.py` — package docstring
- P1-3: SD-29 corrected to reflect actual Table → Color → Plain fallback chain.
- SD-14 marked as implemented (NO_COLOR support added).

### Spec Deviations (Documented)
28 deliberate deviations documented in `__init__.py` module docstring (SD-1 through SD-29, with SD-14 now implemented). SD-29 corrected.

### Tests (updated)
- **+23 new BDD scenarios** covering: P1-1 total validation, P1-4 NO_COLOR, P2-1 DiffLine.line_type alias, P2-2 text color, P2-3 numeric sorting, P2-4 ColumnDef serialization, P2-5 Unicode box-drawing, P3-3 add_rows limit, P3-5 10-thread stress test, P3-6 summary truncation, P3-7 explicit format for color/table, P3-8 zero delta.
- **Robot tests**: Updated box-drawing assertion for Unicode chars.

## Verification

| Check | Result |
|-------|--------|
| Pyright | 0 errors, 1 pre-existing warning |
| Ruff lint | All passed |
| Unit tests | 393 features, 11,344 scenarios, 0 failures |
| Integration tests | All passed |
| E2E tests | All passed |
| Coverage | 97% overall (threshold: 97%) |

## Review Fixes Applied (Luis Review #2412)

| ID | Severity | Fix |
|----|----------|-----|
| P1-1 | High | `set_progress()` and `session.progress()` validate `total >= 0` |
| P1-2 | High | `_snapshot_to_dict` includes `timing` field |
| P1-3 | High | SD-29 documentation corrected |
| P1-4 | High | `NO_COLOR` env var respected |
| P2-1 | Medium | `DiffLine.type` → `line_type` with alias |
| P2-2 | Medium | TextBlock gets color treatment |
| P2-3 | Medium | Numeric column sorting |
| P2-4 | Medium | ColumnDef always serializes all fields |
| P2-5 | Medium | Unicode box-drawing characters |
| P2-6 | Medium | Shared `compute_column_widths()` helper |
| P3-1 | Low | Separate ID counters |
| P3-2 | Low | TextHandle.close() delegates to super() |
| P3-3 | Low | add_rows batch limit test |
| P3-5 | Low | 10-thread stress test |
| P3-6 | Low | Summary truncation test |
| P3-7 | Low | Explicit format tests for color/table |
| P3-8 | Low | Zero delta rejected |

### Deferred Items
| ID | Reason |
|----|--------|
| P3-9 | snapshot() lock scope — acceptable correctness trade-off |
| P3-10 | CLEVERAGENTS_FORMAT env var — documented as SD-15, requires CLI framework changes |

Reviewed-on: cleveragents/cleveragents-core#812
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 08:46:46 +00:00
hurui200320 3837327564 feat(plan): enforce decision type phase-gating at recording time (#973) 
## Summary

Adds phase-gating validation to `DecisionService.record_decision()` that enforces the specification's constraint: certain decision types are only valid during specific plan phases. This prevents invalid decisions (e.g., `tool_invocation` during Strategize, `strategy_choice` during Execute) from being persisted.

### Changes

- **Exception** (`cleveragents.core.exceptions`): Added `DecisionPhaseViolationError(BusinessRuleViolation)` with `decision_type`, `plan_phase`, and `allowed_types` attributes.
- **Phase constants** (`cleveragents.domain.models.core.decision`):
  - `resource_selection` added to `EXECUTE_TYPES` — now phase-agnostic (Strategize or Execute) per ADR-007 L72 and ADR-033 L74.
  - `subplan_spawn` / `subplan_parallel_spawn` in both sets; code comment documents divergence from ADRs per M4 subplan model (ticket #931).
  - `USER_INTERVENTION` remains phase-agnostic (both sets).
  - Module-level docstring table updated to match actual assignments.
  - `is_any_phase_type` property updated to check membership in both sets dynamically (was hardcoded to `USER_INTERVENTION` only).
- **Phase-gating module** (`cleveragents.application.services.phase_gating`):
  - Extracted from `DecisionService` to reduce `decision_service.py` line count (1010 → 913) and isolate the phase-gating concern.
  - `PHASE_ALLOWED_TYPES` typed as `Mapping[PlanPhase, frozenset[DecisionType]]`.
  - `resolve_plan_phase()` helper: supports explicit parameter, DB lookup, and graceful skip.
  - `validate_phase_gating()` enforcement raises `DecisionPhaseViolationError`.
  - Exception narrowing: DB lookup catches `(DatabaseError, OperationalError, OSError)` instead of bare `except Exception` — only absorbs infrastructure failures, not programming errors.
  - `# TODO(pg-migration):` marker on TOCTOU race documentation for future PostgreSQL migration.
- **Decision service** (`cleveragents.application.services.decision_service`):
  - Added `plan_phase` parameter to `record_decision()`.
  - Invalid `plan_phase` string now raises `ValidationError` (was uncaught `ValueError`).
  - Imports and delegates to `phase_gating` module for all phase-gating logic.
  - `PHASE_ALLOWED_TYPES` re-exported in `__all__` for backward compatibility.
- **CHANGELOG**: Added behavioral change entry for `resource_selection` reclassification.
- **Backward compatibility**: Phase-gating is opt-in — when neither `plan_phase` is provided nor a UnitOfWork is wired, validation is skipped, preserving all existing callers.
- **Unrelated drive-by reverted**: Removed `ULID_PATTERN` from `decision.py` `__all__` (was an unrelated export addition).
- **Tests**:
  - 36 Behave scenarios covering valid/invalid types per phase, phase-agnostic acceptance, DB-based resolution (Strategize and Execute plans), unknown plan in DB, PlanPhase enum pass-through, error attributes, and ungated phases.
  - 11 new Behave scenarios for `is_any_phase_type`: 4 dual-phase types (true) + 7 single-phase types (false), including `prompt_definition` root test.
  - 6 Robot Framework integration tests with stderr assertions.
  - Updated `consolidated_decision.feature` for new `EXECUTE_TYPES` member count (8 members).
  - Test cleanup now calls `uow.engine.dispose()` before file deletion.
  - `tempfile.mktemp()` replaced with `tempfile.mkstemp()`.
  - Inline imports moved to module top-level per CONTRIBUTING.md.
  - Flaky concurrency test timing increased in `subplan_execution_steps.py`.

### Review Round 1 + 2 Fixes

| # | Finding | Resolution |
|---|---------|------------|
| P1-1 | `except Exception` too broad in `_resolve_plan_phase` | Narrowed to `(DatabaseError, OperationalError, OSError)` — matches codebase pattern |
| P2-2 | `decision_service.py` at 1010 lines | Extracted to `phase_gating.py` module (1010 → 913 lines) |
| P2-3 | TOCTOU race — no programmatic guard | Added `# TODO(pg-migration):` marker with actionable guidance |
| P2-4 | `resource_selection` reclassification needs CHANGELOG | Added CHANGELOG entry documenting behavioral change |
| P2-7 | `is_any_phase_type` BDD gap for dual-phase types | Added 11 parametrized scenarios covering all 4 dual-phase + 7 single-phase types |
| P3-5 | `ULID_PATTERN` export is unrelated drive-by | Reverted — removed from `decision.py` `__all__` |
| P3-6 | `decision.py` at 514 lines (now 513) | No action — reviewer accepted as marginally over |

### Quality Gates

| Session | Result |
|---------|--------|
| lint | PASS |
| typecheck | PASS (0 errors) |
| unit_tests | PASS (11,153 scenarios, 0 failures) |
| integration_tests | PASS (1,563 tests, 0 failures) |
| e2e_tests | PASS (16 tests, 0 failures) |
| coverage_report | 97% (threshold: 97%) |

Closes #931

Reviewed-on: cleveragents/cleveragents-core#973
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 07:53:43 +00:00
hurui200320 0b415a6e5e test: add TDD bug-capture test for #969 — plan correct plan_id handling (#1051) 
## Summary

Add TDD bug-capture tests for bug #969 (`plan correct` expects `decision_id` but M3 acceptance test passes `plan_id`). These tests prove the bug exists and will serve as regression guards once the fix in #969 is merged.

### Changes

- **Behave test** (`features/tdd_plan_correct_plan_id.feature`): Two scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_969` — one for `--mode revert` and one for `--mode append` — that invoke `plan correct <plan_id>` (without `--plan` flag) and assert the command resolves the plan_id to its root decision as `target_decision_id`. Both modes are tested because the bug affects `target_decision_id` resolution **before** mode-specific branching.

- **Step definitions** (`features/steps/tdd_plan_correct_plan_id_steps.py`): Mock setup for DI container (DecisionService), CorrectionService, and `_resolve_active_plan_id`. Uses `tpcpid` step prefix per project conventions.

- **Shared fixtures** (`features/mocks/tdd_plan_correct_plan_id_fixtures.py`): Centralised constants, patch targets, mock builders (`make_decision_ns`, `make_mock_container`, `make_correction_svc`, `make_default_decisions`, `make_default_container`), and `build_cli_args` helper. Both the Behave steps and Robot helper import from this shared module, eliminating code duplication and drift risk.

- **Robot test** (`robot/tdd_plan_correct_plan_id.robot`): Two integration-level tests (revert + append) with `tdd_expected_fail tdd_bug tdd_bug_969` tags, exercising the same code paths via the helper script.

- **Robot helper** (`robot/helper_tdd_plan_correct_plan_id.py`): Standalone helper that exits 0 with sentinel when the bug is fixed, exits 1 when the bug is present. Imports shared fixtures from `features/mocks/`.

- **Changelog** (`CHANGELOG.md`): Added entry under Unreleased for #979.

### Bug Description

The `correct_decision` function in `cleveragents.cli.commands.plan` declares `decision_id` as its first positional argument. When the M3 acceptance test calls `plan correct <plan_id> --mode revert --guidance "..."`, the plan_id is captured as `decision_id` and used directly as `target_decision_id` in `svc.request_correction()`. Since the plan_id is not a valid decision_id, the correction service cannot find the targeted decision. The same bug path is exercised by `--mode append`.

### How TDD Expected-Fail Works

- The `@tdd_expected_fail` tag causes the test framework to invert the result: the test passes CI when the underlying assertion fails (proving the bug exists) and fails CI if the assertion passes (bug was fixed without removing the tag).
- When bug #969 is fixed, the developer removes the `@tdd_expected_fail` tag, and the test runs normally as a regression guard.

### Quality Gates

All nox sessions pass:
- `nox -s lint` ✅
- `nox -s typecheck` ✅ (0 errors)
- `nox -s unit_tests` ✅ (387 features, 11121 scenarios, 0 failures)
- `nox -s integration_tests` ✅ (1561 tests, 0 failures)
- `nox -s e2e_tests` ✅ (16 tests, 0 failures)
- `nox -s coverage_report` ✅ (≥97%)

Closes #979

Reviewed-on: cleveragents/cleveragents-core#1051
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 07:31:12 +00:00
hurui200320 c741329f70 test: add TDD bug-capture test for #968 — plan explain plan_id handling (#1052) 
## Summary

Add TDD bug-capture tests for bug #968: the `plan explain` CLI command fails with rc=1 when given a plan_id because `explain_decision_cmd` treats its argument as a decision_id and `svc.get_decision(plan_id)` raises `DecisionNotFoundError`.

These tests capture the exact failure condition described in #968 and are tagged with `@tdd_expected_fail`, `@tdd_bug`, and `@tdd_bug_968`. The `@tdd_expected_fail` tag inverts the test result so that CI passes while the bug is unfixed. Once bug #968 is fixed, the `@tdd_expected_fail` tag will be removed and the tests will serve as permanent regression tests.

Closes #978

## Changes

### Behave Tests (`features/tdd_plan_explain_plan_id.feature`)
- **Scenario 1: Plan explain succeeds when given a plan_id with decisions** — Mocks `DecisionService` so `get_decision(plan_id)` raises `DecisionNotFoundError` (the bug) and `list_decisions(plan_id)` returns decisions. Invokes `plan explain <plan_id>` via CliRunner and asserts rc=0 with decision details visible. Verifies `list_decisions` was called with the correct `plan_id`.
- **Scenario 2: Plan explain with plan_id shows root decision question** — Same setup, asserts the root decision question appears in the output.

Both scenarios currently fail (rc=1, proving the bug exists), but the `@tdd_expected_fail` tag makes CI pass.

### Robot Tests (`robot/tdd_plan_explain_plan_id.robot`)
- **TDD Plan Explain Succeeds With Plan ID** — Integration test using a helper script that records decisions via `DecisionService`, invokes `plan explain <plan_id>` via subprocess, and asserts rc=0.
- **TDD Plan Explain With Plan ID Shows Root Question** — Same setup, asserts the root decision question appears in the output.

Both tests carry `tdd_expected_fail`, `tdd_bug`, and `tdd_bug_968` tags.

### Step Definitions (`features/steps/tdd_plan_explain_plan_id_steps.py`)
Step implementations for the Behave feature, following the established patterns from `plan_explain_cli_coverage_steps.py`. Includes `list_decisions.assert_called_once_with(plan_id)` verification to ensure the fix exercises the fallback lookup path.

### Helper Script (`robot/helper_tdd_plan_explain_plan_id.py`)
Python helper for Robot tests, following the established pattern from `helper_tdd_checkpoint_real_rollback.py`. Uses a shared `_run_plan_explain()` helper to avoid subprocess invocation duplication, sets `NO_COLOR=1` in subprocess environment to prevent ANSI escape codes.

### CHANGELOG
Added entry under `## Unreleased` describing the TDD bug-capture tests for bug #968.

## Review Fixes Applied (Iteration 1)

1. **Imports moved to module level** (Major) — Three imports in `_setup_plan_with_decisions()` moved to module level with `# noqa: E402`, consistent with all other `robot/helper_tdd_*.py` files.
2. **Mock accurately models real behavior** (Minor) — `svc.get_decision.return_value = None` changed to `svc.get_decision.side_effect = DecisionNotFoundError(...)` to match the real `DecisionService.get_decision()` which raises `DecisionNotFoundError`, not returns `None`.
3. **Strengthened assertion** (Minor) — `assert "decision_id" in output or "question" in output` changed to AND for a more precise assertion that verifies both fields are present.
4. **Removed `# type: ignore[arg-type]`** (Minor) — Changed `kwargs: dict[str, object]` to `kwargs: dict` and removed the `# type: ignore[arg-type]` directive, per CONTRIBUTING.md type safety rules.
5. **Handled `subprocess.TimeoutExpired`** (Minor) — Both `subprocess.run()` calls wrapped in `try/except subprocess.TimeoutExpired` with descriptive `_fail()` messages.
6. **Resolved timeout race condition** (Minor) — Inner `subprocess.run` timeout reduced from 60s to 45s so the helper catches timeout before Robot's 60s outer timeout kills it.
7. **Defensive setup check** (Nit, addressed) — Added `list_decisions()` verification after `record_decision()` to distinguish setup failures from the actual bug.

## Review Fixes Applied (Iteration 2)

1. **Added CHANGELOG.md entry** (Major) — Added entry under `## Unreleased` describing the TDD bug-capture tests for bug #968, per CONTRIBUTING.md §"Pull Request Process".
2. **Fixed Gherkin step text and feature comments** (Minor) — Renamed step from "returns None" to "raises DecisionNotFoundError" in feature file and step decorator. Updated feature comment from "returns None" to "raises DecisionNotFoundError". Step text now accurately describes the mock behavior.
3. **Fixed Robot helper output assertion logic** (Minor) — Changed `if "decision" not in combined.lower() and "question" not in combined.lower()` to `or` so both keywords must be present, matching the Behave test's AND-based validation.
4. **Added `NO_COLOR=1` to subprocess environment** (Minor) — Added `_make_subprocess_env()` helper that copies `os.environ` and sets `NO_COLOR=1`, following the established pattern from `robot/helper_e2e_common.py`.
5. **Extracted shared subprocess helper** (Nit, addressed) — Deduplicated subprocess invocation blocks into `_run_plan_explain(plan_id)` function, reducing code duplication across the two test subcommands.
6. **Added `list_decisions` mock verification** (Nit, addressed) — Added `svc.list_decisions.assert_called_once_with(plan_id)` in the Behave step that checks decision details, making the test contract clearer.
7. **Rebased on current master** (Nit, addressed) — Branch rebased on `origin/master` per CONTRIBUTING.md §"Branch Hygiene".

## Quality Gates
- ✅ `nox -e lint` — All checks passed
- ✅ `nox -e typecheck` — 0 errors
- ✅ `nox -e unit_tests` — 391 features, 11175 scenarios, 0 failed
- ✅ `nox -e integration_tests` — All tests passed (including Tdd Plan Explain Plan Id)
- ✅ `nox -e e2e_tests` — 16 tests, 0 failed
- ✅ `nox -e coverage_report` — 97% coverage (≥97% threshold met)

Reviewed-on: cleveragents/cleveragents-core#1052
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 07:21:11 +00:00
hurui200320 cbf8bcc993 test(e2e): E2E acceptance criteria for M5 (v3.4.0) — ACMS v1 and context scaling (#811) 
## Summary

Add `robot/e2e/m5_acceptance.robot` with **21 zero-mock E2E test cases** (in addition to existing M5 test suite) covering all M5 (v3.4.0) acceptance criteria:

1. **Context Assembly** — add/list/show/clear files in the context pipeline
2. **Context Scaling** — 10,000+ file project setup with simulate plumbing *(structural)*
3. **Context Policy Configuration** — per-view include/exclude paths, file-size limits
4. **Budget Enforcement** — max_file_size / max_total_size constraint storage *(structural)*
5. **Context Analysis** — ACMS pipeline inspect (tier schema) and simulate (JSON schema) *(structural)*
6. **Plan Execution** — real LLM calls via `openai/gpt-4o-mini` (`plan use` + `plan resume`)

### Structural vs. Behavioural Scope

Tests in sections 1b–4 that use `project context simulate` or `inspect` are **structural / plumbing validations** — they verify CLI execution, JSON serialization, and stored configuration but do **not** exercise actual ACMS indexing or budget enforcement because the `ContextTierService` is an in-memory singleton that starts empty per CLI process. Each affected test has a `[Documentation]` note explaining this limitation. Behavioural ACMS validation is deferred until the full indexing pipeline is wired.

### Production Bug Fixes

| Fix | File | Description |
|-----|------|-------------|
| `session.flush()` → `session.commit()` | `project_context.py` | Policy changes silently lost on `session.close()` |
| `contextlib.suppress` rollback wrapper | `project_context.py` | Prevents rollback failure from masking original commit exception |
| Add `session_factory` DI provider | `container.py` | `project context` commands hit `AttributeError` |
| `providers.Factory` → `providers.Singleton` | `container.py` | Avoid creating duplicate engines per call |
| Add Gemini API key pattern | `redaction.py` | `AIzaSy...` keys now redacted in logs |

### Review Feedback Addressed (Tenth Pass — @CoreRasurae Review #2410)

| # | Severity | Finding | Fix |
|---|----------|---------|-----|
| P3-1 | Medium | "Clear Context" test tautological — never asserts files were present before clearing | Added `Should Contain ${list_before.stdout} config.py` precondition check after `context-load` and before `clear` |
| P3-2 | Medium | Policy/budget verification uses substring matching (`Should Contain 262144`) | Replaced with `Extract JSON From Stdout` + `$rv.get('max_file_size') == 262144` parsed JSON assertions using `resolved_view` dict access |
| P3-3 | Medium | Plan resume doesn't verify `phase` value, only existence | Added `Should Not Be Equal As Strings ${phase} queued` assertion to verify plan transitioned from queued |
| P3-4 | Medium | Plan JSON extraction inconsistency (`rindex` vs `Extract JSON From Stdout`) | Replaced fragile `rindex`-based extraction with `Extract JSON From Stdout` keyword for consistency |
| P3-6 | Medium | Context show summary weak content assertions | Added `Should Not Contain` guards against traceback/error output to reject false positives |
| P3-8 | Medium | `_SafeSession` singleton may accumulate dirty state after rollback | Changed `_SafeSession.close()` from pure no-op to `real.rollback()` to reset session state between calls |
| P3-14 | Medium | No test for `_save_policy_json` rollback path | Added BDD scenario "Save policy rollback re-raises after commit failure" with monkey-patched commit |
| P3-15 | Medium | No test for `_save_policy_json` on nonexistent project | Added BDD scenario "Save policy on nonexistent project row updates zero rows" verifying silent 0-row behavior |
| P4-1 | Low | Plan resume TRY/EXCEPT swallows assertion details | Moved field assertions outside TRY block; TRY only guards JSON extraction |
| P4-2 | Low | `Safe Parse Json Field` logs stale error context | Fixed to track and report both Strategy 1 and Strategy 2 error contexts separately |
| P4-4 | Low | SQLite WAL/SHM files not cleaned in regression test | Added cleanup loop for `-wal` and `-shm` suffixes alongside `.db` file |

### Deferred Items (Out of Scope)

| ID | Severity | Reason |
|----|----------|--------|
| P2-1 | High | `execution_environment` silently dropped on subsequent `context set` — pre-existing production code bug in `_write_policy()`, not introduced by this PR |
| P2-2 | High | Unhandled `ValidationError` on corrupt policy blob — pre-existing `_read_policy()` code, not changed by this PR |
| P2-3 | High | Silent no-op UPDATE when `ns_projects` row missing — pre-existing `_save_policy_json` logic; this PR only changed error handling |
| P3-5 | Medium | Structural tests cannot detect regressions — already honestly documented in every affected test's `[Documentation]` block |
| P3-7 | Medium | View inheritance/override behavior not tested — nice-to-have, not in ticket acceptance criteria |
| P3-9 | Medium | `context_set` double-writes when `execution_environment` set — pre-existing production logic |
| P3-10 | Medium | `budget_tokens=0` silently replaced by default (falsy `or`) — pre-existing production code bug |
| P3-11 | Medium | `context set` replaces entire view instead of merging — pre-existing design choice |
| P3-12 | Medium | GEMINI_API_KEY propagated but potentially unused — security-first: propagating for redaction testing |
| P3-13 | Medium | `reset_container()` doesn't dispose Singleton resources — pre-existing container lifecycle issue |
| M5 | Medium | `_build_session_factory` engine never disposed — production code architecture, out of scope for testing ticket |
| M6 | Medium | Missing `check_same_thread`/`isolation_level` — production code architecture, out of scope for testing ticket |
| L1 | Low | `plan resume` not in spec CLI synopsis — informational |
| L2 | Low | Context summary assertions depend on exact CLI wording — acceptable stability risk |
| L3 | Low | Gemini regex minimum length slightly loose — acceptable security-first trade-off |
| L4 | Low | Missing Google OAuth2 credential patterns — out of scope for this PR |
| P4-3 | Low | `Run CLI` keyword duplicated — different purpose (uses `${WS}` as default cwd), not a true duplicate |
| P4-5–P4-9 | Low | Various additional E2E coverage gaps — nice-to-have, not in ticket acceptance criteria |

### Quality Gates

| Gate | Result |
|------|--------|
| lint | PASS |
| typecheck | PASS (0 errors) |
| unit_tests | **393/393** features, 11,210 scenarios |
| integration_tests | **1,576/1,576** |
| e2e_tests | **37/37** (21 M5 + 12 M6 + 2 smoke + 2 M1) |
| coverage_report | **97%** (threshold: 97%) |

### Files Changed

| File | Change |
|------|--------|
| `robot/e2e/m5_acceptance.robot` | **NEW** — 21 E2E test cases with honest structural documentation, parsed JSON assertions, prerequisite skip guards on all sections, safe assertion messages |
| `robot/e2e/common_e2e.resource` | `on_timeout=kill` + return code checks + safe key evaluation via `os.environ.get` + fixed stale error logging in `Safe Parse Json Field` |
| `robot/e2e/m1_acceptance.robot` | `on_timeout=kill` on git log |
| `robot/e2e/m2_acceptance.robot` | `on_timeout=kill` + return code checks + safe assertion messages (no stderr embedding) |
| `src/cleveragents/application/container.py` | Add `_build_session_factory` + `session_factory` Singleton |
| `src/cleveragents/cli/commands/project_context.py` | `flush()` → `commit()` + `contextlib.suppress` rollback |
| `src/cleveragents/shared/redaction.py` | Add Gemini API key pattern |
| `noxfile.py` | Propagate `GEMINI_API_KEY` in e2e_tests |
| `CHANGELOG.md` | 4 entries for #745 |
| `features/application_container_coverage_boost.feature` | Updated title + 3 scenarios |
| `features/steps/application_container_coverage_boost_steps.py` | Step defs for `_build_session_factory` |
| `features/consolidated_security.feature` | 2 Gemini API key redaction scenarios |
| `features/project_context_cli_coverage_boost.feature` | `flush()→commit()` regression test + rollback path + nonexistent project tests |
| `features/steps/project_context_cli_coverage_boost_steps.py` | Separate engines for regression test + `try/finally` cleanup + `_SafeSession.close()` state reset + rollback/nonexistent test steps + WAL/SHM cleanup |

Closes #745

ISSUES CLOSED: #745

Reviewed-on: cleveragents/cleveragents-core#811
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 06:53:06 +00:00
hurui200320 774dfedc6b test: add TDD bug-capture test for #967 — plan execute phase processing (#1050) 
## Summary

This PR adds TDD bug-capture tests for bug #967 — `plan execute` only transitions state without running strategize or execute phase processing.

### Motivation

Bug #967 describes that the `plan execute` CLI command originally only called `service.execute_plan(plan_id)`, which is a state transition only (Strategize/COMPLETE → Execute/QUEUED). When a plan was in Strategize/QUEUED state (immediately after `plan use`), the command failed because `execute_plan()` requires Strategize/COMPLETE. The CLI should detect the plan's current phase and run `PlanExecutor.run_strategize()` before transitioning.

Per the project's TDD Bug Fix Workflow (`CONTRIBUTING.md`), the first step in fixing any bug is to write a test that captures the buggy behavior. Since the fix for #967 is already present in the codebase (the CLI handler already orchestrates properly), the `@tdd_expected_fail` tags have been removed and these tests serve as **permanent regression guards** ensuring the fix is never reverted.

### Design Approach

**All `@tdd_expected_fail` scenarios were rewritten to exercise the CLI orchestration layer** — the actual code path affected by bug #967. This satisfies AC4: "The test is specific enough that it will pass normally (without the tag) only when the bug is genuinely fixed."

- **Scenarios 1, 2, 4** (previously `@tdd_expected_fail`): Use Typer's `CliRunner` with mocked services to invoke the `plan execute` CLI command handler directly. This tests the orchestration logic in `plan.py` — the exact code that was buggy.
- **Scenario 3** (positive control): Uses real `PlanLifecycleService` (in-memory) and `PlanExecutor` (stub actors) to demonstrate that proper service-level orchestration works.
- **Robot tests**: Replicate the CLI orchestration logic using real services to verify at the integration level.

### Changes

#### Behave Unit Tests
- `features/tdd_plan_execute_phase_processing.feature` — 4 scenarios
- `features/steps/tdd_plan_execute_phase_processing_steps.py` — Step definitions using CliRunner and mocked services

**Scenarios:**
1. **CLI execute command handles plan in Strategize/QUEUED state** — Invokes `plan execute` via CliRunner on a QUEUED plan. Verifies the CLI succeeds and the plan reaches Execute phase.
2. **CLI execute command orchestrates full lifecycle for QUEUED plan** — Verifies `run_strategize()` and `run_execute()` are both called by the CLI handler.
3. **Positive control — proper orchestration transitions QUEUED plan to Execute** — Demonstrates that `run_strategize()` → `execute_plan()` works correctly at the service level. Always passes.
4. **CLI auto-discovery finds plans in Strategize/QUEUED state** — Invokes `plan execute` with no plan_id. Verifies the auto-discovery filter includes QUEUED plans.

#### Robot Integration Tests
- `robot/tdd_plan_execute_phase_processing.robot` — 4 test cases matching the Behave scenarios
- `robot/helper_tdd_plan_execute_phase_processing.py` — Helper script replicating CLI orchestration logic

#### CHANGELOG
- `CHANGELOG.md` — Added entry under "## Unreleased" describing the new tests.

### Quality Gates
- `nox -e lint`: ✅ passed
- `nox -e typecheck`: ✅ passed (0 errors)
- `nox -e unit_tests`: ✅ passed (391 features, 11177 scenarios, 0 failures)
- `nox -e integration_tests`: ✅ passed (1572 tests, 0 failures)
- `nox -e e2e_tests`: ✅ passed (16 tests, 0 failures)
- `nox -e coverage_report`: ✅ 97% coverage

### Review Cycle 2 Fixes
- **Critical #1**: Rewrote `@tdd_expected_fail` scenarios to exercise the CLI orchestration layer via CliRunner instead of testing service/executor APIs that are correct by design. Removed `@tdd_expected_fail` tags since the bug fix is already in the codebase.
- **Major #2**: Added CHANGELOG entry.
- **Major #3**: Rebased onto current `master`.
- **Minor #4**: Added `ProcessingState.QUEUED` assertion to positive control scenario.
- **Minor #5**: Narrowed exception handler from bare `except Exception` to `except (PlanError, PlanNotReadyError)`.
- **Minor #7**: Changed Robot suite to use `Setup Test Environment With Database Isolation`.
- **Minor #8**: Added `on_timeout=kill` to all Robot `Run Process` calls.
- **Minor #12**: Added docstring to `_fail()` helper.
- **Minor #13**: Removed redundant `Settings()` instantiation.

### Known Limitations
- The `@tdd_expected_fail` tags were removed because the bug fix for #967 is already in the codebase. If this PR is merged before the #967 fix PR, the tags would need to be re-added. However, the CHANGELOG and existing CLI code confirm the fix is already on `master`.

Closes #977

Reviewed-on: cleveragents/cleveragents-core#1050
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 06:41:26 +00:00
hurui200320 747d8d3c9a test(cli): TDD failing tests for init --yes non-interactive (bug #783) (#1049) 
## Summary

Adds TDD bug-capture tests proving that `agents init --yes` fails to bypass the
migration approval prompt in a TTY environment (bug #783). These tests follow the
mandatory TDD bug-fix workflow defined in CONTRIBUTING.md §Bug Fix Workflow.

### Changes

- **Behave feature** (`features/tdd_init_yes_no_input.feature`): Two scenarios
  tagged `@tdd_expected_fail @tdd_bug @tdd_bug_783` that invoke `agents init --yes`
  in a fresh environment with the migration prompt simulated as declining (TTY with
  no input). The tests verify the command exits successfully and does not display the
  "Apply migrations now?" prompt.

- **Behave steps** (`features/steps/tdd_init_yes_no_input_steps.py`): Step definitions
  that create an isolated temp directory, clear auto-apply and database-URL environment
  variables, patch `sys.stdin` on the real `sys` module (correct mock target per project
  convention), and replace `MigrationRunner._default_prompt_for_migration` with a
  function that returns `False` (simulating a TTY prompt where the user declines).
  Temp directory and env var cleanup is registered via `context.add_cleanup()`.

- **Robot Framework test** (`robot/tdd_init_yes_no_input.robot`): Two integration test
  cases tagged `tdd_bug`, `tdd_bug_783`, `tdd_expected_fail` matching the Behave scenarios.

- **Robot helper** (`robot/helper_tdd_init_yes_no_input.py`): Helper script that exercises
  the same code path using the same mock strategy as the Behave steps.

### Bug Reproduction Mechanism

The core challenge is that `CliRunner` replaces `sys.stdin` during `invoke()`, making
direct `isatty()` patching ineffective for controlling the prompt path. The mock strategy
addresses this with a two-pronged approach:

1. **`patch.object(sys, "stdin", mock_stdin)`** — Patches `sys.stdin` directly on the
   real `sys` module (correct mock target per the project's established pattern in
   `features/steps/migration_runner_steps.py`). Documents the intent of simulating a TTY.

2. **`patch.object(MigrationRunner, "_default_prompt_for_migration", ...)`** — Replaces
   the prompt function with one that returns `False` (simulating a TTY user declining
   migration). This is the mechanism that actually exercises the bug path, since
   CliRunner's stdin replacement bypasses the `isatty()` check.

3. **`CLEVERAGENTS_DATABASE_URL` with non-template filename** — Sets the database URL
   to a path whose filename does not match the `before_scenario` template-DB prefixes,
   forcing the real Alembic migration path instead of the test fast-path.

With bug #783 present, `require_confirmation=True` is hardcoded in
`UnitOfWork._ensure_database_initialized()`, so the prompt fires, returns `False`,
causing `MigrationNotApprovedError` and a non-zero exit code. After the fix, `--yes`
should bypass the prompt entirely.

### Root Cause (for the bug fix developer)

`init_command()` in `cleveragents.cli.commands.project` receives the `--yes` flag but
only uses it to control output format. It does not forward `yes` to the migration runner.
The migration runner's `init_or_upgrade()` is called via
`unit_of_work._ensure_database_initialized()` with `require_confirmation=True` hardcoded.

### Scenario 2 Limitation

While the bug is present, Scenario 2's first assertion (exit code 0) fails and Behave
skips subsequent steps. The "contains Initialized" assertion is only evaluated after the
bug is fixed, providing distinct post-fix regression value.

### Quality Gate Results

- `nox -s lint` — ✅ passed
- `nox -s typecheck` — ✅ passed (0 errors)
- `nox -s unit_tests` — ✅ passed (387 features, 11121 scenarios, 0 failed)
- `nox -s integration_tests` — ✅ passed (1561 tests, 0 failed)
- `nox -s e2e_tests` — ✅ passed (16 tests, 0 failed)
- `nox -s coverage_report` — ✅ passed (97% coverage)

Closes #842

Reviewed-on: cleveragents/cleveragents-core#1049
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-19 06:31:20 +00:00
freemo ad98d41d61 feat(a2a): implement _cleveragents/ extension method routing 
Add spec-aligned _cleveragents/ prefixed extension method routing to
the A2A local facade per ADR-047. The facade now supports 42 total
operations: 31 new extension methods across 6 families plus 11
legacy proprietary names retained for backward compatibility.

Extension method families implemented:
- _cleveragents/plan/* (13 methods): use, execute, apply, cancel,
  status, tree, explain, correct, diff, artifacts, prompt, rollback,
  list. Plan operations delegate to PlanLifecycleService when wired;
  new operations (cancel, tree, explain, correct, artifacts, prompt,
  rollback, list) have stub handlers returning safe defaults.
- _cleveragents/registry/* (6 methods): tool/list, resource/list,
  actor/list, skill/list, action/list, project/list. Tool and
  resource list delegate to existing services; entity lists are stubs.
- _cleveragents/context/* (4 methods): show (delegates to existing
  handler), inspect, simulate, set (stubs).
- _cleveragents/health/* (2 methods): check, diagnostics/run.
- _cleveragents/sync/* (3 methods): pull, push, status (stubs).
- _cleveragents/namespace/* (3 methods): list, show, members (stubs).

Legacy proprietary names (session.create, plan.create, etc.) continue
to work via the same handler map, marked deprecated. Updated existing
test assertions for the expanded operation count (11 -> 42).

ISSUES CLOSED: #876
 2026-03-18 17:33:30 +00:00
freemo 21e9a65c33 fix(plan): implement end-to-end subplan execution orchestration 
Replace the metadata-only subplan spawn with real child Plan domain object
creation. SubplanService.spawn() now creates full Plan instances with proper
PlanIdentity (linking parent_plan_id and root_plan_id), sets them to
PlanPhase.STRATEGIZE / ProcessingState.QUEUED, and returns them in the
SpawnResult.child_plans list for downstream lifecycle orchestration.

Key changes:
- SubplanService.spawn() now creates Plan domain objects for each spawn entry
  with complete PlanIdentity linking, inheriting the parent plan's namespace,
  actors, project links, definition_of_done, and access settings.
- SpawnResult dataclass extended with child_plans: list[Plan] field to carry
  the created child plans alongside the existing metadata and statuses.
- Parent plan's subplan_statuses list is updated with SubplanStatus entries
  tracking each child's lifecycle state.
- Fixed Pyright type errors: added missing definition_of_done, reusable,
  read_only, and server parameters to Plan and NamespacedName constructors.
- Removed @tdd_expected_fail tags from TDD test files since the bug is fixed.
- Added Behave scenarios and Robot integration tests verifying child plan
  creation, lifecycle phase/state, and parent tracking.

ISSUES CLOSED: #823
 2026-03-18 17:07:07 +00:00
freemo ec4c39aecf fix(plan): implement real checkpoint rollback via git reset 
Replace the simulated rollback in CheckpointService.rollback_to_checkpoint()
with real git operations. The method now executes git reset --hard <sandbox_ref>
followed by git clean -fd inside the sandbox working directory, reverting all
tracked file changes and removing untracked files added after the checkpoint.

Key changes:
- CheckpointService.rollback_to_checkpoint() now calls _git_reset_hard() and
  _git_clean() via subprocess against the sandbox path, enforcing sandbox
  boundary by confining all git operations to cwd=sandbox_path.
- Added _resolve_sandbox_path() to extract sandbox validation into a dedicated
  method, supporting both lifecycle-service and in-memory fallback paths.
- Added _validate_sandbox() to verify the sandbox path is a directory containing
  a .git subdirectory before executing git operations.
- Added _git_changed_paths() to compute the diff between HEAD and the target ref
  before reset, providing accurate restored file counts in the result.
- Domain event emission (CHECKPOINT_RESTORED) added via the optional event_bus,
  following the same EventBus protocol pattern used by other services.
- Updated existing Robot Framework helper (helper_checkpoint_rollback.py) to use
  a real temporary git workspace instead of a fake sandbox path, since
  _validate_sandbox() now enforces that the path is a real git repository.
- Removed @tdd_expected_fail tags from TDD test files since the bug is now fixed.
- Added new Behave scenarios and Robot integration tests for real file reversion,
  file removal after rollback, domain event emission, and sandbox boundary
  enforcement.

ISSUES CLOSED: #822
 2026-03-18 16:51:14 +00:00
aditya d9e51d98f8 fix(tui,cli,tests): harden persona/input modes and stabilize parallel test execution 
- Add shell safety controls for REPL/TUI (`looks_dangerous`, confirmation gate, timeout handling, and env-based shell disable guard).
- Secure persona workflows with strict name/path validation, safe import/export resolution, atomic+locked registry writes, and malformed YAML resilience.
- Unify persona models and wiring by reusing canonical TUI schema/registry, adding DI providers, and lazy-loading TUI exports to avoid circular imports.
- Improve reference discovery with ignored-directory filtering, symlink-safe walking, and TTL caching for CLI/TUI reference catalogs.
- Expand Behave/Robot coverage for safety/error paths and parallel-isolation behavior; add shared `features/mocks/fake_repl_input.py` helper.
- Fix parallel-run flakiness via deterministic cleanup/reload patterns and watchdog polling fallback when inotify limits are reached.

ISSUES CLOSED: #695
 2026-03-18 10:09:06 +00:00
aditya c02f3842ae feat(tui): implement persona system and reference/command input modes 
- Add `agents tui` command wiring and Textual app scaffolding for interactive TUI startup.
- Implement TUI persona schema/registry/state with local YAML persistence and per-session persona binding.
- Add three input-mode flows: Normal (`@` references), Command (`/` slash commands), and Shell (`!` passthrough).
- Introduce TUI widgets/overlays for prompt, persona bar, reference picker, and slash command interactions.
- Extend REPL command routing to support persona/session commands and reference/shell dispatch behavior.
- Add test coverage: Behave features + step definitions, Robot TUI smoke test, and ASV fuzzy-reference benchmark.

ISSUES CLOSED: #695
 2026-03-18 10:09:05 +00:00
hurui200320 48ecf4c00c fix(cli): add --execution-env-priority flag to plan use (#972) 
## Summary

Adds the missing `--execution-env-priority` flag to the `agents plan use` command, aligning the CLI with the specification (spec line 12501). The flag accepts `fallback` (default) or `override` and controls execution environment routing precedence per ADR-043:

- **`override`**: The specified execution environment always wins, bypassing devcontainer auto-detection.
- **`fallback`**: The specified environment defers to auto-detected devcontainers or project-level overrides.

### Changes

- **Domain model** (`cleveragents.domain.models.core.plan`):
  - Added `ExecutionEnvPriority` StrEnum with `FALLBACK`/`OVERRIDE` values.
  - Changed `execution_env_priority` field type to `ExecutionEnvPriority | None` (leverages Pydantic enum validation).
  - Added `@model_validator` enforcing that `execution_env_priority` requires `execution_environment` (domain-level fail-fast invariant). Verified `validate_assignment=True` is set on `Plan.model_config`.
  - Updated `Plan.as_cli_dict()` to include `execution_environment` and `execution_env_priority`, with fallback default of `"fallback"` for pre-migration data where `execution_env_priority` is `None`.
- **CLI** (`cleveragents.cli.commands.plan`):
  - Added `--execution-env-priority` parameter to `use_action`.
  - Validation: priority requires `--execution-environment`, enum value validation, case-insensitive input.
  - Defaults to `"fallback"` when `--execution-environment` is set without explicit priority.
  - Updated `_print_lifecycle_plan` and `_plan_spec_dict` to display the priority, defaulting to `"fallback"` for pre-migration data.
  - Updated `_plan_spec_dict` docstring to mention new keys.
  - Hoisted `ExecutionEnvPriority` import to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan` (no longer conditional on execution environment being set).
  - Guarded `service.save_plan(plan)` with `has_overrides` flag so it is only called when CLI overrides were actually applied.
- **Persistence** (`cleveragents.infrastructure.database`):
  - Added `execution_environment` (`String(255)`, nullable) and `execution_env_priority` (`String(20)`, nullable) columns to `LifecyclePlanModel`.
  - Updated `from_domain()`/`to_domain()` for round-trip serialization including `ExecutionEnvPriority` enum reconstruction. Uses direct attribute access (`plan.execution_environment`) instead of defensive `getattr` — Plan fields always exist on the Pydantic BaseModel.
  - Updated `LifecyclePlanRepository.update()` to persist both fields using direct attribute access.
  - Added Alembic migration `m4_003_plan_env_columns` adding both columns to the `v3_plans` table. Uses `String(255)` for `execution_environment` to accommodate namespaced resource names. Descends from `m6_005_profile_guards_json`.
- **Service** (`cleveragents.application.services.plan_lifecycle_service`):
  - Added `save_plan()` public convenience method for callers that need to re-persist after post-creation mutations.
- **Tests**:
  - 18 Behave scenarios covering:
    - CLI acceptance criteria (valid values, defaults, validation errors, output display, case-insensitive input, service invocation with `call_args` verification).
    - Domain model validator invariant: construction with priority but no environment raises `ValueError`; construction with both fields succeeds.
    - `ExecutionEnvPriority` enum: values verification, `StrEnum` subclass assertion.
    - `Plan.as_cli_dict()`: includes both fields when set, omits both when `None`, defaults priority to `"fallback"` for pre-migration data.
    - DB round-trip serialization: `from_domain()` → `to_domain()` preserves both fields; preserves `None` values.
  - 5 Robot Framework integration tests.
  - Updated pre-existing `SimpleNamespace`-based plan test fixtures in `database_models_lifecycle_coverage_steps`, `database_models_new_coverage_steps`, `database_models_coverage_r2_steps`, and `repositories_error_handling_coverage_steps` to include `execution_environment` and `execution_env_priority` attributes.
  - Simplified Robot helper `sys.path` pattern to standard approach.
- **Changelog**: Updated per CONTRIBUTING.md requirements.

### Review Fixes (Brent Edwards, Review #2384)

- **P2 #1 — Defensive `getattr`**: Replaced `getattr(plan, "execution_environment", None)` with direct `plan.execution_environment` access in both `LifecyclePlanModel.from_domain()` and `LifecyclePlanRepository.update()`. Plan is a Pydantic BaseModel with `default=None`, so the field always exists. Updated 4 pre-existing `SimpleNamespace`-based test fixtures to include the new attributes.
- **P2 #2 — Late conditional import**: Hoisted `ExecutionEnvPriority` import from inside conditional blocks to function-entry deferred imports in both `_plan_spec_dict` and `_print_lifecycle_plan`.
- **P3 — Migration naming**: Acknowledged as deferred (cosmetic only). Updated `m4_003` to descend from `m6_005_profile_guards_json` (post-rebase chain fix).
- **Rebase**: Branch rebased onto latest `master` with merge conflict in `CHANGELOG.md` resolved.

### Deferred Items

- **Partial failure atomicity** (#8 from review): If `save_plan()` fails after `use_action()` succeeds, the plan exists in the DB without CLI overrides. This requires service-layer restructuring beyond the scope of this ticket.
- **Alembic migration naming** (#11 from review): Migration `m4_003` depends on `m6_005`, creating non-sequential naming. Renaming an existing migration risks breaking the chain for anyone who has already applied it.

### Quality Gates

| Session | Result |
|---------|--------|
| lint | PASS |
| typecheck | PASS (0 errors) |
| unit_tests | PASS (11,125 scenarios, 0 failures) |
| integration_tests | PASS (1,562 tests, 0 failures) |
| coverage_report | 97% (threshold: 97%) |

Closes #886

Reviewed-on: cleveragents/cleveragents-core#972
Reviewed-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
 2026-03-18 08:13:56 +00:00
aditya 2764fcef5c fix(actor,preflight,tests): resolve PR #975 review findings and stabilize full-suite coverage runs 
Address review-driven fixes across actor schema, preflight guardrails, docs/examples,
and Behave/Robot coverage: unify preflight warning behavior with shared role-warning logic,
resolve actor-name to config payloads in production preflight flow, harden response_format
validation/coercion edge cases, extract duplicated helper logic, and expand negative-path
test coverage. Also fix cross-scenario patcher leakage in step modules to eliminate
full-run-only coverage failures.
 2026-03-18 06:58:39 +00:00
aditya 26ad778aee feat(estimation): add estimation actor YAML template and role-aware registration validation 
- Add `role_hint` and `response_format` support to actor schema.
- Add non-fatal estimation-role compatibility warnings in actor registration CLI flows.
- Add preflight warning path when `estimation` actor is missing `response_format`.
- Add `examples/actors/estimator.yaml` and update actor examples documentation/tests.
- Update integration helper expectations (m1/m2/m3/m6) for missing provider config in local test env.

ISSUES CLOSED: #650
 2026-03-18 06:46:14 +00:00
hamza.khyari 5d6cb099ad feat(resource): add deferred virtual resource types 
Add 3 deferred virtual resource types (remote, submodule, symlink) with
equivalence metadata for physical-to-virtual resource linking.

Depends on: #662 (child_types reference types introduced by #662)

- Type definitions extracted to _resource_registry_virtual_deferred.py
  for consistency with _resource_registry_virtual.py (#329)
- YAML configs with equivalence criteria per spec, spec reference comments
- Bootstrap registration via BUILTIN_TYPES spread, hidden from
  resource add scaffolding (user_addable: false)
- Equivalence structural validation in ResourceTypeSpec model validator:
  criteria must be a non-empty list of non-empty strings; virtual types
  must have sandbox_strategy=none, user_addable=false, handler=None,
  all capabilities false
- Behave tests (52 scenarios), Robot tests (7), ASV benchmarks
- DB roundtrip tests verifying virtual types survive bootstrap persistence
- Negative tests: missing equivalence/name/kind, manual add rejection
  for all 3 virtual types (register_resource guard), invalid criteria
  elements (non-string, empty string)

ISSUES CLOSED: #331
 2026-03-18 02:30:00 +00:00
hamza.khyari c14ce65d61 feat(resource): add virtual core resource types 
Add 6 built-in virtual resource types (file, directory, commit, branch, tag,
tree) with equivalence metadata for content-hash and git-object identity
matching.

- YAML configs under examples/resource-types/
- Bootstrap registration with virtual types hidden from resource add scaffolding
- Equivalence criteria per spec (content_hash, merkle_hash, git SHA identity)
- Behave tests (83 scenarios), Robot tests, ASV benchmarks
- Documentation in docs/reference/resource_types_builtin.md

ISSUES CLOSED: #329
 2026-03-18 01:45:25 +00:00
freemo c65e8a5285 feat(resource): add cloud infrastructure resources 
Implement cloud resource types (aws, gcp, azure) with credential
fields, region/tenant metadata, and stubbed sandbox strategies.
Credential resolution uses environment variables and profile names
with no secrets logged.

Key changes:
- Add CloudResourceHandler with aws/gcp/azure type definitions
- Add credential resolution from env vars and profile names
- Add stubbed sandbox strategies (validate config, raise NotImplementedError)
- Register cloud types in bootstrap_builtin_types
- Credential masking via existing redaction patterns
- Add Behave BDD tests, Robot integration tests, ASV benchmarks

ISSUES CLOSED: #343
 2026-03-17 13:14:37 -04:00
Luis Mendes cb583021df test(e2e): E2E acceptance criteria for M6 (v3.5.0) — autonomy hardening 
Implemented Robot Framework E2E test suite for M6 autonomy hardening
acceptance criteria.  Tests exercise the real CleverAgents CLI with zero
mocking, covering session lifecycle, automation profiles, project setup,
plan lifecycle via A2A facade, guard enforcement, and a full autonomy
acceptance flow.  LLM-dependent tests use Skip If No LLM Keys for
graceful degradation when API keys are unavailable.

Hardened shared E2E keywords (common_e2e.resource):
- Safe JSON parsing with rfind-bounded extraction and error wrapping (C1, M1)
- Multi-object fallback: last-line reverse scan for multi-JSON output (M5)
- Moved Safe Parse Json Field to common_e2e.resource for reuse (L1)
- Migrated deprecated Run Keyword If to IF/ELSE blocks (M4)
- Added cwd parameter to Run CleverAgents Command (M6)
- API key protection via inline evaluation instead of RF variables (S1)
- Git return-code assertions in Create Temp Git Repo (L1)
- Removed unused Collections library import (L2)
- Warning log on directory removal failure instead of silent ignore (L6)

Hardened m6_acceptance.robot:
- Force Tags E2E instead of per-test [Tags] (L3)
- Per-test [Teardown] for resource cleanup, including Init test (M2, M9, L5)
- Collision-safe uuid4 hex suffix instead of randint (L7)
- Initialized session_id to EMPTY before test body for safe teardown (L2)
- ELSE branches with WARN log on conditional assertions (M3, M4, M6)
- Guard enforcement checks specific automation-profile fields (M5)
- Strengthened ci assertions with case-sensitive matching and JSON
  field parsing for config get verification (M9)
- Eliminated Python code injection via string interpolation (H2)
- Added assertions to Full Flow Apply Step keyword (H4)
- Removed redundant config set in Guard test (M10)
- Accurate CHANGELOG entry describing actual test scope (C2)

Post-review fixes applied:
- Verify all 8 built-in profiles (manual, review, supervised, cautious,
  trusted, auto, ci, full-auto) instead of 4 (M1)
- Session delete confirms removal via re-list (M2)
- Full Flow Apply Step verifies plan phase transition (M3)
- Plan execute output asserts plan_id presence + phase parsing (M4)
- Safe Parse Json Field gains two-strategy approach: outer-bracket
  extraction then last-line reverse scan fallback (M5)
- Removed redundant automation profile reset in Config test (L1)
- JSON-quoted assertions ("ci", "auto") prevent false-positive
  substring matches on short profile names (L3)

New E2E tests covering remaining acceptance criteria:
- Guard enforcement with custom profile: registers a profile with
  explicit denylist, budget cap, and tool-call limits, then verifies
  all guard fields via automation-profile show (AC-4 / H1)
- Profile precedence resolution: sets global profile to "review",
  creates plan with --automation-profile trusted, asserts plan output
  shows "trusted" not "review" (AC-5 / C2)
- Event queue via plan lifecycle transitions: creates plan, captures
  initial state, executes, verifies state transition proving domain
  event bus delivered and processed events (AC-3 / C1)
- Hierarchical decomposition via plan tree: creates plan with full-auto
  profile, executes, runs plan tree --format json, verifies decision
  nodes and children structure (AC-6 / C3)

Robot Framework uses dots as hierarchy separators in suite names
(e.g. "E2E.M6 Acceptance").  The E2E Suite Setup keyword replaced
spaces with underscores but preserved dots, producing directory names
like "E2E.M6_Acceptance".  The CLI init command derives the project
name from Path.cwd().name, and Project.validate_name() rejects dots
("Name must be alphanumeric with hyphens, underscores, or spaces").

Added a second Replace String call to convert dots to underscores so
the sanitized suite name passes Project name validation.

Also increased subprocess timeouts in m3_e2e_verification.robot (60s->120s)
and m4_e2e_verification.robot (30s->120s) to prevent flaky CI failures from
Python startup overhead.

The plan lifecycle E2E tests call "plan use local/code-review" but
actions are user-defined entities that must be explicitly registered
via "action create --config <yaml>" before use.  Without the action,
plan use failed with "Action 'local/code-review' not found" and all
LLM-dependent tests skipped.

Added action registration to M6 Suite Setup that dynamically selects
the actor matching the available API key (anthropic/claude-sonnet-4
when ANTHROPIC_API_KEY is set, openai/gpt-4o otherwise) and creates
the action YAML inline.

ISSUES CLOSED: #746
 2026-03-17 10:13:23 +00:00
Luis Mendes ab911dbdc4 fix(cli): write machine-readable formats directly to stdout bypassing Rich line-wrapping 
The format_output() function returned a string that callers passed to
Rich console.print(), which wraps long lines at terminal width.  This
injected literal newline characters into JSON string values (e.g. in
definition_of_done fields), producing invalid JSON that downstream
parsers could not decode (JSONDecodeError: Invalid control character).

For machine-readable formats (json, yaml, plain), format_output() now
writes the rendered output directly to sys.stdout and returns an empty
string.  This preserves the exact serialization from json.dumps/
yaml.dump without Rich text processing artifacts.

Refs: #746
 2026-03-17 09:53:57 +00:00
Luis Mendes 2acb957d83 fix(cli): replace in-memory automation-profile repository with database-backed persistence 
The automation-profile CLI commands used an _InMemoryProfileRepository
(a Python dict) that lost all data between CLI process invocations.
Profiles created with "automation-profile add" were invisible to
subsequent "automation-profile show" or "list" calls because each CLI
command is a separate process with a fresh empty dict.

Changes:
- Replaced _InMemoryProfileRepository with the real
  AutomationProfileRepository from the infrastructure layer, wired
  via the DI container following the same pattern as tool.py and
  session.py.
- Added auto_commit support to AutomationProfileRepository (matching
  the existing SessionRepository pattern) so that CLI commands running
  outside a UnitOfWork commit each operation automatically.
- Added safety_json and guards_json Text columns to the
  automation_profiles table (Alembic migration m6_005) for full-fidelity
  round-trip of the AutomationGuard and SafetyProfile sub-models.
  Previously, guards and several safety fields (max_cost_per_plan,
  max_retries_per_step, etc.) were silently dropped on persistence.
- Updated _from_domain, _to_domain, and _update_row to serialize and
  deserialize the full guard and safety sub-models via JSON, with
  backward-compatible fallback to legacy scalar columns.

Refs: #746
 2026-03-17 09:53:53 +00:00
brent.edwards c13a62a2f2 Merge branch 'master' into feature/m5-robot-tdd-tags 2026-03-16 23:06:19 +00:00
brent.edwards d0ca129d90 Merge branch 'master' into feature/m5-robot-tdd-tags 2026-03-16 23:05:43 +00:00
freemo 2688c85769 feat(extensibility): implement Custom Sandbox Strategy Registration via SandboxStrategy Protocol 
Implement SandboxStrategyProtocol, a 9-method @runtime_checkable Protocol enabling
third-party sandbox strategy registration. Includes:

- SandboxStrategyProtocol with create/read/write/diff/commit/rollback/checkpoint/
  restore_checkpoint/cleanup methods
- SandboxRef (frozen dataclass) and DiffView/DiffEntry (Pydantic models)
- SandboxStrategyRegistry with config-driven registration, Protocol validation,
  thread safety, and clear/list/has operations
- BuiltInSandboxStrategyAdapter wrapping existing Sandbox implementations to conform
  to the new Protocol
- CustomStrategyConfig for YAML/dict-based strategy registration
- SandboxFactory integration with custom_registry parameter,
  has_custom_strategy() and get_custom_strategy_class()
- 25 Behave BDD scenarios (85 steps) covering protocol, registry, adapter, config,
  and factory integration
- 8 Robot Framework integration tests with real filesystem operations
- ASV benchmarks for registry and adapter operations
- Developer documentation

ISSUES CLOSED: #586
 2026-03-16 22:50:27 +00:00
hurui200320 a5de448856 feat(testing): implement @tdd_expected_fail tag handling in Robot Framework 
Implements the three-tag TDD bug-capture system in Robot Framework via a
Listener v3 module, paralleling the Behave implementation. Tests tagged
tdd_expected_fail that fail have their result inverted to PASS (bug still
exists); tests that unexpectedly pass are inverted to FAIL with guidance.

Addresses all 15 findings from code review (PR !673, reviewer hamza.khyari):

P2 fixes:
- Added idempotency guard (_processed_tests set) to prevent double-inversion
  when the listener is loaded twice in the same process.
- Rewrote normal-test-unaffected check to run alongside a tdd_expected_fail
  fixture in a single Robot invocation, proving the listener is loaded and
  selectively applies rather than being a tautological pass.

P3 fixes:
- Added output.xml existence guard with clear diagnostics in _run_fixture.
- Documented intentional use of data.tags (static definition) vs result.tags
  (runtime-modifiable) in end_test docstring.
- Added SKIP status test fixture and integration test case.
- Added message content assertion in cmd_expected_fail_inverted.
- Tightened substring assertions to match specific error text.
- Added tdd_expected_fail-alone fixture (both companions missing).
- Added close() hook to clear _validation_errors and _processed_tests.
- Simplified _run_fixture return type to tuple[str, str].
- Changed listener path resolution from CWD-relative to __file__-relative
  in noxfile.py (integration_tests, slow_integration_tests, e2e_tests).

P4 fixes:
- Added __all__ declaration to helper module.
- Changed module docstring from "mirroring" to "paralleling".
- Added comment documenting accepted XML parsing risk (self-generated XML).

Additional fixes:
- Increased M4 E2E plan-tree test timeout from 30s to 120s (pre-existing
  timeout failure unrelated to this feature).

Quality gates (post-rebase onto latest master):
- nox -s lint: PASS
- nox -s typecheck: PASS (0 errors)
- nox -s unit_tests: PASS (10,700 scenarios)
- nox -s integration_tests: PASS (1,505 tests)
- nox -s coverage_report: PASS (97.9% >= 97% threshold)
- nox -s benchmark: PASS
- nox -s docs: PASS
- nox -s build: PASS
- nox -s security_scan: PASS
- nox -s dead_code: PASS

ISSUES CLOSED: #628
 2026-03-16 22:45:55 +00:00
hamza.khyari 89eaee008d feat(acms): implement UKO Layer 3 Technology Vocabularies (uko-py, uko-ts, uko-rs, uko-java) 
Implement Layer 3 technology-specific UKO vocabulary extensions for Python,
TypeScript, Rust, and Java with language-specific classes, properties, and
DetailLevelMap insertions.

- 4 OWL/Turtle ontology files with language-specific semantic classes
- DetailLevelMap insertion logic with correct integer reassignment
- Provenance contract (5 required fields per spec)
- Full 4-layer chain resolution (Layer 3 -> Layer 2 -> Layer 1 -> Layer 0)
- Comprehensive Behave test suite (63 scenarios)

ISSUES CLOSED: #576
 2026-03-16 12:11:08 +00:00
brent.edwards 3119383529 Merge branch 'master' into tdd/m6-checkpoint-real-rollback 2026-03-16 03:15:10 +00:00
brent.edwards 3eecb79003 test(plan): TDD failing tests for checkpoint real rollback (bug #822) 
TDD expected-fail tests proving bug #822 exists:
CheckpointService.rollback_to_checkpoint() returns a successful
RollbackResult but does not execute git reset --hard. Files modified
after the checkpoint remain unchanged after rollback.

Also fixes Robot Framework timeout robustness across the entire test
suite: all Run Process calls now use on_timeout=kill (prevents
SIGTERM-induced -15 exit codes under CI load) and timeouts increased
to 120s (prevents premature kills during heavy parallel execution).

ISSUES CLOSED: #839
 2026-03-16 01:45:50 +00:00
brent.edwards b67dc63eda test(plan): TDD failing tests for subplan spawn orchestration (bug #823) 
Write Behave scenario and Robot Framework test proving that
SubplanService.spawn() only creates metadata (SubplanStatus records
and SpawnMetadata) without creating real child Plan domain objects
or triggering lifecycle progression. Tests are tagged
@tdd_expected_fail so CI passes via result inversion.

ISSUES CLOSED: #838
 2026-03-16 01:12:24 +00:00
freemo 065171f21c test(e2e): E2E acceptance criteria for M2 (v3.1.0) — actor compiler and LLM integration 
Add Robot Framework E2E test suite robot/e2e/m2_acceptance.robot exercising
M2 acceptance criteria with zero mocking. Test creates a temp git repo with
sample project files, registers a custom actor via CLI, sets up resource and
project, creates an action referencing the actor, and runs the full plan
lifecycle (use → execute strategize → execute → diff → apply). Validates
actor YAML compilation, skill registry, tool lifecycle, and LLM integration
through real CLI invocations with real provider API keys. Uses flexible
structural assertions and expected_rc=None for LLM-dependent commands.

ISSUES CLOSED: #742
 2026-03-15 19:32:44 -04:00
freemo 5f07316641 fix: wire DI persistence and plan execute/apply for M1 lifecycle 
Fixed 5 bugs preventing the M1 E2E acceptance test from passing:

1. _get_lifecycle_service() in action.py and plan.py bypassed the DI
   container, creating PlanLifecycleService without UnitOfWork. All
   plan/action data was in-memory only and lost between subprocess
   calls. Now uses container.plan_lifecycle_service() for DB persistence.

2. `plan execute` CLI only called service.execute_plan() (a pure state
   transition) without running PlanExecutor phase processing. Rewrote
   to detect the plan's current phase/state and dispatch synchronously:
   Strategize/queued → run_strategize(), Strategize/complete → transition
   + run_execute(), Execute/queued → run_execute().

3. `plan apply` CLI had no plan_id argument. Added optional positional
   plan_id with _lifecycle_apply_with_id() that drives the plan through
   Apply/queued → Apply/processing → Apply/applied.

4. Preflight guardrail in start_strategize() built action_registry from
   the in-memory _actions dict only. Added get_action(plan.action_name)
   call to load the action from DB into cache before the guardrail check.

5. Robot Framework Create File syntax used continuation lines producing
   9 arguments instead of 1. Fixed to use Catenate SEPARATOR=\n then
   pass single variable to Create File. Also fixed --branch main to
   --branch master (git init default).

update mocks for execute_plan CLI changes across unit and integration tests

The new execute_plan() command calls _get_plan_executor() and
service.get_plan(plan_id) for phase/state detection. Existing tests
only mocked _get_lifecycle_service, so MagicMock defaults caused
phase/state comparisons to fail.

Changes across 14 files:
- Patch _get_plan_executor in all test setups that invoke the CLI
  execute command (Behave step files + Robot helper scripts)
- Set service.get_plan.return_value to real Plan objects with correct
  phase/state so the execute_plan dispatch logic works
- Fix error-path tests to use STRATEGIZE/COMPLETE plans so the error
  side_effects are actually reached
- Fix "Multiple plans eligible" → "Multiple plans ready" message text
  to match existing test expectations

increase Robot Framework subprocess timeouts for CI resource contention

Three integration tests were timing out in CI due to resource contention
when pabot runs multiple test suites in parallel. All three pass locally
and the timeouts were simply too tight for constrained CI environments.

- tdd_session_create_di.robot: 30s → 90s (DI container init + DB setup)
- database_integration.robot: 60s → 120s (Run Python Script keyword)
- m3_e2e_verification.robot: 60s → 120s (correction-live-revert spawns
  3 sequential CLI subprocesses with full container initialization)

ISSUES CLOSED: #789
 2026-03-15 20:50:02 +00:00
freemo cb3b7aab44 test(e2e): E2E acceptance criteria for M1 (v3.0.0) — minimal plan execution flow 
Added Robot Framework E2E test suite for M1 milestone acceptance criteria.
Tests the complete plan lifecycle (action create → resource add → project
create → plan use → plan execute strategize → plan execute → plan diff →
plan apply) with real LLM API keys and no mocking.

Key implementation details:
- Uses openai/gpt-4o-mini as strategy/execution actor (cost-effective)
- Simple definition_of_done: "Create a file called HELLO.md"
- Creates isolated temp git repo via Create Temp Git Repo keyword
- Extracts plan ID via ULID regex from plain-text output
- Uses expected_rc=None for LLM-dependent steps (execute, diff, apply)
  to handle non-deterministic LLM behavior gracefully
- Flexible structural assertions: checks rc, output presence, git log
- Skips gracefully when no LLM API keys (ANTHROPIC/OPENAI) are set
- Tagged [E2E] so it runs only in nox -s e2e_tests session

ISSUES CLOSED: #741
 2026-03-15 20:50:02 +00:00
freemo ec450e9085 fix(test): fix tolerant exit code and missing RC check in resource CLI test 
- Replace `rc == 0 or rc == 1` with strict `rc == 0` in resource type
  list test so failures are no longer silently accepted
- Capture and assert the return code of the Suite Setup database schema
  creation to fail fast if the setup itself is broken

apply strict RC checks to resource_cli.robot

- Capture return value of Run Process in Suite Setup and assert rc==0
- Replace tolerant RC check (rc==0 or rc==1) with strict Should Be Equal
  As Integers check for resource type list test case

broaden exception handling in resource CLI commands

Add catch-all `except Exception` handler after each `except
CleverAgentsError` block in all 14 resource CLI command handlers.
This ensures unexpected exceptions (e.g. sqlalchemy.exc.OperationalError)
are caught and displayed gracefully instead of producing raw tracebacks.

re-raise typer.Abort/Exit in broad exception handlers

The `except Exception` handlers added in the previous commit
inadvertently caught typer.Abort and typer.Exit, which are
subclasses of Exception (via click.exceptions).  This turned
successful CLI exits into aborts and double-handled already-caught
errors, breaking integration tests that rely on normal typer exit
behaviour.

Add an isinstance guard to re-raise typer.Abort and typer.Exit
before the catch-all handler runs.

Fixes #896
 2026-03-14 15:47:03 -04:00
freemo a69af285a8 fix(test): remove retry masking and output filtering in database integration test 
- Remove silent retry logic in Run Python Script that masked intermittent
  failures by retrying on empty stdout + non-zero RC
- Change output filter from 'in' to 'startswith' so only lines beginning
  with log prefixes are filtered, not lines containing them mid-string
- Remove [error from the filter list so error lines are never silently
  discarded from test output

Fixes #898
 2026-03-14 03:45:18 +00:00