test/cli-docstring-example-validation
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e201aa07c9 |
fix(resource): add overlay to ResourceTypeConfigSchema._VALID_STRATEGIES
CI / push-validation (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 28s
CI / build (pull_request) Successful in 28s
CI / lint (pull_request) Successful in 53s
CI / quality (pull_request) Successful in 52s
CI / typecheck (pull_request) Successful in 54s
CI / security (pull_request) Successful in 1m27s
CI / unit_tests (pull_request) Successful in 8m0s
CI / docker (pull_request) Successful in 1m54s
CI / coverage (pull_request) Successful in 15m56s
CI / integration_tests (pull_request) Successful in 29m30s
CI / status-check (pull_request) Successful in 3s
CI / helm (push) Successful in 28s
CI / lint (push) Successful in 43s
CI / push-validation (push) Successful in 21s
CI / build (push) Successful in 50s
CI / quality (push) Successful in 1m6s
CI / e2e_tests (push) Successful in 55s
CI / typecheck (push) Successful in 1m27s
CI / security (push) Successful in 1m27s
CI / benchmark-regression (push) Failing after 39s
CI / unit_tests (push) Successful in 15m13s
CI / docker (push) Successful in 1m36s
CI / integration_tests (push) Successful in 25m29s
CI / coverage (push) Successful in 16m5s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (push) Has been cancelled
Implemented test coverage and documentation updates to ensure overlay is properly recognized and communicated across the sandbox strategy validation surface.
- Verification: Confirmed that overlay already exists in the _VALID_STRATEGIES frozenset within src/cleveragents/resource/schema.py (introduced in a prior commit that closed issue #2827).
- Comprehensive audit: Audited strategy validation points to ensure consistency:
- _VALID_STRATEGIES in schema.py
- _IMPLEMENTED_STRATEGIES and _SUPPORTED_STRATEGIES in factory.py
- SandboxStrategy enum definitions in both resource.py and resource_type.py
All include overlay as an allowed strategy.
- Tests: Added a new BDD scenario titled: "Invalid sandbox_strategy error message lists overlay as allowed" to features/consolidated_resource.feature. This asserts that validation errors explicitly list overlay among the allowed values when an invalid strategy is used.
- Documentation: Updated docstring in src/cleveragents/infrastructure/sandbox/strategy_registry.py to reflect that the built-in SandboxFactory handles six strategies, including snapshot and overlay.
- Design considerations: The fix itself was already applied in a previous commit (
|
||
|
|
bdd1ea4f3a |
fix(checkpoint): wire CheckpointManager into PlanExecutor execution path
CI / push-validation (pull_request) Successful in 10s
CI / build (pull_request) Successful in 16s
CI / helm (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 30s
CI / lint (pull_request) Successful in 35s
CI / quality (pull_request) Successful in 35s
CI / security (pull_request) Successful in 50s
CI / integration_tests (pull_request) Successful in 4m3s
CI / e2e_tests (pull_request) Successful in 4m14s
CI / unit_tests (pull_request) Successful in 5m13s
CI / docker (pull_request) Successful in 8s
CI / coverage (pull_request) Successful in 11m2s
CI / status-check (pull_request) Successful in 1s
CheckpointManager was never wired into PlanExecutor — the CLI factory constructed PlanExecutor without a checkpoint_manager (defaulted to None), silently skipping all checkpoint hooks. Fix: - Register CheckpointManager as Singleton in DI container - Resolve container singleton in _get_plan_executor() and pass to PlanExecutor constructor - Bridge infra→domain: _try_create_checkpoint() now persists last_checkpoint_id on the plan via _commit_plan(), raises PlanError if persistence fails - Default checkpointable=True for writable+sandboxable resources and write-capable tools (model_validators on ResourceCapabilities and ToolCapability) - Validate that non-writable/non-sandboxable resources cannot be checkpointable (ValueError guard) - Add post-execute A2A facade notification using plan.status to avoid duplicate execute→execute transition errors Tests: - 10 Behave scenarios covering DI wiring, singleton identity, checkpoint creation, plan metadata update, rollback, graceful fallback, no-arg constructor, capability defaults (positive + 2 negative) - Updated consolidated_resource, consolidated_skill, and Robot helper_skill_flatten for new checkpointable defaults ISSUES CLOSED: #1253 |
||
|
|
dbf3b8d7f7 |
fix(resources): remove overlay from SandboxStrategy enum - not in spec
CI / lint (pull_request) Successful in 27s
CI / typecheck (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 38s
CI / security (pull_request) Successful in 58s
CI / build (pull_request) Successful in 29s
CI / helm (pull_request) Successful in 23s
CI / unit_tests (pull_request) Successful in 6m46s
CI / e2e_tests (pull_request) Successful in 18m2s
CI / integration_tests (pull_request) Successful in 22m24s
CI / docker (pull_request) Successful in 1m20s
CI / coverage (pull_request) Successful in 11m11s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m8s
Add 'overlay' to _VALID_STRATEGIES frozenset in ResourceTypeConfigSchema validator so that resource type YAML configs specifying sandbox_strategy: overlay are accepted without raising ValueError. The overlay strategy is defined in the specification for fs-mount resources using OverlayFS. The domain model SandboxStrategy enum correctly includes OVERLAY = 'overlay', but the YAML schema validator in src/cleveragents/resource/schema.py was missing it from _VALID_STRATEGIES, causing a validation error on valid configurations. Changes: - Add 'overlay' to _VALID_STRATEGIES in src/cleveragents/resource/schema.py - Add BDD scenario: 'Schema accepts overlay sandbox strategy' - Add step definitions for the new scenario ISSUES CLOSED: #2827 |
||
|
|
8a87262f86 |
feat(sandbox): implement overlay filesystem sandbox strategy (#994)
CI / build (push) Successful in 17s
CI / lint (push) Successful in 3m41s
CI / quality (push) Successful in 3m47s
CI / unit_tests (push) Successful in 3m58s
CI / typecheck (push) Successful in 4m17s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m22s
CI / docker (push) Successful in 1m22s
CI / e2e_tests (push) Successful in 8m57s
CI / coverage (push) Successful in 11m16s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 19m45s
CI / integration_tests (push) Failing after 20m53s
## Summary Implement the overlay filesystem sandbox strategy with OverlayFS support and userspace fallback. ### Implementation **`OverlaySandbox`** (`infrastructure/sandbox/overlay.py`, 497 lines): - Detects OverlayFS availability at runtime via `/proc/filesystems` + `os.geteuid() == 0` check - **Real OverlayFS mode** (requires root): creates upper/work/merged dirs, mounts overlay filesystem, captures writes in upper layer - **Userspace fallback** (default in CI/containers): `shutil.copytree` the original into merged dir, tracks changes via `filecmp` diff on commit - `create()`: sets up directory structure, mounts if available - `commit()`: copies changed/added files from overlay to original, removes deleted files - `rollback()`: unmounts (or removes) merged, recreates from scratch - `cleanup()`: unmounts, removes all temp dirs, idempotent ### Domain Model Updates - Added `OVERLAY = "overlay"` to `SandboxStrategy` enum in both `resource_type.py` and `resource.py` - Added `STRATEGY_OVERLAY` to `SandboxFactory`, registered for `fs-mount`, `fs-directory`, `fs-file` resources ### Tests - **22 Behave scenarios**: full lifecycle (create/commit/rollback/cleanup), status transitions, path traversal guard, fallback detection, error handling - **6 Robot integration tests**: end-to-end overlay sandbox operations ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,917 scenarios) | | `nox -s coverage_report` | 97% (>= 97%) | Closes #880 Reviewed-on: #994 Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com> |
||
|
|
b2e923173f |
perf(tests): consolidate 141 trivially small feature files into 34 domain groups
CI / lint (pull_request) Successful in 18s
CI / typecheck (pull_request) Successful in 32s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 33s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / unit_tests (pull_request) Successful in 2m5s
CI / integration_tests (pull_request) Successful in 2m50s
CI / docker (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 4m19s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 18s
CI / security (push) Successful in 30s
CI / build (push) Successful in 22s
CI / typecheck (push) Successful in 58s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m6s
CI / docker (push) Successful in 50s
CI / integration_tests (push) Successful in 3m21s
CI / coverage (push) Successful in 4m12s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (pull_request) Successful in 24m46s
Consolidate 141 BDD feature files that each complete in under 0.1 seconds into 25 domain-grouped feature files, reducing subprocess count from 339 to ~223. Each consolidated file groups scenarios from the same domain/module that share step definitions and fixtures. All scenarios are preserved with clear comment headers indicating their original source file. This reduces subprocess overhead by ~116 invocations (141 original files replaced by 25 consolidated files), targeting the 42% of subprocess count that contributed only 0.2% of actual test runtime. ISSUES CLOSED: #485 |