Commit Graph

17 Commits

Author SHA1 Message Date
freemo 02250473ad fix(ci): restore all CI quality gates to passing on master
Fix all failing CI quality gates (lint, unit_tests, format) without
suppressing any quality enforcement.

Root causes and fixes:

1. Format: features/steps/plan_namespaced_name_tdd_steps.py had trailing
   whitespace; fixed by running ruff format.

2. Unit tests - A2A JSON-RPC 2.0 migration (commit 9c6d6915) renamed
   A2aRequest fields (operation→method, request_id→id, a2a_version→jsonrpc)
   and A2aResponse fields (status+data→result, request_id→id) but did not
   update all step files and feature files:
   - a2a_jsonrpc_wire_format_steps.py: added use_step_matcher('re') and
     reset to 'parse' at end to prevent parallel test interference
   - a2a_facade_wiring_steps.py: updated operation= to method=, .status/.data
     to .result
   - a2a_facade_steps.py: updated request_id→id, a2a_version→jsonrpc,
     A2aResponse(request_id=..., status=...) to new API
   - m6_facade_steps.py: updated all old API usage
   - devcontainer_cleanup_steps.py: updated A2aRequest(operation=...)
   - plan_prompt_command_steps.py: updated A2aRequest(operation=...)
   - wf03_plan_prompt_confidence_steps.py: updated A2aRequest(operation=...)
   - consolidated_misc.feature: updated old A2aRequest/A2aResponse scenarios

3. Unit tests - Session CLI output changed (commit 0d5d9cf0 and others):
   - 'Session Created' → 'Session created' (lowercase)
   - 'Session Details' → 'Session Summary'
   - 'Sessions (N total)' → 'Sessions'
   - session list JSON: top-level 'total' → nested 'summary.total'
   - Fixed in: session_cli.feature, session_cli_coverage_boost.feature,
     session_cli_uncovered_branches.feature, session_list_error.feature,
     tdd_session_create_persist_steps.py

4. Unit tests - Plan list output changed (commit 1a07a891):
   - 'V3 Lifecycle Plans' → 'Plans'
   - 'Lifecycle Plans' → 'Plans'
   - Name column removed (restored in source)
   - Invariants column removed (restored in source)
   - Project truncation removed (restored in source)
   - Fixed in: plan_cli_cancel_revert_coverage.feature,
     plan_lifecycle_cli_coverage.feature, plan_cli_coverage_boost_steps.py,
     plan.py (source code restored)

5. Unit tests - Plan apply command now requires ULID (commit 300a5d6d):
   - plan_cli_coverage_r3.feature: updated 'PLAN-001' to valid ULID
   - plan_cli_coverage_r3_steps.py: added --yes flag, added new step for
     no-eligible-plans path

6. Unit tests - Various source code bugs:
   - ThoughtBlock: converted from @dataclass to Pydantic BaseModel
     (architecture test requires all dataclasses to use Pydantic)
   - session.py: added DatabaseError handling to export, import, tell commands
   - database.py: fixed rollback_to() to reuse checkpoint connection for writes
   - database.py: added _get_checkpoint_conn() helper
   - check-tls-cert.py: fixed SSLCertVerificationError.reason AttributeError

7. Unit tests - Test step bugs:
   - error_recovery_coverage_boost_steps.py: fixed invalid ULID _PLAN_ID
   - session_service_coverage_steps.py: fixed 'sha256:' prefix bug in checksum
   - database_models_new_coverage_steps.py: added 'name' field to session mock
   - async_audit_recording_steps.py: fixed Settings(audit_async=False) via env var
   - coverage_threshold_config_steps.py: added --coverage-min pattern support
   - m5_acms_smoke_steps.py: updated usage hint text
   - actor_cli_yaml_steps.py: updated 'Removed actor' → 'Actor removed'
   - aimodelscredentials_steps.py: set context.imported_class in import step
   - domain_base_model.feature: added missing 'When I examine model_config' step
   - tui_first_run_steps.py: fixed module reload to restore cleveragents.tui.*
     modules after test (prevented patch interference in subsequent tests)
   - tui_first_run_steps.py: added set_search('') step for empty string
   - resource_handler_base_coverage_r3_steps.py: use _MinimalHandler instead
     of DatabaseResourceHandler for NotImplementedError tests
   - resource_handler_crud.feature: updated to test new DatabaseHandler behavior
   - resource_handler_sandbox.feature: updated to test new DatabaseHandler behavior
   - tdd_json_decode_crash_persistence.feature: fixed @tdd_bug → @tdd_issue tags

8. Parallel test interference:
   - All step files using use_step_matcher('re') now reset to 'parse' at end
     to prevent global matcher state leaking to subsequent step files
2026-04-04 20:38:16 +00:00
freemo ee980ee117 feat(resource): implement DevcontainerHandler missing protocol methods (#1286)
Implements the four missing protocol methods on DevcontainerHandler required by Epic #825 (ResourceHandler Protocol Completion):

- delete(): uses 'devcontainer exec rm -rf' to delete files/dirs inside the container; returns DeleteResult(success=False) for missing/stopped containers rather than raising.
- list_children(): uses 'devcontainer exec ls -1' to enumerate workspace entries; returns an empty list on container failure.
- diff(): compares content hashes between the devcontainer workspace and another filesystem location; uses EMPTY_CONTENT_HASH sentinel to treat both-absent as no-change.
- create_sandbox(): delegates to BaseResourceHandler.create_sandbox with lazy activation (same DETECTED/STOPPED/FAILED guard as resolve()).

All methods raise ValueError for resources with no location.

Adds 18 Behave BDD scenarios covering success paths, failure/stopped-container paths, empty-path edge cases, and ValueError guards.

ISSUES CLOSED: #1242

Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-02 17:08:52 +00:00
freemo ec52acffb7 feat(resource): implement DatabaseResourceHandler CRUD and checkpoint methods
Implements all missing CRUD and checkpoint methods on DatabaseResourceHandler per issue #1241 (Epic #825 ResourceHandler Protocol Completion):

- read(): SQLite queries sqlite_master for schema; remote returns connection-info
- write(): SQLite executes SQL statements; remote returns not-supported result
- delete(): SQLite executes DROP TABLE IF EXISTS; remote returns not-supported
- list_children(): SQLite lists tables/views from sqlite_master; remote returns []
- diff(): compares schemas via content_hash on both sides
- create_checkpoint(): SQLite creates SAVEPOINT on open connection; remote uses content-hash fallback
- rollback_to(): SQLite executes ROLLBACK TO SAVEPOINT; remote returns not-supported

All methods handle errors gracefully (no unhandled exceptions). Adds comprehensive Behave BDD tests covering all methods for both SQLite and remote database types.

ISSUES CLOSED: #1241

Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-02 17:08:05 +00:00
hamza.khyari 18b9d61e35 feat(resource): implement ResourceHandler content_hash method
Add content_hash(resource, *, algorithm='sha256') -> str to the
ResourceHandler protocol and all handler implementations:

- Protocol: new content_hash method on ResourceHandler (protocol.py)
- BaseResourceHandler: default impl hashes file content or directory
  entry names; returns EMPTY_CONTENT_HASH sentinel for missing
  resources
- GitCheckoutHandler: hashes git rev-parse HEAD through the requested
  algorithm for consistent digest format
- FsDirectoryHandler: recursive walk hashing sorted relative paths
  and file contents (content-only, ignores metadata)
- DevcontainerHandler: hashes devcontainer.json config file
- DatabaseResourceHandler: hashes connection string; for SQLite
  file-based DBs, hashes the database file content
- _DefaultHandler: delegates to BaseResourceHandler

EMPTY_CONTENT_HASH sentinel is the SHA-256 of empty input
(e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855).

Hash algorithm is configurable via the algorithm parameter (default
sha256, accepts any hashlib.new()-compatible name).

Behave tests (10 scenarios): sentinel for missing/nonexistent,
determinism, different content produces different hash, fs-directory
recursive hash with change detection, git-checkout hash, configurable
algorithm (sha256 vs sha512), protocol compliance for all 4 handlers.

ISSUES CLOSED: #837
2026-04-01 13:02:24 +00:00
hamza.khyari 7bb77aee5c feat(resource): implement ResourceHandler sandbox and checkpoint methods
Extend the ResourceHandler protocol with four sandbox/lifecycle methods:
create_sandbox (idempotent), create_checkpoint, rollback_to, project_access.

Fixes from self-review:
- Instance-level checkpoint dict instead of ClassVar (no shared state)
- Git rollback uses 'git reset --hard' + 'git clean -fd' (not just checkout)
- FsDirectory rollback skips .git to preserve git metadata
- project_access separates ImportError (local mode) from ValueError (reject)
- Added discard_checkpoints() cleanup method
- Git rollback test verifies file content restored

ISSUES CLOSED: #836
2026-03-27 10:50:15 +00:00
hamza.khyari 90e5bbb99c feat(resource): implement ResourceHandler CRUD and discovery methods
Extend the ResourceHandler protocol with six content operations (read,
write, delete, list_children, diff, discover_children) and four frozen
dataclass result types (Content, WriteResult, DeleteResult, DiffResult).

Handler implementations:
- GitCheckoutHandler: read via git show (binary-safe), write/delete via
  filesystem ops, list via git ls-tree, diff via git diff --no-index
  with locale-safe shortstat parsing, discover via git ls-tree -d
- FsDirectoryHandler: full CRUD via pathlib/os/difflib/shutil
- DevcontainerHandler: read/write/discover via devcontainer exec
- CloudResourceHandler: NotImplementedError stubs for protocol compliance
- DatabaseResourceHandler: inherits base NotImplementedError stubs

Security:
- Path traversal guard (_safe_resolve) on all read/write/delete ops
  using os.sep-suffixed startswith check to prevent prefix collisions
- Empty-path deletion rejected with PermissionError

Tests:
- 22 Behave scenarios (115 steps): CRUD for FsDirectory and GitCheckout,
  path traversal rejection (3 scenarios), NotImplementedError defaults
- 2 Robot integration tests: read -> write -> diff cycle on real temp
  directories and git repos

ISSUES CLOSED: #827
2026-03-24 13:53:41 +00:00
freemo c65e8a5285 feat(resource): add cloud infrastructure resources
Implement cloud resource types (aws, gcp, azure) with credential
fields, region/tenant metadata, and stubbed sandbox strategies.
Credential resolution uses environment variables and profile names
with no secrets logged.

Key changes:
- Add CloudResourceHandler with aws/gcp/azure type definitions
- Add credential resolution from env vars and profile names
- Add stubbed sandbox strategies (validate config, raise NotImplementedError)
- Register cloud types in bootstrap_builtin_types
- Credential masking via existing redaction patterns
- Add Behave BDD tests, Robot integration tests, ASV benchmarks

ISSUES CLOSED: #343
2026-03-17 13:14:37 -04:00
freemo c054675167 feat(resource): add database resources
Implement database resource types (postgres, mysql, sqlite, duckdb)
with connection args, auth handling, and transaction-based sandbox
strategy using BEGIN/ROLLBACK/COMMIT wrappers.

Key changes:
- Add DatabaseResourceHandler with 4 database type definitions
- Implement TransactionSandbox for transaction_rollback strategy
- Wire TransactionSandbox into SandboxFactory
- Register database types in bootstrap_builtin_types
- Add connection validation with credential masking
- Add Behave BDD tests, Robot integration tests, ASV benchmarks

ISSUES CLOSED: #342
2026-03-10 19:29:27 +00:00
CoreRasurae cf67ba0a86 feat(devcontainer): add lazy activation and lifecycle management
Implemented lazy container activation for devcontainer-instance resources
with ContainerLifecycleState enum tracking six states (inactive, starting,
active, stopping, stopped, error) with validated transitions. Extended
DevcontainerHandler with devcontainer up CLI integration and JSON output
parsing for container start. Added periodic health checking via
devcontainer exec ping with configurable interval. Added agents resource
stop and agents resource rebuild CLI commands for manual lifecycle
control. Wired session close and plan completion hooks to automatic
container cleanup. Includes lifecycle state persistence in resource
registry with timestamped transitions. Added Behave BDD tests, Robot
integration tests, and ASV activation latency benchmarks.

- Added remoteWorkspaceFolder absolute-path validation
- Aligned spec: handler name, rebuild types, --yes flag on stop/rebuild
- Added registry re-read in stop_container success path for consistency
- Added session_id field to ContainerLifecycleTracker for scoped cleanup
- Scoped stop_all_active_containers to session_id when provided
- Wired _cleanup_devcontainers into fail_apply and fail_execute
- Wired start_health_check into activate_container success path
- Restructured facade session close to always run container cleanup
  even without session service (F4)
- Re-read tracker from registry in activate_container success path
- Added evict_terminal_trackers to cap registry growth
- Updated devcontainer_resources.md: health check auto-start, scoped
  cleanup hooks, known limitations for eviction and sandbox_strategy
- Wired evict_terminal_trackers into stop_all_active_containers so
  terminal-state trackers are actually evicted in production
- Made stop_container idempotent: returns early when container is
  already in a terminal state instead of raising ValueError
- Fixed benchmark health check thread leak in TimeActivationLatency
  by clearing registry after each timing loop
- Added rebuild pass-through (--reset-container flag to devcontainer up)
- Added host_workspace_path field on ContainerLifecycleTracker so
  health probes use the host-side path for devcontainer exec
- Wired lazy activation into DevcontainerHandler.resolve() for
  devcontainer-instance resources in non-running states
- Changed _default_strategy from SNAPSHOT to NONE (container
  itself provides isolation; SandboxFactory raises NotImplementedError
  for snapshot)
- Restricted _STOPPABLE_TYPES to devcontainer-instance only
  (container-instance is not directly stoppable via CLI)

ISSUES CLOSED: #514
2026-03-10 12:17:51 +00:00
hamza.khyari 135481f9a6 fix(resource): address PR #618 review findings (round 1)
- P2-1: Add field_validator('inherits') on ResourceTypeSpec
- P2-2: Remove 'properties' from _COLLECTION_FIELDS (scalar override)
- P2-3: Add behave scenarios for exception rollback paths
- P2-4: Add ordering comment to BUILTIN_TYPES
- P2-5: Define RegistryHost Protocol, remove type:ignore on mixins
- P2-6: Add if_not_exists=True on migration index creation
- P3-1: Use deque.popleft() in get_ancestors for O(1) BFS
- P3-2: Filter private attrs in _to_dict vars() fallback
- P3-3: Replace _handler_cache import with clear_handler_cache()
- P3-4: Add migration downgrade test for m6_004
- P3-5: Extract ResourceDagMixin to _resource_registry_dag.py

All files remain under 500-line CONTRIBUTING limit.
Lint, typecheck, and 85 behave scenarios pass.
2026-03-10 04:06:02 +00:00
hamza.khyari 23d8a53f6a fix(resource): address review findings for resource type inheritance
Fix 25 of 27 review findings from PR #618 code review:

P1 (Must Fix):
- F3: Fix silent data corruption in _merge_collection for properties
  dict fields (was falling through to string list merge)
- F1: Split 1023-line step file into 3 files + helper module (all <500)
- F4: Add SELECT FOR UPDATE lock on parent type in register_type to
  prevent TOCTOU race in concurrent registrations
- F6: Fix docs claiming exceptions inherit from CleverAgentsError
  (they inherit from ValueError)
- F7: Fix docs incorrectly describing validate_chain return type

P2 (Should Fix):
- F9: Replace dict[str, Any] with TypeRegistryMap type alias
- F10: Replace import logging with structlog in inheritance.py
- F11: Add __all__ to inheritance.py
- F13: Fix _load_type_registry to derive built_in from namespace column
- F14: Add warning log for unregistered types in resolve_inheritance_chain
- F15: Return defensive copies from resolve_fields
- F16: Add chain validation to bootstrap_builtin_types
- F17: Narrow except Exception to specific types in step files
- F18: Add side-effect verification scenarios after error cases
- F19: Always include inherits key in JSON output for consistent schema
- F20: Log actual exception instead of hardcoded string in CLI

P3 (Nit):
- F21: Reject whitespace-only inherits values in validate_chain
- F23: Wrap chain errors in HandlerResolutionError in resolver
- F24: Return defensive copies from as_cli_dict
- F25: Replace tautological assertion with ResourceHandler isinstance
- F26: Add whitespace inherits test scenario
- F27: Fix find_subtypes docstring to note it excludes ancestor_name

Deferred:
- F2: type: ignore in step files — pyright only checks src/, matches
  existing project pattern (91 occurrences in resource_dag_steps.py)
- F5: CLI integration tests require full DI container setup
- F8: resource_registry_service.py size is pre-existing (971 on master)
- F12: Coverage boost file changes are test adaptations, not scope creep
- F22: FK constraint intentionally omitted per docs (SQLite compat)
2026-03-10 03:39:32 +00:00
hamza.khyari bb8175aa11 feat(resource): add resource type inheritance and polymorphic tool matching
Implement ADR-042 single-inheritance for resource types with polymorphic
tool and handler resolution.

Core changes:
- Add `inherits` field to ResourceTypeConfigSchema, ResourceTypeSpec, and
  ResourceTypeModel with chain depth validation (max 5), circular
  inheritance detection, and built-in-from-custom guard
- New inheritance.py module (390 lines): resolve_inheritance_chain(),
  validate_chain(), is_subtype_of(), resolve_fields(), find_subtypes()
- Wire inheritance into ResourceRegistryService: chain validation on
  register_type(), resolve_type_inheritance_chain(), is_subtype_of()
- Add find_tools_for_resource() to ToolRegistry with polymorphic matching
  that walks the inheritance chain
- Add resolve_handler_polymorphic() to handler resolver
- Alembic migration m6_004_resource_type_inherits adds inherits column
  and index to resource_types table

CLI changes:
- `agents resource type list` shows Inherits column
- `agents resource type show` displays inheritance chain

Tests:
- 30 BDD scenarios in resource_type_inheritance.feature (all pass)
- 18 Robot Framework test cases
- 14 ASV benchmark timing methods across 4 suites

Docs:
- docs/reference/resource_type_inheritance.md (full reference)
- Updated docs/schema/resource_type.schema.yaml
- CHANGELOG entry for #513

ISSUES CLOSED: #513
2026-03-10 03:39:32 +00:00
freemo 279c6112ae feat(resource): add devcontainer resource type and auto-discovery handler
Added three new built-in resource types: container-instance,
devcontainer-instance, and devcontainer-file. The devcontainer-instance
type inherits from container-instance per ADR-042 and is auto-discovered
when git-checkout or fs-directory resources contain .devcontainer/
directories per ADR-043.

Implementation includes:
- DevcontainerHandler extending BaseResourceHandler with snapshot strategy
- Auto-discovery module scanning .devcontainer/devcontainer.json and
  root .devcontainer.json with JSON validation
- CLI support for devcontainer-instance and container-instance via
  agents resource add with --path and --image flags
- Behave feature with 22 scenarios covering manual registration,
  auto-discovery, invalid JSON handling, and protocol conformance
- Robot integration tests with 10 test cases for CLI round-trip and
  DAG hierarchy validation
- ASV benchmarks measuring discovery throughput with varying subdirectory
  counts, handler resolver cache performance, and result construction
- Reference documentation at docs/reference/devcontainer_resources.md

ISSUES CLOSED: #511
2026-03-02 22:09:37 -05:00
khyari hamza 1a0a501f66 refactor(resource): address PR review feedback for handler runtime
- extract BaseResourceHandler to eliminate ~90% duplication between
  GitCheckoutHandler and FsDirectoryHandler
- raise RuntimeError when sandbox.context is None instead of silent
  empty string fallback
- add threading.Lock to resolver handler cache for thread safety
- type resource_lookup/type_lookup as Callable instead of Any
- log original HandlerResolutionError at DEBUG before fallback
- use behave.runner.Context in step definitions per repo convention
2026-02-23 22:20:44 +00:00
khyari hamza bfc6ab0fb1 feat(resource): add handler runtime for git-checkout and fs-directory 2026-02-23 22:20:44 +00:00
Jeffrey Phillips Freeman 5a7703473a feat(resource): add git-checkout and fs-directory resource types 2026-02-15 10:44:14 -05:00
Jeffrey Phillips Freeman 1f06bb2f76 feat(resource): add resource type model + schema loader 2026-02-15 15:19:09 +00:00