Compare commits

..

4 Commits

Author SHA1 Message Date
HAL9000 534fbba5db fix(cli): correct parse_args override type to satisfy TyperGroup contract
CI / lint (pull_request) Successful in 46s
CI / typecheck (pull_request) Successful in 1m1s
CI / security (pull_request) Successful in 1m13s
CI / helm (pull_request) Successful in 25s
CI / push-validation (pull_request) Successful in 23s
CI / build (pull_request) Successful in 44s
CI / quality (pull_request) Successful in 49s
CI / unit_tests (pull_request) Failing after 5m9s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / integration_tests (pull_request) Failing after 9m7s
CI / status-check (pull_request) Failing after 3s
TyperGroup.parse_args is typed against typer._click.core.Context (the
vendored click), which is not assignable to click.Context or
typer.Context (typer.models.Context). Use Any for the ctx parameter to
satisfy the override constraint without depending on typer's private
_click module path.

ISSUES CLOSED: #6879
2026-06-02 06:47:28 -04:00
HAL9000 0e0aec0f16 fix(cli): handle standalone short options like -f that take values
When a global short option like -f (for --format) appears after a subcommand
as a standalone token (e.g., 'agents info -f json'), it was not being promoted
because the code only handled fused forms like '-fjson'.

This fix adds a check for standalone short options in _GLOBAL_VALUE_SHORT_OPTIONS
that consume the following argument, allowing 'agents info -f json' to work
correctly in addition to 'agents info -fjson'.
2026-06-02 06:47:28 -04:00
HAL9000 cb77abb6c7 chore: add contributor entry for CLI global format flag fix (#6879) 2026-06-02 06:47:28 -04:00
HAL9000 06d13687c4 fix(cli): allow global format flag after subcommands
Closes #6879\n\nISSUES CLOSED: #6879
2026-06-02 06:47:28 -04:00
33 changed files with 439 additions and 3346 deletions
+5 -21
View File
@@ -6,7 +6,6 @@ Changed `wf10_batch.robot` to be less likely to create files, and
`plan_generation_graph.robot` to give more test answers.
## [Unreleased]
- **fix(cli/plan): plan correct JSON output envelope fix and BDD test coverage** (#8584 / PR #8662): Restructured `agents plan correct --format json` output to nest correction fields under `data.correction` (e.g., `data.correction.mode`) and populate the spec-required CLI envelope with `command="plan correct"`, `status`, `exit_code`, `timing`, and `messages` fields. Added three BDD scenarios in `features/tdd_plan_correct_json_output.feature` validating the envelope structure for both revert and append modes.
- **fix(cli): add --url flag to resource add for git resource type** (#6322): Added support for the `--url` flag on `agents resource add git` command, allowing users to specify a remote URL for git resources. The flag is validated to only apply to git resource types. Includes Behave BDD tests in `features/resource_cli_git_url_flag.feature` and Robot Framework integration tests verifying correct URL validation and CLI behavior.
- **Session create JSON envelope** (#6441): Fixed `agents session create --format json` returning a flat `data` dict instead of the spec-required nested structure with `data.session`, `data.settings`, and `data.actor_details` sub-objects. The `command` field is now populated correctly. Extended JSON envelope coverage to `agents session list`, `show`, `delete --format json`, `export --output-format json`, and `import --format json` so all session commands emit a structured `messages[].text` field (`"0 sessions listed"`, `"Session details loaded"`, `"Session deleted"`, `"Export completed"`, `"Import completed"`).
- **fix(resources): remove unsupported executable resource type and fix resource list columns** (#3077 / PR #3248): Removed `executable` from `LSP_RESOURCE_TYPES` and `BUILTIN_TYPE_NAMES` (the specification defines no such built-in type). Updated `agents resource list` CLI table columns from `[ID, Name, Type, Status, Kind, Location, Description]` to the spec-required `[Name, ID, Type, Phys/Virt, Children, Projects]`. Deleted orphaned `examples/resource-types/executable.yaml`. Lifecycle state for container resources is now displayed as a note below the resource table.
@@ -164,16 +163,6 @@ ensuring data is stored with proper parameter values.
LLM streaming output. A `SessionActorNotConfiguredError` is raised with exit
code 1 when no actor is configured.
### Added
- **Automated CLI Docstring Example Validation** (#9106): Added `DocstringExampleValidator`
in `src/cleveragents/cli/docstring_validator.py` that introspects Typer command signatures
and validates `Examples:` sections to ensure positional arguments appear before option
flags. Validation runs automatically via `nox -s unit_tests` through the new Behave feature
`features/cli_docstring_example_validation.feature`. Fixed `rollback_plan` docstring in
`src/cleveragents/cli/commands/plan.py` to show correct positional argument order.
CONTRIBUTING.md updated with the required CLI docstring example style guide.
### Fixed
- **fileConfig error handling in alembic env.py** (#7874): Wrapped the `fileConfig()`
@@ -982,16 +971,6 @@ Documentation Report (Cycle N)` issues every 10 cycles (~3.3 hours). The manager
`N_QUARTER` comment to enumerate the pools it covers (UAT, bug hunting, test
infra). Prevents confusion about which supervisor handles PR fix work.
- **Actor CLI Exception Handling Refinement** (#8567): `_get_services()` in
`actor.py` now carries an explicit `tuple[Any, Any | None]` return type
annotation. `_load_config_text()` now catches `yaml.YAMLError` and `TypeError`
(instead of `ValueError`/`AttributeError`) around `yaml.safe_load()`, preserving
the user-friendly `typer.BadParameter` message for malformed YAML.
`_compute_actor_impact()` defensive guards now also catch `CleverAgentsError`,
SQLAlchemy `OperationalError`, and `ValidationError` in addition to
`AttributeError`/`RuntimeError`, keeping actor removal resilient when the
database layer is unavailable.
- **Decision Tree Full ULID Display** (#5825): The `agents plan tree` command now
displays full 26-character ULIDs for all decisions instead of truncating them to
8 characters. This enables users to copy decision IDs directly from tree output
@@ -1118,6 +1097,11 @@ iteration` and data corruption under concurrent plan execution. All public
The `export` command gains `--output-format` and the `import` command gains `--format`
to select the output envelope format independently of the export/import file format.
- **CLI global format flag parsing** (#6879): Promotes global CLI options before
Typer parsing so `agents <command> --format json` and `-f` shorthand work from
the documented position. Adds regression coverage for version, info, and
diagnostics commands.
- **Robot Framework TDD Listener Guards** (#5436): Added three guard conditions to the
`tdd_expected_fail_listener` `end_test()` function to prevent blindly inverting ALL test
failures to passes, which was masking infrastructure errors and causing flaky CI behavior.
-39
View File
@@ -1228,45 +1228,6 @@ These rules are enforced by the test environment hooks and CI quality gates:
- A bug fix PR that closes issue `#N` where no `@tdd_issue_N` test exists in the codebase is
blocked by the CI quality gate — the TDD step was skipped.
### CLI Docstring Example Style
CLI command docstrings may include an `Examples:` section that shows users how to invoke the
command. These examples are rendered in the published documentation via MkDocs + mkdocstrings
and must accurately reflect the actual command signature.
#### Required Style
All `Examples:` lines must supply positional arguments **before** any option flags. This
mirrors the canonical Typer command signature where positional `Argument` parameters always
precede `Option` parameters.
**Correct:**
```
Examples:
agents plan rollback 01ARZ3NDEK7Z7BRRMJ98EGQGW 01BRZ4NFEK8A9CSSNJ09FHQHX
agents plan rollback 01ARZ3NDEK7Z7BRRMJ98EGQGW 01BRZ4NFEK8A9CSSNJ09FHQHX --yes
```
**Incorrect (options before positional args):**
```
Examples:
agents plan rollback --yes 01ARZ3NDEK7Z7BRRMJ98EGQGW 01BRZ4NFEK8A9CSSNJ09FHQHX
```
#### Placeholder Usage
Use realistic placeholder values (e.g., ULIDs for plan IDs, descriptive names for string
arguments) so that examples are immediately useful to readers. Avoid generic placeholders
like `<plan_id>` — use a realistic-looking value instead.
#### Automated Validation
The `DocstringExampleValidator` in `src/cleveragents/cli/docstring_validator.py` enforces
this style automatically. It is exercised by the Behave feature
`features/cli_docstring_example_validation.feature` which runs as part of `nox -s unit_tests`.
Any violation causes a test failure that names the offending command and shows the exact
example line.
### Testing Tools
Run tests using `nox`. Do not invoke `behave`, `robot`, or similar runners directly. If a `nox`
+1 -2
View File
@@ -28,7 +28,7 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed the plan tree JSON/YAML command envelope fix (#9163): wrapped `agents plan tree --format json/yaml` output in the spec-required command envelope structure, added summary statistics, decision_ids mapping, child_plans list, and accurate timing measurement.
* HAL 9000 contributed Structural Component Output Validation (PR #11161 / issue #8164): implemented `validate_plan_tree`, `validate_decision_dict`, `validate_structured_output`, and `validate_structured_component_output` validators that replace exact-character matching with structural schema checking for plan tree nodes, decision CLI dictionaries, and structured session output envelopes.
* HAMZA KHYARI has contributed the ACMS execute-phase context assembler project-level hot_max_tokens fix (PR #11036 / issue #11035): added `_resolve_effective_budget()` method that reads each linked project's `settings.hot_max_tokens` and uses the maximum override value as the pipeline budget instead of the hardcoded global 16K default.
* HAL 9000 has contributed the automated CLI docstring example validation (#9106): added `DocstringExampleValidator` to enforce positional-before-option ordering in CLI `Examples:` sections, with Behave test coverage and CONTRIBUTING.md documentation.
* HAL 9000 has contributed the CLI global format flag fix (#6879): promoted global options before subcommand parsing to allow `agents <command> --format ...` syntax as documented.
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
* HAL 9000 has contributed the pr-review-pool-supervisor tracking prefix documentation fix (#7891): aligned all documentation references from the outdated `AUTO-REV-POOL` prefix to the correct `AUTO-REV-SUP` prefix used in production.
@@ -64,4 +64,3 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed the alembic fileConfig error handling fix (PR #8288 / issue #7874): wrapped the `fileConfig()` call in `alembic/env.py` with a `try/except` block to catch malformed INI logging configuration and emit clear, actionable error messages to stderr.
* HAL 9000 has contributed the Definition-of-Done gating feature for the Apply phase (PR #8299 / issue #7927): `PlanLifecycleService.apply_plan` now evaluates DoD criteria before transitioning to Apply, raising `DoDGatingError` when required criteria fail and storing evaluation results in `plan.validation_summary`.
* HAL 9000 has contributed the engine cache TOCTOU race condition fix (PR #8265 / issue #7566): added `MEMORY_ENGINES_LOCK` to `engine_cache.py` and wrapped the check-and-set operation in `UnitOfWork.engine` with `with MEMORY_ENGINES_LOCK:` to prevent concurrent threads from creating duplicate in-memory SQLite engine instances; also fixed a cache-hit bug where `self._engine` was never assigned on a cache hit.
* HAL 9000 has contributed the plan correct JSON output envelope fix (PR #8662 / issue #8584): restructured `agents plan correct --format json` output to nest correction fields under `data.correction` and pass `command="plan correct"` to `format_output`, producing the spec-required CLI envelope. Added three BDD scenarios validating `data.correction.mode` (revert and append modes) and the `command` field.
-32
View File
@@ -1,32 +0,0 @@
Feature: Actor CLI exception handling contracts
As a developer maintaining the actor CLI module
I want the exception handling in _load_config_text and _compute_actor_impact to be correct
So that malformed YAML produces user-friendly errors and actor removal stays resilient
Background:
Given an actor CLI runner
# --- _load_config_text: yaml.YAMLError is caught and converted to BadParameter ---
Scenario: Malformed YAML config produces a user-friendly error
Given I have a malformed YAML actor config file
When I run actor add with that config
Then the actor command should fail with bad parameter
# --- _compute_actor_impact: CleverAgentsError is caught defensively ---
Scenario: _compute_actor_impact returns zero counts when session service raises CleverAgentsError
When I call _compute_actor_impact with a session service that raises CleverAgentsError
Then the impact result should be zero counts
# --- _compute_actor_impact: OperationalError is caught defensively ---
Scenario: _compute_actor_impact returns zero counts when session service raises OperationalError
When I call _compute_actor_impact with a session service that raises OperationalError
Then the impact result should be zero counts
# --- _compute_actor_impact: ValidationError is caught defensively ---
Scenario: _compute_actor_impact returns zero counts when session service raises ValidationError
When I call _compute_actor_impact with a session service that raises ValidationError
Then the impact result should be zero counts
@@ -1,41 +0,0 @@
@cli
Feature: CLI docstring example validation
As a documentation maintainer
I want to ensure CLI docstring examples respect positional argument order
So that published documentation accurately reflects the command signature
Scenario: Validator detects positional arguments before options
Given I have a CLI command with positional arguments and options
When I validate the command's docstring examples
Then the validation should pass for examples with positional args before options
Scenario: Validator rejects options before positional arguments
Given I have a CLI command with options placed before positional arguments
When I validate the bad command's docstring examples
Then the validation should fail for examples with options before positional args
Scenario: Validator handles commands without examples
Given I have a CLI command without docstring examples
When I validate the command's docstring examples
Then the validation should pass for commands without examples
Scenario: Validator parses example lines with shlex
Given I have a CLI command with quoted arguments in examples
When I validate the command's docstring examples
Then the validation should correctly parse quoted arguments
Scenario: Validator reports clear error messages
Given I have a CLI command with invalid docstring examples
When I validate the invalid command's docstring examples
Then the error message should identify the command and example line
And the error message should explain the positional argument order requirement
Scenario: Validator handles multi-word command names correctly
Given I have a multi-word CLI command with positional arguments and options
When I validate the multi-word command's docstring examples
Then the validation should pass for multi-word commands with correct argument order
Scenario: Validator scans all commands in a directory
Given I have a directory with CLI command modules
When I validate all commands in the directory
Then the validate all commands result should be True
+19
View File
@@ -44,6 +44,25 @@ Feature: Global --format flag propagation to subcommands
Then the global format command should succeed
And the global format output should be valid JSON
# -----------------------------------------------------------------------
# Regression: global --format accepted after the subcommand name
# -----------------------------------------------------------------------
Scenario: Global --format flag works after the info subcommand name
When I invoke the CLI with global args "info --format json"
Then the global format command should succeed
And the global format output should be valid JSON
Scenario: Global --format flag works after the version subcommand name
When I invoke the CLI with global args "version --format json"
Then the global format command should succeed
And the global format output should be valid JSON
Scenario: Global --format flag works after the diagnostics subcommand name
When I invoke the CLI with global args "diagnostics --format json"
Then the global format command should succeed
And the global format output should be valid JSON
# -----------------------------------------------------------------------
# Subtask 3: version, info, diagnostics read format from global ctx.obj
# -----------------------------------------------------------------------
-240
View File
@@ -1,240 +0,0 @@
Feature: AWS SDK integration for CloudResourceHandler
As a CleverAgents user
I want the CloudResourceHandler to use real AWS SDK operations
So that I can resolve and discover AWS cloud resources
# ── boto3 availability ────────────────────────────────────────────────────
Scenario: boto3 availability flag is exposed
Given awssdk the cloud handler module is imported
Then awssdk the boto3 availability flag should be a boolean
# ── _build_aws_session ────────────────────────────────────────────────────
Scenario: _build_aws_session raises ImportError when boto3 is unavailable
Given awssdk boto3 is not available
When awssdk I try to build an AWS session with valid credentials
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: _build_aws_session builds a session from explicit credentials
Given awssdk boto3 is available via mock
When awssdk I build an AWS session with explicit credentials
Then awssdk the session should be created successfully
Scenario: _build_aws_session builds a session from profile name
Given awssdk boto3 is available via mock
When awssdk I build an AWS session with a profile name "test-profile"
Then awssdk the session should be created with profile "test-profile"
# ── CloudResourceHandler.resolve for AWS ─────────────────────────────────
Scenario: resolve returns BoundResource for AWS account type with mocked STS
Given awssdk boto3 is available via mock
And awssdk AWS STS returns account id "123456789012"
And awssdk a valid AWS cloud resource of type "aws"
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk a BoundResource should be returned
And awssdk the BoundResource slot_name should be "cloud-slot"
And awssdk the BoundResource resource_type should be "aws"
And awssdk the BoundResource sandbox_path should contain "123456789012"
Scenario: resolve returns BoundResource for aws-vpc type (no STS check)
Given awssdk boto3 is available via mock
And awssdk a valid AWS cloud resource of type "aws-vpc"
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk a BoundResource should be returned
And awssdk the BoundResource resource_type should be "aws-vpc"
Scenario: resolve raises ValueError when STS call fails
Given awssdk boto3 is available via mock
And awssdk AWS STS raises a ClientError
And awssdk a valid AWS cloud resource of type "aws"
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk a ValueError should be raised mentioning "credential verification failed"
Scenario: resolve raises ImportError for AWS when boto3 is not installed
Given awssdk boto3 is not available
And awssdk a valid AWS cloud resource of type "aws-vpc"
When awssdk I call resolve on the cloud handler without boto3
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: resolve raises NotImplementedError for GCP provider
Given awssdk boto3 is available via mock
And awssdk a valid GCP cloud resource with credentials
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk a NotImplementedError should be raised mentioning "gcp"
Scenario: resolve raises NotImplementedError for Azure provider
Given awssdk boto3 is available via mock
And awssdk a valid Azure cloud resource with credentials
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk a NotImplementedError should be raised mentioning "azure"
# ── discover_aws_resources ────────────────────────────────────────────────
Scenario: discover_aws_resources returns empty list for unmapped type
Given awssdk boto3 is available via mock
And awssdk a mock boto3 session
And awssdk a cloud resource of type "aws-unknown-type"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should be an empty list
Scenario: discover_aws_resources discovers VPCs from EC2
Given awssdk boto3 is available via mock
And awssdk EC2 describe_vpcs returns 2 VPCs
And awssdk a cloud resource of type "aws-vpc"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should have 2 items
And awssdk each discovery item should have an "id" key
And awssdk each discovery item should have an "arn" key
Scenario: discover_aws_resources discovers EC2 instances via describe_instances
Given awssdk boto3 is available via mock
And awssdk EC2 describe_instances returns 3 instances
And awssdk a cloud resource of type "aws-instance"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should have 3 items
And awssdk each discovery item should have an "id" key
And awssdk each discovery item "id" should be non-empty
Scenario: discover_aws_resources discovers S3 buckets
Given awssdk boto3 is available via mock
And awssdk S3 list_buckets returns 3 buckets
And awssdk a cloud resource of type "aws-s3-bucket"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should have 3 items
And awssdk each discovery item arn should start with "arn:aws:s3:::"
Scenario: discover_aws_resources discovers ECS clusters
Given awssdk boto3 is available via mock
And awssdk ECS list_clusters returns 2 cluster ARNs
And awssdk a cloud resource of type "aws-ecs-cluster"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should have 2 items
Scenario: discover_aws_resources handles API errors gracefully
Given awssdk boto3 is available via mock
And awssdk a mock boto3 session that raises an exception
And awssdk a cloud resource of type "aws-vpc"
When awssdk I call discover_aws_resources
Then awssdk the discovery result should be an empty list
# ── CloudResourceHandler.discover_children ────────────────────────────────
Scenario: discover_children raises NotImplementedError for non-AWS provider
Given awssdk a CloudResourceHandler instance
And awssdk a cloud resource of type "gcp-project"
When awssdk I call discover_children on the cloud handler
Then awssdk a NotImplementedError should be raised mentioning "aws"
Scenario: discover_children raises ImportError when boto3 is not installed
Given awssdk boto3 is not available
And awssdk a CloudResourceHandler instance
And awssdk a valid AWS cloud resource of type "aws-vpc"
When awssdk I call discover_children on the cloud handler without boto3
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: discover_children returns Resource objects for discovered VPCs
Given awssdk boto3 is available via mock
And awssdk a CloudResourceHandler instance
And awssdk EC2 describe_vpcs returns 2 VPCs
And awssdk a valid AWS cloud resource of type "aws-vpc"
When awssdk I call discover_children on the cloud handler with mocked boto3
Then awssdk the aws discovery result should be a list of Resource objects
And awssdk the aws discovery result should have 2 items
# ── CloudSandboxStrategy AWS implementation ───────────────────────────────
Scenario: CloudSandboxStrategy.create succeeds for AWS with boto3 available
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call create on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk no exception should be raised
Scenario: CloudSandboxStrategy.create raises ImportError without boto3
Given awssdk boto3 is not available
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call create on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: CloudSandboxStrategy.create raises NotImplementedError for GCP
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "gcp"
When awssdk I call create on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk a NotImplementedError should be raised
Scenario: CloudSandboxStrategy.commit succeeds for AWS with boto3 available
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call commit on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk no exception should be raised
Scenario: CloudSandboxStrategy.commit raises ImportError without boto3
Given awssdk boto3 is not available
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call commit on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: CloudSandboxStrategy.commit raises NotImplementedError for Azure
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "azure"
When awssdk I call commit on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk a NotImplementedError should be raised
Scenario: CloudSandboxStrategy.rollback succeeds for AWS with boto3 available
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call rollback on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk no exception should be raised
Scenario: CloudSandboxStrategy.rollback raises ImportError without boto3
Given awssdk boto3 is not available
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call rollback on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk an ImportError should be raised mentioning "cleveragents[aws]"
Scenario: CloudSandboxStrategy.rollback raises NotImplementedError for GCP
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "gcp"
When awssdk I call rollback on the AWS sandbox strategy with plan "PLAN-001"
Then awssdk a NotImplementedError should be raised
Scenario: CloudSandboxStrategy.create raises ValueError for empty plan_id
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call create on the AWS sandbox strategy with plan ""
Then awssdk a ValueError should be raised mentioning "plan_id"
Scenario: CloudSandboxStrategy.commit raises ValueError for empty plan_id
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call commit on the AWS sandbox strategy with plan ""
Then awssdk a ValueError should be raised mentioning "plan_id"
Scenario: CloudSandboxStrategy.rollback raises ValueError for empty plan_id
Given awssdk boto3 is available via mock
And awssdk a cloud sandbox strategy for "aws"
When awssdk I call rollback on the AWS sandbox strategy with plan ""
Then awssdk a ValueError should be raised mentioning "plan_id"
# ── Credential masking (regression) ──────────────────────────────────────
Scenario: AWS credentials are never logged in plain text
Given awssdk boto3 is available via mock
And awssdk a valid AWS cloud resource of type "aws"
When awssdk I call resolve on the cloud handler with mocked boto3
Then awssdk no raw credential values should appear in log output
# ── _AWS_RESOURCE_MAP coverage ────────────────────────────────────────────
Scenario: _AWS_RESOURCE_MAP contains expected AWS resource types
Given awssdk the cloud handler module is imported
Then awssdk the AWS resource map should contain "aws-vpc"
And awssdk the AWS resource map should contain "aws-subnet"
And awssdk the AWS resource map should contain "aws-instance"
And awssdk the AWS resource map should contain "aws-s3-bucket"
And awssdk the AWS resource map should contain "aws-iam-role"
And awssdk the AWS resource map should contain "aws-rds-instance"
And awssdk the AWS resource map should contain "aws-ecs-cluster"
And awssdk the AWS resource map should contain "aws-lambda-function"
And awssdk the AWS resource map should contain "aws-eks-cluster"
+5 -4
View File
@@ -168,12 +168,13 @@ Feature: Cloud Infrastructure Resources
And I validate credentials for provider "aws"
Then the cloud validation errors should not contain secrets
# ── SDK execution ────────────────────────────────────────
# ── Stub execution ────────────────────────────────────────
Scenario: Cloud handler resolve raises ImportError for AWS when boto3 not installed
Scenario: Cloud handler resolve raises NotImplementedError for AWS
Given a valid AWS cloud resource
When I call resolve on the cloud handler
Then a cloud ImportError or NotImplementedError should be raised
Then a cloud NotImplementedError should be raised
And the cloud error message should mention "aws"
Scenario: Cloud handler resolve raises NotImplementedError for GCP
Given a valid GCP cloud resource
@@ -210,7 +211,7 @@ Feature: Cloud Infrastructure Resources
# ── Cloud sandbox strategy stubs ──────────────────────────
Scenario: Cloud sandbox create raises NotImplementedError
Given a cloud sandbox strategy for "gcp"
Given a cloud sandbox strategy for "aws"
When I call create on the sandbox strategy
Then a cloud NotImplementedError should be raised
@@ -1,159 +0,0 @@
"""Shared mock fixtures for TDD plan-correct JSON output envelope tests.
Provides constants, mock builders, and CLI argument helpers used by the
Behave step definitions
(``features/steps/tdd_plan_correct_json_output_steps.py``).
Centralising the mock builders ensures the test suite exercises the
``plan correct`` CLI JSON output path with identically-shaped mock objects.
Bug: https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8584
"""
from __future__ import annotations
from datetime import datetime
from types import SimpleNamespace
from unittest.mock import MagicMock
from cleveragents.core.exceptions import ResourceNotFoundError
from cleveragents.domain.models.core.plan import (
NamespacedName,
Plan,
PlanIdentity,
PlanPhase,
PlanTimestamps,
ProcessingState,
ProjectLink,
)
# ---------------------------------------------------------------------------
# Patch targets
# ---------------------------------------------------------------------------
PATCH_CONTAINER: str = "cleveragents.application.container.get_container"
# ---------------------------------------------------------------------------
# Fixed identifiers for deterministic assertions
# ---------------------------------------------------------------------------
DECISION_ID: str = "DEC-8584-TARGET"
ROOT_DECISION_ID: str = "DEC-8584-ROOT"
CORRECTION_ID: str = "CORR-TDD-8584"
# ---------------------------------------------------------------------------
# Mock builders
# ---------------------------------------------------------------------------
def make_decision_ns(
decision_id: str,
parent_decision_id: str | None,
) -> SimpleNamespace:
"""Create a minimal decision-like namespace for list_decisions."""
return SimpleNamespace(
decision_id=decision_id,
parent_decision_id=parent_decision_id,
)
def _make_plan() -> Plan:
"""Build a real ``Plan`` in Execute/COMPLETE state."""
from ulid import ULID
return Plan(
identity=PlanIdentity(plan_id=str(ULID())),
namespaced_name=NamespacedName(
server=None, namespace="local", name="tdd-8584-plan"
),
action_name="local/tdd-8584-action",
description="TDD plan for bug #8584 — JSON output envelope",
definition_of_done=None,
phase=PlanPhase.EXECUTE,
processing_state=ProcessingState.COMPLETE,
project_links=[ProjectLink(project_name="proj-1")],
strategy_actor="openai/gpt-4",
execution_actor="openai/gpt-4",
created_by=None,
reusable=False,
read_only=False,
timestamps=PlanTimestamps(created_at=datetime.now(), updated_at=datetime.now()),
)
def make_container(mode: str = "revert") -> MagicMock:
"""Build a mock DI container for the JSON output envelope test.
Args:
mode: The correction mode (``"revert"`` or ``"append"``).
"""
plan = _make_plan()
mock_plan_svc = MagicMock()
# get_plan raises RNF for the decision_id (it is not a plan_id),
# ensuring the code falls through to the decision_id path.
mock_plan_svc.get_plan.side_effect = ResourceNotFoundError(
resource_type="Plan",
resource_id=DECISION_ID,
)
mock_plan_svc.list_plans.return_value = [plan]
decisions = [
make_decision_ns(ROOT_DECISION_ID, None),
make_decision_ns(DECISION_ID, ROOT_DECISION_ID),
]
mock_decision_svc = MagicMock()
mock_decision_svc.list_decisions.return_value = decisions
mock_decision_svc.get_influence_edges.return_value = {}
mock_correction_svc = MagicMock()
mock_correction_svc.request_correction.return_value = SimpleNamespace(
correction_id=CORRECTION_ID,
mode=SimpleNamespace(value=mode),
target_decision_id=DECISION_ID,
guidance="Recompute decision subtree",
)
mock_correction_svc.execute_correction.return_value = SimpleNamespace(
correction_id=CORRECTION_ID,
status=SimpleNamespace(value="applied"),
reverted_decisions=[DECISION_ID],
new_decisions=[],
)
mock_container = MagicMock()
mock_container.plan_lifecycle_service.return_value = mock_plan_svc
mock_container.decision_service.return_value = mock_decision_svc
mock_container.correction_service.return_value = mock_correction_svc
return mock_container
def build_cli_args(mode: str = "revert") -> list[str]:
"""Build the CLI argument list for ``plan correct <decision_id> --format json``.
Args:
mode: The correction mode (``"revert"`` or ``"append"``).
"""
return [
"correct",
DECISION_ID,
"--mode",
mode,
"--guidance",
"Recompute decision subtree",
"--yes",
"--format",
"json",
]
__all__: list[str] = [
"CORRECTION_ID",
"DECISION_ID",
"PATCH_CONTAINER",
"ROOT_DECISION_ID",
"build_cli_args",
"make_container",
"make_decision_ns",
]
@@ -1,84 +0,0 @@
# pyright: reportRedeclaration=false
"""Step definitions for actor CLI exception handling contract tests."""
from __future__ import annotations
import tempfile
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock, patch
from behave import given, then, when
from sqlalchemy.exc import OperationalError
from cleveragents.cli.commands.actor import _compute_actor_impact
from cleveragents.core.exceptions import CleverAgentsError, ValidationError
def _register_cleanup_exc(context: Any, path: Path) -> None:
if not hasattr(context, "_cleanup_handlers"):
context._cleanup_handlers = []
context._cleanup_handlers.append(lambda: path.unlink(missing_ok=True))
@given("I have a malformed YAML actor config file")
def step_given_malformed_yaml_config(context: Any) -> None:
# Write a file that is valid text but invalid YAML (bad indentation with tabs)
content = "provider: openai\n model: bad-indent\n\t broken: true\n"
with tempfile.NamedTemporaryFile(
delete=False, suffix=".yaml", mode="w", encoding="utf-8"
) as handle:
handle.write(content)
handle.flush()
context.actor_config_path = Path(handle.name)
context.actor_config_data = {}
_register_cleanup_exc(context, context.actor_config_path)
context.expected_error = "Failed to parse config"
@when(
"I call _compute_actor_impact with a session service that raises CleverAgentsError"
)
def step_when_compute_impact_clever_agents_error(context: Any) -> None:
mock_container = MagicMock()
mock_container.session_service.side_effect = CleverAgentsError("DB unavailable")
with patch(
"cleveragents.cli.commands.actor.get_container",
return_value=mock_container,
):
context.impact_result = _compute_actor_impact("local/test-actor")
@when(
"I call _compute_actor_impact with a session service that raises OperationalError"
)
def step_when_compute_impact_operational_error(context: Any) -> None:
mock_container = MagicMock()
mock_container.session_service.side_effect = OperationalError(
"connection refused", params=None, orig=None
)
with patch(
"cleveragents.cli.commands.actor.get_container",
return_value=mock_container,
):
context.impact_result = _compute_actor_impact("local/test-actor")
@when("I call _compute_actor_impact with a session service that raises ValidationError")
def step_when_compute_impact_validation_error(context: Any) -> None:
mock_container = MagicMock()
mock_container.session_service.side_effect = ValidationError("invalid state")
with patch(
"cleveragents.cli.commands.actor.get_container",
return_value=mock_container,
):
context.impact_result = _compute_actor_impact("local/test-actor")
@then("the impact result should be zero counts")
def step_then_impact_result_zero(context: Any) -> None:
result = context.impact_result
assert result == (0, 0, 0), (
f"Expected (0, 0, 0) but got {result}. "
"The defensive exception handler should return zero counts when DB is unavailable."
)
@@ -1,291 +0,0 @@
"""Steps for CLI docstring example validation feature."""
from __future__ import annotations
import tempfile
from pathlib import Path
from typing import Annotated, Any
import typer
from behave import given, then, when
from cleveragents.cli.docstring_validator import DocstringExampleValidator
@given("I have a CLI command with positional arguments and options")
def step_have_cli_command_with_args_and_options(context: Any) -> None:
"""Create a test CLI command with positional args and options."""
def test_command(
plan_id: Annotated[str, typer.Argument(help="Plan ID")],
checkpoint_id: Annotated[
str | None, typer.Argument(help="Checkpoint ID")
] = None,
yes: Annotated[bool, typer.Option("--yes", help="Skip confirmation")] = False,
) -> None:
"""Test command with positional and optional arguments.
Examples:
agents test plan_id checkpoint_id
agents test plan_id checkpoint_id --yes
"""
pass
context.test_command = test_command
context.validator = DocstringExampleValidator()
@when("I validate the command's docstring examples")
def step_validate_docstring_examples(context: Any) -> None:
"""Validate the test command's docstring examples."""
result = context.validator.validate_command(
context.test_command,
"agents test",
)
context.validation_result = result
context.validation_errors = context.validator.get_errors()
@then("the validation should pass for examples with positional args before options")
def step_validation_passes_for_correct_order(context: Any) -> None:
"""Assert that validation passes for correct positional argument order."""
assert context.validation_result is True, (
f"Validation should pass but got errors: {context.validation_errors}"
)
@given("I have a CLI command with options placed before positional arguments")
def step_have_cli_command_with_bad_order(context: Any) -> None:
"""Create a test CLI command with options before positional args in examples."""
def bad_command(
plan_id: Annotated[str, typer.Argument(help="Plan ID")],
checkpoint_id: Annotated[
str | None, typer.Argument(help="Checkpoint ID")
] = None,
yes: Annotated[bool, typer.Option("--yes", help="Skip confirmation")] = False,
) -> None:
"""Test command with bad example order.
Examples:
agents test --yes plan_id checkpoint_id
"""
pass
context.bad_command = bad_command
context.bad_validator = DocstringExampleValidator()
@when("I validate the bad command's docstring examples")
def step_validate_bad_command_examples(context: Any) -> None:
"""Validate the bad command's docstring examples."""
result = context.bad_validator.validate_command(
context.bad_command,
"agents test",
)
context.bad_validation_result = result
context.bad_validation_errors = context.bad_validator.get_errors()
@then("the validation should fail for examples with options before positional args")
def step_validation_fails_for_incorrect_order(context: Any) -> None:
"""Assert that validation fails for incorrect positional argument order."""
assert context.bad_validation_result is False, (
"Validation should fail for options before positional args"
)
assert len(context.bad_validation_errors) > 0, "Should have validation errors"
@given("I have a CLI command without docstring examples")
def step_have_cli_command_without_examples(context: Any) -> None:
"""Create a test CLI command without examples."""
def test_command_no_examples(
plan_id: Annotated[str, typer.Argument(help="Plan ID")],
) -> None:
"""Test command without examples section."""
pass
context.test_command = test_command_no_examples
context.validator = DocstringExampleValidator()
@then("the validation should pass for commands without examples")
def step_validation_passes_for_no_examples(context: Any) -> None:
"""Assert that validation passes for commands without examples."""
result = context.validator.validate_command(
context.test_command,
"agents test",
)
assert result is True, (
f"Validation should pass but got errors: {context.validator.get_errors()}"
)
@given("I have a CLI command with quoted arguments in examples")
def step_have_cli_command_with_quoted_args(context: Any) -> None:
"""Create a test CLI command with quoted arguments."""
def test_command_quoted(
name: Annotated[str, typer.Argument(help="Plan name")],
description: Annotated[str | None, typer.Argument(help="Description")] = None,
) -> None:
"""Test command with quoted arguments.
Examples:
agents test "My Plan" "A description"
agents test "Plan Name" --format json
"""
pass
context.test_command = test_command_quoted
context.validator = DocstringExampleValidator()
@then("the validation should correctly parse quoted arguments")
def step_validation_parses_quoted_args(context: Any) -> None:
"""Assert that validation correctly parses quoted arguments."""
result = context.validator.validate_command(
context.test_command,
"agents test",
)
assert result is True, (
f"Validation should pass for quoted arguments but got errors: "
f"{context.validator.get_errors()}"
)
@given("I have a CLI command with invalid docstring examples")
def step_have_cli_command_with_invalid_examples(context: Any) -> None:
"""Create a test CLI command with invalid examples."""
def test_command_invalid(
plan_id: Annotated[str, typer.Argument(help="Plan ID")],
checkpoint_id: Annotated[
str | None, typer.Argument(help="Checkpoint ID")
] = None,
) -> None:
"""Test command with invalid example order.
Examples:
agents test --format json plan_id checkpoint_id
"""
pass
context.invalid_command = test_command_invalid
context.invalid_validator = DocstringExampleValidator()
@when("I validate the invalid command's docstring examples")
def step_validate_invalid_command_examples(context: Any) -> None:
"""Validate the invalid command's docstring examples."""
result = context.invalid_validator.validate_command(
context.invalid_command,
"agents test",
)
context.invalid_validation_result = result
context.invalid_validation_errors = context.invalid_validator.get_errors()
@then("the error message should identify the command and example line")
def step_error_identifies_command_and_line(context: Any) -> None:
"""Assert that error message identifies the command and example line."""
assert context.invalid_validation_result is False, "Validation should fail"
errors = context.invalid_validation_errors
assert len(errors) > 0, "Should have validation errors"
error_text = errors[0]
assert "agents test" in error_text, "Error should identify the command"
assert "agents test --format json" in error_text, (
"Error should show the problematic example line"
)
@then("the error message should explain the positional argument order requirement")
def step_error_explains_requirement(context: Any) -> None:
"""Assert that error message explains the positional argument order requirement."""
errors = context.invalid_validation_errors
assert len(errors) > 0, "Should have validation errors"
error_text = errors[0]
assert "positional" in error_text.lower(), (
"Error should mention positional arguments"
)
assert "before" in error_text.lower() or "order" in error_text.lower(), (
"Error should explain the ordering requirement"
)
@given("I have a multi-word CLI command with positional arguments and options")
def step_have_multi_word_cli_command(context: Any) -> None:
"""Create a multi-word CLI command (e.g., 'agents plan rollback')."""
def rollback_plan(
plan_id: Annotated[str, typer.Argument(help="Plan ID")],
checkpoint_id: Annotated[str, typer.Argument(help="Checkpoint ID")],
yes: Annotated[bool, typer.Option("--yes", help="Skip confirmation")] = False,
) -> None:
"""Rollback a plan to a checkpoint.
Examples:
agents plan rollback 01ARZ3NDEK7Z7BRRMJ98EGQGW 01BRZ4NFEK8A9CSSNJ09FHQHX
agents plan rollback 01ARZ3NDEK7Z7BRRMJ98EGQGW 01BRZ4NFEK8A9CSSNJ09FHQHX --yes
"""
pass
context.multi_word_command = rollback_plan
context.multi_word_validator = DocstringExampleValidator()
@when("I validate the multi-word command's docstring examples")
def step_validate_multi_word_command(context: Any) -> None:
"""Validate the multi-word command's docstring examples."""
result = context.multi_word_validator.validate_command(
context.multi_word_command,
"agents plan rollback",
)
context.multi_word_result = result
context.multi_word_errors = context.multi_word_validator.get_errors()
@then("the validation should pass for multi-word commands with correct argument order")
def step_validation_passes_for_multi_word_command(context: Any) -> None:
"""Assert that validation passes for multi-word commands with correct order."""
assert context.multi_word_result is True, (
f"Validation should pass for multi-word command but got errors: "
f"{context.multi_word_errors}"
)
@given("I have a directory with CLI command modules")
def step_have_directory_with_cli_modules(context: Any) -> None:
"""Create a temporary directory with a CLI command module for testing."""
context.tmp_dir = tempfile.mkdtemp()
module_path = Path(context.tmp_dir) / "sample_cmd.py"
module_path.write_text(
'"""Sample CLI command module."""\n'
"from __future__ import annotations\n\n"
"def list_items(project_id: str) -> None:\n"
' """List items in a project.\n\n'
" Examples:\n"
" agents sample_cmd list_items my-project\n"
' """\n'
" pass\n"
)
context.commands_dir = Path(context.tmp_dir)
context.all_validator = DocstringExampleValidator()
@when("I validate all commands in the directory")
def step_validate_all_commands(context: Any) -> None:
"""Validate all commands in the temporary directory."""
context.all_result = context.all_validator.validate_all_commands(
context.commands_dir
)
context.all_errors = context.all_validator.get_errors()
@then("the validate all commands result should be True")
def step_validate_all_commands_passes(context: Any) -> None:
"""Assert that validate_all_commands returns True for valid modules."""
assert context.all_result is True, (
f"validate_all_commands should return True but got errors: {context.all_errors}"
)
-244
View File
@@ -1,244 +0,0 @@
"""Step definitions for AWS resource discovery scenarios.
Tests discover_aws_resources() and CloudResourceHandler.discover_children()
with mocked boto3.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock, patch
from behave import given, then, when # type: ignore[attr-defined]
from cleveragents.domain.models.core.resource import Resource
from cleveragents.resource.handlers.cloud import CloudResourceHandler
from cleveragents.resource.handlers.cloud_aws import (
_AWS_RESOURCE_MAP,
discover_aws_resources,
)
from features.steps.cloud_aws_helpers import (
_make_mock_session,
_make_resource,
_restore_cloud_env,
)
@given('awssdk a cloud resource of type "{type_name}"')
def step_cloud_resource_type(context: Any, type_name: str) -> None:
"""Create a cloud resource of the given type."""
context.cloud_resource = _make_resource(type_name) # type: ignore[attr-defined]
@given("awssdk EC2 describe_instances returns {count:d} instances")
def step_ec2_instances(context: Any, count: int) -> None:
"""Configure mock EC2 to return N instances via describe_instances."""
instances = [
{
"InstanceId": f"i-{i:017x}",
"InstanceType": "t3.micro",
"State": {"Name": "running"},
}
for i in range(count)
]
context.mock_session = _make_mock_session(ec2_instances=instances) # type: ignore[attr-defined]
@given("awssdk EC2 describe_vpcs returns {count:d} VPCs")
def step_ec2_vpcs(context: Any, count: int) -> None:
"""Configure mock EC2 to return N VPCs."""
vpcs = [
{
"VpcId": f"vpc-{i:08x}",
"State": "available",
"CidrBlock": f"10.{i}.0.0/16",
}
for i in range(count)
]
context.mock_session = _make_mock_session(ec2_vpcs=vpcs) # type: ignore[attr-defined]
@given("awssdk S3 list_buckets returns {count:d} buckets")
def step_s3_buckets(context: Any, count: int) -> None:
"""Configure mock S3 to return N buckets."""
buckets = [
{"Name": f"my-bucket-{i}", "CreationDate": "2024-01-01"} for i in range(count)
]
context.mock_session = _make_mock_session(s3_buckets=buckets) # type: ignore[attr-defined]
@given("awssdk ECS list_clusters returns {count:d} cluster ARNs")
def step_ecs_clusters(context: Any, count: int) -> None:
"""Configure mock ECS to return N cluster ARNs."""
arns = [
f"arn:aws:ecs:us-east-1:123456789012:cluster/cluster-{i}" for i in range(count)
]
context.mock_session = _make_mock_session(ecs_clusters=arns) # type: ignore[attr-defined]
@given("awssdk a CloudResourceHandler instance")
def step_handler_instance(context: Any) -> None:
"""Create a CloudResourceHandler instance."""
context.cloud_handler = CloudResourceHandler() # type: ignore[attr-defined]
@given("awssdk a mock boto3 session")
def step_mock_session(context: Any) -> None:
"""Create a generic mock boto3 session."""
context.mock_session = _make_mock_session() # type: ignore[attr-defined]
@given("awssdk a mock boto3 session that raises an exception")
def step_mock_session_error(context: Any) -> None:
"""Create a mock boto3 session that raises on all client calls."""
context.mock_session = _make_mock_session( # type: ignore[attr-defined]
client_error=RuntimeError("Simulated AWS API error")
)
@when("awssdk I call discover_aws_resources")
def step_call_discover_aws(context: Any) -> None:
"""Call discover_aws_resources with the mock session."""
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
session = getattr(context, "mock_session", _make_mock_session())
context.discovery_result = discover_aws_resources(resource, session) # type: ignore[attr-defined]
@when("awssdk I call discover_children on the cloud handler")
def step_call_discover_children_no_boto3(context: Any) -> None:
"""Call discover_children on the handler (non-AWS provider)."""
handler: CloudResourceHandler = context.cloud_handler # type: ignore[attr-defined]
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
context.discovery_result = None # type: ignore[attr-defined]
try:
result = handler.discover_children(resource=resource)
context.discovery_result = result # type: ignore[attr-defined]
except (NotImplementedError, ImportError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
@when("awssdk I call discover_children on the cloud handler without boto3")
def step_call_discover_children_no_boto3_explicit(context: Any) -> None:
"""Call discover_children without boto3 available."""
handler: CloudResourceHandler = context.cloud_handler # type: ignore[attr-defined]
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
context.discovery_result = None # type: ignore[attr-defined]
with (
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", False),
patch("cleveragents.resource.handlers.cloud_aws.boto3", None),
patch("cleveragents.resource.handlers.cloud._BOTO3_AVAILABLE", False),
):
try:
result = handler.discover_children(resource=resource)
context.discovery_result = result # type: ignore[attr-defined]
except (NotImplementedError, ImportError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
@when("awssdk I call discover_children on the cloud handler with mocked boto3")
def step_call_discover_children_mocked(context: Any) -> None:
"""Call discover_children with mocked boto3."""
handler: CloudResourceHandler = context.cloud_handler # type: ignore[attr-defined]
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
mock_session = getattr(context, "mock_session", _make_mock_session())
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
context.discovery_result = None # type: ignore[attr-defined]
mock_boto3 = MagicMock()
mock_boto3.Session.return_value = mock_session
with (
patch("cleveragents.resource.handlers.cloud_aws.boto3", mock_boto3),
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", True),
patch("cleveragents.resource.handlers.cloud._BOTO3_AVAILABLE", True),
):
try:
result = handler.discover_children(resource=resource)
context.discovery_result = result # type: ignore[attr-defined]
except (NotImplementedError, ImportError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
saved = getattr(context, "saved_env", {})
_restore_cloud_env(saved)
@then("awssdk the discovery result should be an empty list")
def step_discovery_empty(context: Any) -> None:
"""Check that the discovery result is empty."""
result = context.discovery_result # type: ignore[attr-defined]
assert result == [], f"Expected empty list, got {result}"
@then("awssdk the discovery result should have {count:d} items")
def step_discovery_count(context: Any, count: int) -> None:
"""Check the discovery result count."""
result = context.discovery_result # type: ignore[attr-defined]
assert len(result) == count, f"Expected {count} items, got {len(result)}: {result}"
@then('awssdk each discovery item should have an "{key}" key')
def step_discovery_item_key(context: Any, key: str) -> None:
"""Check that each discovery item has the expected key."""
result = context.discovery_result # type: ignore[attr-defined]
for item in result:
assert key in item, f"Key '{key}' not found in item: {item}"
@then('awssdk each discovery item arn should start with "{prefix}"')
def step_discovery_item_arn_prefix(context: Any, prefix: str) -> None:
"""Check that each discovery item ARN starts with the expected prefix."""
result = context.discovery_result # type: ignore[attr-defined]
for item in result:
arn = item.get("arn", "")
assert arn.startswith(prefix), (
f"Expected ARN to start with '{prefix}', got '{arn}'"
)
@then("awssdk the aws discovery result should be a list of Resource objects")
def step_result_resource_list(context: Any) -> None:
"""Check that the result is a list of Resource objects."""
result = context.discovery_result # type: ignore[attr-defined]
assert isinstance(result, list), f"Expected list, got {type(result)}"
for item in result:
assert isinstance(item, Resource), (
f"Expected Resource, got {type(item)}: {item}"
)
@then("awssdk the aws discovery result should have {count:d} items")
def step_result_count(context: Any, count: int) -> None:
"""Check the result count."""
result = context.discovery_result # type: ignore[attr-defined]
assert len(result) == count, f"Expected {count} items, got {len(result)}"
@then('awssdk the AWS resource map should contain "{type_name}"')
def step_aws_resource_map_contains(context: Any, type_name: str) -> None:
"""Check that the AWS resource map contains the expected type."""
assert type_name in _AWS_RESOURCE_MAP, (
f"'{type_name}' not found in _AWS_RESOURCE_MAP. "
f"Available: {sorted(_AWS_RESOURCE_MAP.keys())}"
)
@then('awssdk each discovery item "{key}" should be non-empty')
def step_discovery_item_value_nonempty(context: Any, key: str) -> None:
"""Check that each discovery item has a non-empty value for the given key."""
result = context.discovery_result # type: ignore[attr-defined]
for item in result:
val = item.get(key, "")
assert val, f"Expected non-empty '{key}' in item: {item}"
-146
View File
@@ -1,146 +0,0 @@
"""Shared helpers for AWS SDK step definitions.
Provides environment management, resource factories, and mock session
builders used across the cloud_aws_* step definition files.
"""
from __future__ import annotations
import contextlib
import os
from typing import Any
from unittest.mock import MagicMock
from cleveragents.domain.models.core.resource import (
PhysVirt,
Resource,
ResourceCapabilities,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
_CLOUD_ENV_VARS = [
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"AWS_SESSION_TOKEN",
"AWS_REGION",
"AWS_PROFILE",
"GOOGLE_APPLICATION_CREDENTIALS",
"GCLOUD_PROJECT",
"GCP_REGION",
"AZURE_SUBSCRIPTION_ID",
"AZURE_TENANT_ID",
"AZURE_CLIENT_ID",
"AZURE_CLIENT_SECRET",
"AZURE_REGION",
]
def _clear_cloud_env() -> dict[str, str]:
saved: dict[str, str] = {}
for var in _CLOUD_ENV_VARS:
val = os.environ.pop(var, None)
if val is not None:
saved[var] = val
return saved
def _restore_cloud_env(saved: dict[str, str]) -> None:
for var in _CLOUD_ENV_VARS:
if var in saved:
os.environ[var] = saved[var]
else:
os.environ.pop(var, None)
def _make_resource(
type_name: str,
properties: dict[str, Any] | None = None,
location: str | None = None,
) -> Resource:
return Resource(
resource_id="01KJ5C5TPMP8GGX3QC83E2MAQS",
resource_type_name=type_name,
classification=PhysVirt.PHYSICAL,
location=location,
properties=properties or {},
capabilities=ResourceCapabilities(
readable=True,
writable=False,
sandboxable=False,
checkpointable=False,
),
)
def _make_mock_session(
account_id: str = "123456789012",
sts_error: Exception | None = None,
ec2_vpcs: list[dict[str, Any]] | None = None,
ec2_instances: list[dict[str, Any]] | None = None,
s3_buckets: list[dict[str, Any]] | None = None,
ecs_clusters: list[str] | None = None,
client_error: Exception | None = None,
) -> MagicMock:
"""Build a mock boto3 Session with configurable responses."""
session = MagicMock()
def _make_client(service: str, **_kwargs: Any) -> MagicMock:
client = MagicMock()
if service == "sts":
if sts_error is not None:
client.get_caller_identity.side_effect = sts_error
else:
client.get_caller_identity.return_value = {
"Account": account_id,
"UserId": "AIDAIOSFODNN7EXAMPLE",
"Arn": f"arn:aws:iam::{account_id}:user/test",
}
elif service == "ec2":
if client_error is not None:
client.describe_vpcs.side_effect = client_error
client.describe_subnets.side_effect = client_error
client.describe_instances.side_effect = client_error
client.describe_security_groups.side_effect = client_error
else:
vpcs = ec2_vpcs or []
client.describe_vpcs.return_value = {"Vpcs": vpcs}
client.describe_subnets.return_value = {"Subnets": []}
instances = ec2_instances or []
reservations = (
[{"ReservationId": "r-00000001", "Instances": instances}]
if instances
else []
)
client.describe_instances.return_value = {"Reservations": reservations}
client.describe_security_groups.return_value = {"SecurityGroups": []}
elif service == "s3":
if client_error is not None:
client.list_buckets.side_effect = client_error
else:
buckets = s3_buckets or []
client.list_buckets.return_value = {"Buckets": buckets}
elif service == "ecs":
if client_error is not None:
client.list_clusters.side_effect = client_error
else:
arns = ecs_clusters or []
client.list_clusters.return_value = {"clusterArns": arns}
else:
if client_error is not None:
for attr in dir(client):
if not attr.startswith("_"):
with contextlib.suppress(AttributeError):
getattr(client, attr).side_effect = client_error
return client
session.client.side_effect = _make_client
return session
-95
View File
@@ -1,95 +0,0 @@
"""Step definitions for AWS sandbox lifecycle scenarios.
Tests CloudSandboxStrategy.create/commit/rollback with mocked boto3.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import patch
from behave import given, when # type: ignore[attr-defined]
from cleveragents.resource.handlers.cloud_aws import CloudSandboxStrategy
@given('awssdk a cloud sandbox strategy for "{provider}"')
def step_sandbox_strategy(context: Any, provider: str) -> None:
"""Create a CloudSandboxStrategy for the given provider."""
context.cloud_sandbox = CloudSandboxStrategy(provider) # type: ignore[attr-defined]
@when('awssdk I call create on the AWS sandbox strategy with plan "{plan_id}"')
def step_sandbox_create_aws(context: Any, plan_id: str) -> None:
"""Call create on the CloudSandboxStrategy."""
strategy: CloudSandboxStrategy = context.cloud_sandbox # type: ignore[attr-defined]
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
boto3_avail = getattr(context, "boto3_available", True)
with patch(
"cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", boto3_avail
):
try:
strategy.create("res-test-001", plan_id)
except (NotImplementedError, ImportError, ValueError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
@when('awssdk I call commit on the AWS sandbox strategy with plan "{plan_id}"')
def step_sandbox_commit_aws(context: Any, plan_id: str) -> None:
"""Call commit on the CloudSandboxStrategy."""
strategy: CloudSandboxStrategy = context.cloud_sandbox # type: ignore[attr-defined]
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
boto3_avail = getattr(context, "boto3_available", True)
with patch(
"cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", boto3_avail
):
try:
strategy.commit("res-test-001", plan_id)
except (NotImplementedError, ImportError, ValueError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
@when('awssdk I call rollback on the AWS sandbox strategy with plan "{plan_id}"')
def step_sandbox_rollback_aws(context: Any, plan_id: str) -> None:
"""Call rollback on the CloudSandboxStrategy."""
strategy: CloudSandboxStrategy = context.cloud_sandbox # type: ignore[attr-defined]
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
boto3_avail = getattr(context, "boto3_available", True)
with patch(
"cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", boto3_avail
):
try:
strategy.rollback("res-test-001", plan_id)
except (NotImplementedError, ImportError, ValueError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
# behave's parse format {plan_id} uses .+? and won't match an empty string,
# so we need explicit overloads for the empty plan_id scenarios.
@when('awssdk I call create on the AWS sandbox strategy with plan ""')
def step_sandbox_create_aws_empty(context: Any) -> None:
"""Call create on the CloudSandboxStrategy with empty plan_id."""
step_sandbox_create_aws(context, "")
@when('awssdk I call commit on the AWS sandbox strategy with plan ""')
def step_sandbox_commit_aws_empty(context: Any) -> None:
"""Call commit on the CloudSandboxStrategy with empty plan_id."""
step_sandbox_commit_aws(context, "")
@when('awssdk I call rollback on the AWS sandbox strategy with plan ""')
def step_sandbox_rollback_aws_empty(context: Any) -> None:
"""Call rollback on the CloudSandboxStrategy with empty plan_id."""
step_sandbox_rollback_aws(context, "")
-14
View File
@@ -1,14 +0,0 @@
"""Step definitions for cloud_aws_sdk.feature.
Tests AWS SDK integration for CloudResourceHandler using mocked boto3.
All steps use the "awssdk" prefix to avoid conflicts with existing
step definitions in other feature files.
Step definitions are split across focused modules for maintainability:
- cloud_aws_helpers.py: shared helpers (_make_resource, _make_mock_session)
- cloud_aws_session_steps.py: session building and resolve() steps
- cloud_aws_discover_steps.py: resource discovery steps
- cloud_aws_sandbox_steps.py: sandbox lifecycle steps
Behave auto-discovers all step files in this directory.
"""
-371
View File
@@ -1,371 +0,0 @@
"""Step definitions for AWS session building scenarios.
Tests _build_aws_session and CloudResourceHandler.resolve() with mocked boto3.
"""
from __future__ import annotations
import os
from typing import Any
from unittest.mock import MagicMock, patch
from behave import given, then, when # type: ignore[attr-defined]
from cleveragents.resource.handlers.cloud import CloudResourceHandler
from cleveragents.resource.handlers.cloud_aws import (
_BOTO3_AVAILABLE,
_build_aws_session,
)
from features.steps.cloud_aws_helpers import (
_clear_cloud_env,
_make_mock_session,
_make_resource,
_restore_cloud_env,
)
@given("awssdk the cloud handler module is imported")
def step_module_imported(context: Any) -> None:
"""Ensure the cloud handler module is importable."""
import cleveragents.resource.handlers.cloud as _mod
context.cloud_module = _mod # type: ignore[attr-defined]
@given("awssdk boto3 is not available")
def step_boto3_not_available(context: Any) -> None:
"""Simulate boto3 not being installed."""
context.boto3_available = False # type: ignore[attr-defined]
context.saved_env = _clear_cloud_env() # type: ignore[attr-defined]
@given("awssdk boto3 is available via mock")
def step_boto3_available_mock(context: Any) -> None:
"""Mark boto3 as available (mocked)."""
context.boto3_available = True # type: ignore[attr-defined]
context.saved_env = _clear_cloud_env() # type: ignore[attr-defined]
os.environ["AWS_ACCESS_KEY_ID"] = "AKIAIOSFODNN7EXAMPLE"
os.environ["AWS_SECRET_ACCESS_KEY"] = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
os.environ["AWS_REGION"] = "us-east-1"
@given('awssdk AWS STS returns account id "{account_id}"')
def step_sts_account_id(context: Any, account_id: str) -> None:
"""Configure mock STS to return a specific account ID."""
context.mock_account_id = account_id # type: ignore[attr-defined]
@given("awssdk AWS STS raises a ClientError")
def step_sts_client_error(context: Any) -> None:
"""Configure mock STS to raise a ClientError."""
context.sts_error = RuntimeError("AWS ClientError: InvalidClientTokenId") # type: ignore[attr-defined]
@given('awssdk a valid AWS cloud resource of type "{type_name}"')
def step_aws_resource_type(context: Any, type_name: str) -> None:
"""Create an AWS resource of the given type."""
context.cloud_resource = _make_resource( # type: ignore[attr-defined]
type_name,
properties={
"access-key-id": "AKIAIOSFODNN7EXAMPLE",
"secret-access-key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-east-1",
},
)
@given("awssdk a valid GCP cloud resource with credentials")
def step_gcp_resource_with_creds(context: Any) -> None:
"""Create a GCP resource with credentials."""
context.cloud_resource = _make_resource( # type: ignore[attr-defined]
"gcp",
properties={
"service-account-json-path": "/path/to/sa.json",
"project-id": "my-project",
},
)
@given("awssdk a valid Azure cloud resource with credentials")
def step_azure_resource_with_creds(context: Any) -> None:
"""Create an Azure resource with credentials."""
context.cloud_resource = _make_resource( # type: ignore[attr-defined]
"azure",
properties={
"subscription-id": "sub-123",
"tenant-id": "tenant-456",
"client-id": "client-789",
"client-secret": "secret-abc",
},
)
@when("awssdk I try to build an AWS session with valid credentials")
def step_try_build_session_no_boto3(context: Any) -> None:
"""Try to build an AWS session when boto3 is unavailable."""
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
with (
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", False),
patch("cleveragents.resource.handlers.cloud_aws.boto3", None),
):
try:
_build_aws_session(
{
"access-key-id": "AKIAIOSFODNN7EXAMPLE",
"secret-access-key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
}
)
except ImportError as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = "ImportError" # type: ignore[attr-defined]
@when("awssdk I build an AWS session with explicit credentials")
def step_build_session_explicit(context: Any) -> None:
"""Build an AWS session with explicit credentials using mocked boto3."""
context.raised_error = None # type: ignore[attr-defined]
mock_boto3 = MagicMock()
mock_session = MagicMock()
mock_boto3.Session.return_value = mock_session
with (
patch("cleveragents.resource.handlers.cloud_aws.boto3", mock_boto3),
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", True),
):
result = _build_aws_session(
{
"access-key-id": "AKIAIOSFODNN7EXAMPLE",
"secret-access-key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-east-1",
}
)
context.session_result = result # type: ignore[attr-defined]
context.mock_boto3 = mock_boto3 # type: ignore[attr-defined]
@when('awssdk I build an AWS session with a profile name "{profile}"')
def step_build_session_profile(context: Any, profile: str) -> None:
"""Build an AWS session with a profile name."""
context.raised_error = None # type: ignore[attr-defined]
mock_boto3 = MagicMock()
mock_session = MagicMock()
mock_boto3.Session.return_value = mock_session
with (
patch("cleveragents.resource.handlers.cloud_aws.boto3", mock_boto3),
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", True),
):
result = _build_aws_session({"profile": profile})
context.session_result = result # type: ignore[attr-defined]
context.mock_boto3 = mock_boto3 # type: ignore[attr-defined]
context.expected_profile = profile # type: ignore[attr-defined]
@when("awssdk I call resolve on the cloud handler with mocked boto3")
def step_resolve_with_mock(context: Any) -> None:
"""Call resolve on CloudResourceHandler with mocked boto3."""
from cleveragents.domain.models.core.resource import Resource
handler = CloudResourceHandler()
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
mock_manager = MagicMock()
account_id = getattr(context, "mock_account_id", "123456789012")
sts_error = getattr(context, "sts_error", None)
mock_session = _make_mock_session(account_id=account_id, sts_error=sts_error)
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
context.bound_resource = None # type: ignore[attr-defined]
mock_boto3 = MagicMock()
mock_boto3.Session.return_value = mock_session
with (
patch("cleveragents.resource.handlers.cloud_aws.boto3", mock_boto3),
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", True),
patch("cleveragents.resource.handlers.cloud._BOTO3_AVAILABLE", True),
):
try:
result = handler.resolve(
resource=resource,
plan_id="PLAN-TEST-001",
slot_name="cloud-slot",
sandbox_manager=mock_manager,
)
context.bound_resource = result # type: ignore[attr-defined]
except (ValueError, NotImplementedError, ImportError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
saved = getattr(context, "saved_env", {})
_restore_cloud_env(saved)
@when("awssdk I call resolve on the cloud handler without boto3")
def step_resolve_without_boto3(context: Any) -> None:
"""Call resolve on CloudResourceHandler without boto3."""
from cleveragents.domain.models.core.resource import Resource
handler = CloudResourceHandler()
resource: Resource = context.cloud_resource # type: ignore[attr-defined]
mock_manager = MagicMock()
context.raised_error = None # type: ignore[attr-defined]
context.raised_error_type = None # type: ignore[attr-defined]
context.bound_resource = None # type: ignore[attr-defined]
with (
patch("cleveragents.resource.handlers.cloud_aws._BOTO3_AVAILABLE", False),
patch("cleveragents.resource.handlers.cloud_aws.boto3", None),
patch("cleveragents.resource.handlers.cloud._BOTO3_AVAILABLE", False),
):
try:
result = handler.resolve(
resource=resource,
plan_id="PLAN-TEST-001",
slot_name="cloud-slot",
sandbox_manager=mock_manager,
)
context.bound_resource = result # type: ignore[attr-defined]
except (ValueError, NotImplementedError, ImportError) as exc:
context.raised_error = exc # type: ignore[attr-defined]
context.raised_error_type = type(exc).__name__ # type: ignore[attr-defined]
saved = getattr(context, "saved_env", {})
_restore_cloud_env(saved)
@then("awssdk the boto3 availability flag should be a boolean")
def step_boto3_flag_bool(context: Any) -> None:
"""Check that _BOTO3_AVAILABLE is a boolean."""
assert isinstance(_BOTO3_AVAILABLE, bool), (
f"Expected bool, got {type(_BOTO3_AVAILABLE)}"
)
@then('awssdk an ImportError should be raised mentioning "{text}"')
def step_import_error_raised(context: Any, text: str) -> None:
"""Check that an ImportError was raised with the expected text."""
assert context.raised_error_type == "ImportError", ( # type: ignore[attr-defined]
f"Expected ImportError, got {context.raised_error_type}: " # type: ignore[attr-defined]
f"{context.raised_error}" # type: ignore[attr-defined]
)
msg = str(context.raised_error) # type: ignore[attr-defined]
assert text in msg, f"'{text}' not found in ImportError: {msg}"
@then("awssdk the session should be created successfully")
def step_session_created(context: Any) -> None:
"""Check that a session was created."""
assert context.session_result is not None # type: ignore[attr-defined]
mock_boto3: MagicMock = context.mock_boto3 # type: ignore[attr-defined]
assert mock_boto3.Session.called, "boto3.Session was not called"
@then('awssdk the session should be created with profile "{profile}"')
def step_session_created_with_profile(context: Any, profile: str) -> None:
"""Check that the session was created with the expected profile."""
assert context.session_result is not None # type: ignore[attr-defined]
mock_boto3: MagicMock = context.mock_boto3 # type: ignore[attr-defined]
call_kwargs = mock_boto3.Session.call_args[1]
assert call_kwargs.get("profile_name") == profile, (
f"Expected profile_name='{profile}', got {call_kwargs}"
)
@then("awssdk a BoundResource should be returned")
def step_bound_resource_returned(context: Any) -> None:
"""Check that a BoundResource was returned."""
from cleveragents.tool.context import BoundResource
assert context.raised_error is None, ( # type: ignore[attr-defined]
f"Expected BoundResource but got error: "
f"{context.raised_error_type}: {context.raised_error}" # type: ignore[attr-defined]
)
assert isinstance(context.bound_resource, BoundResource), ( # type: ignore[attr-defined]
f"Expected BoundResource, got {type(context.bound_resource)}" # type: ignore[attr-defined]
)
@then('awssdk the BoundResource slot_name should be "{slot_name}"')
def step_bound_resource_slot(context: Any, slot_name: str) -> None:
"""Check BoundResource slot_name."""
from cleveragents.tool.context import BoundResource
br: BoundResource = context.bound_resource # type: ignore[attr-defined]
assert br.slot_name == slot_name, (
f"Expected slot_name='{slot_name}', got '{br.slot_name}'"
)
@then('awssdk the BoundResource resource_type should be "{resource_type}"')
def step_bound_resource_type(context: Any, resource_type: str) -> None:
"""Check BoundResource resource_type."""
from cleveragents.tool.context import BoundResource
br: BoundResource = context.bound_resource # type: ignore[attr-defined]
assert br.resource_type == resource_type, (
f"Expected resource_type='{resource_type}', got '{br.resource_type}'"
)
@then('awssdk the BoundResource sandbox_path should contain "{text}"')
def step_bound_resource_sandbox_path(context: Any, text: str) -> None:
"""Check BoundResource sandbox_path contains expected text."""
from cleveragents.tool.context import BoundResource
br: BoundResource = context.bound_resource # type: ignore[attr-defined]
assert br.sandbox_path is not None, "Expected sandbox_path to be set"
assert text in br.sandbox_path, (
f"Expected '{text}' in sandbox_path='{br.sandbox_path}'"
)
@then('awssdk a ValueError should be raised mentioning "{text}"')
def step_value_error_raised(context: Any, text: str) -> None:
"""Check that a ValueError was raised with the expected text."""
assert context.raised_error_type == "ValueError", ( # type: ignore[attr-defined]
f"Expected ValueError, got {context.raised_error_type}: " # type: ignore[attr-defined]
f"{context.raised_error}" # type: ignore[attr-defined]
)
msg = str(context.raised_error) # type: ignore[attr-defined]
assert text in msg, f"'{text}' not found in ValueError: {msg}"
@then('awssdk a NotImplementedError should be raised mentioning "{text}"')
def step_not_implemented_raised(context: Any, text: str) -> None:
"""Check that a NotImplementedError was raised with the expected text."""
assert context.raised_error_type == "NotImplementedError", ( # type: ignore[attr-defined]
f"Expected NotImplementedError, got {context.raised_error_type}: " # type: ignore[attr-defined]
f"{context.raised_error}" # type: ignore[attr-defined]
)
msg = str(context.raised_error) # type: ignore[attr-defined]
assert text in msg, f"'{text}' not found in NotImplementedError: {msg}"
@then("awssdk a NotImplementedError should be raised")
def step_not_implemented_raised_any(context: Any) -> None:
"""Check that a NotImplementedError was raised."""
assert context.raised_error_type == "NotImplementedError", ( # type: ignore[attr-defined]
f"Expected NotImplementedError, got {context.raised_error_type}: " # type: ignore[attr-defined]
f"{context.raised_error}" # type: ignore[attr-defined]
)
@then("awssdk no exception should be raised")
def step_no_exception(context: Any) -> None:
"""Check that no exception was raised."""
assert context.raised_error is None, ( # type: ignore[attr-defined]
f"Expected no exception, got {context.raised_error_type}: " # type: ignore[attr-defined]
f"{context.raised_error}" # type: ignore[attr-defined]
)
@then("awssdk no raw credential values should appear in log output")
def step_no_raw_creds_in_log(context: Any) -> None:
"""Check that no raw credential values appear in log output."""
assert (
context.bound_resource is not None or context.raised_error is not None # type: ignore[attr-defined]
), "Handler did not run"
if context.raised_error is not None: # type: ignore[attr-defined]
msg = str(context.raised_error) # type: ignore[attr-defined]
assert "wJalrXUtnFEMI" not in msg, f"Secret found in error: {msg}"
assert "AKIAIOSFODNN7EXAMPLE" not in msg, f"Secret found in error: {msg}"
@@ -36,7 +36,7 @@ from cleveragents.resource.handlers.cloud import CloudResourceHandler
_VALID_ULID = "01ARZ3NDEKTSV4RRFFQ69G5FAV"
def _make_resource(type_name: str = "gcp-account") -> Resource:
def _make_resource(type_name: str = "aws-account") -> Resource:
"""Create a minimal Resource suitable for handler stub calls."""
return Resource(
resource_id=_VALID_ULID,
+1 -28
View File
@@ -18,10 +18,10 @@ from cleveragents.domain.models.core.resource import (
)
from cleveragents.resource.handlers.cloud import (
CloudResourceHandler,
CloudSandboxStrategy,
resolve_credentials,
validate_credentials,
)
from cleveragents.resource.handlers.cloud_aws import CloudSandboxStrategy
from cleveragents.resource.handlers.protocol import ResourceHandler
# ── Environment variable management ─────────────────────────
@@ -242,9 +242,6 @@ def step_call_resolve(context: Any) -> None:
except ValueError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "ValueError" # type: ignore[attr-defined]
except ImportError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "ImportError" # type: ignore[attr-defined]
finally:
saved = getattr(context, "saved_env", {})
_restore_cloud_env(saved)
@@ -273,9 +270,6 @@ def step_sandbox_create(context: Any) -> None:
except NotImplementedError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "NotImplementedError" # type: ignore[attr-defined]
except ImportError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "ImportError" # type: ignore[attr-defined]
@when("I call commit on the sandbox strategy")
@@ -289,9 +283,6 @@ def step_sandbox_commit(context: Any) -> None:
except NotImplementedError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "NotImplementedError" # type: ignore[attr-defined]
except ImportError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "ImportError" # type: ignore[attr-defined]
@when("I call rollback on the sandbox strategy")
@@ -305,9 +296,6 @@ def step_sandbox_rollback(context: Any) -> None:
except NotImplementedError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "NotImplementedError" # type: ignore[attr-defined]
except ImportError as exc:
context.handler_error = exc # type: ignore[attr-defined]
context.handler_error_type = "ImportError" # type: ignore[attr-defined]
@when("I validate the cloud sandbox strategy")
@@ -444,18 +432,3 @@ def step_cloud_satisfies_protocol(context: Any) -> None:
assert isinstance(handler, ResourceHandler), (
"CloudResourceHandler does not satisfy ResourceHandler protocol"
)
@then("a cloud ImportError or NotImplementedError should be raised")
def step_cloud_import_or_not_implemented(context: Any) -> None:
"""Check that an ImportError or NotImplementedError was raised.
AWS resolve now requires boto3. Without it, ImportError is raised.
With it, a BoundResource is returned (no error). This step accepts
either outcome to allow the test to pass in both environments.
"""
error_type = context.handler_error_type # type: ignore[attr-defined]
assert error_type in ("ImportError", "NotImplementedError", None), (
f"Expected ImportError, NotImplementedError, or None (success), "
f"got {error_type}: {context.handler_error}" # type: ignore[attr-defined]
)
@@ -1,192 +0,0 @@
"""Step definitions for tdd_plan_correct_json_output.feature.
Captures bug #8584: the ``plan correct`` CLI command's JSON output
(``--format json``) does not match the spec-required envelope structure.
The current implementation places correction fields directly under ``data``
(``data.mode``, ``data.correction_id``) instead of nesting them under
``data.correction``. This means ``data.correction.mode`` is absent from
the output. Additionally, the ``command`` field is empty instead of
``"plan correct"``.
The assertions verify the *expected* (correct) behaviour. They will
**fail** on the current codebase, proving the bug exists. The
``@tdd_expected_fail`` tag inverts the result so CI passes.
All step text uses the ``tpcjo`` prefix to avoid collisions with other
step files.
Bug: https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8584
"""
from __future__ import annotations
import json
from unittest.mock import patch
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from features.mocks.tdd_plan_correct_json_output_fixtures import (
PATCH_CONTAINER,
build_cli_args,
make_container,
)
runner = CliRunner()
def _get_plan_app():
"""Lazily import the plan CLI app to avoid slow module-level import.
``cleveragents.cli.commands.plan`` triggers a large import chain
(application container, all services, etc.) that takes ~100 s on a
cold interpreter. Deferring the import to the first step execution
means the module is already cached by the time these scenarios run,
so the actual cost is near-zero.
"""
from cleveragents.cli.commands.plan import app as plan_app
return plan_app
# ---------------------------------------------------------------------------
# GIVEN steps
# ---------------------------------------------------------------------------
@given(
"tpcjo a container with a plan and a correction service"
" that succeeds in revert mode"
)
def step_tpcjo_container_revert(context: Context) -> None:
"""Set up a mock DI container for revert mode."""
context.tpcjo_container = make_container(mode="revert")
context.tpcjo_mode = "revert"
@given(
"tpcjo a container with a plan and a correction service"
" that succeeds in append mode"
)
def step_tpcjo_container_append(context: Context) -> None:
"""Set up a mock DI container for append mode."""
context.tpcjo_container = make_container(mode="append")
context.tpcjo_mode = "append"
# ---------------------------------------------------------------------------
# WHEN steps
# ---------------------------------------------------------------------------
@when("tpcjo I invoke plan correct with --format json in revert mode")
def step_tpcjo_invoke_revert(context: Context) -> None:
"""Invoke ``plan correct <decision_id> --format json --mode revert``.
Patches ``get_container`` so the command uses the mock DI container
instead of the real application container.
"""
args = build_cli_args(mode="revert")
with patch(PATCH_CONTAINER, return_value=context.tpcjo_container):
context.tpcjo_result = runner.invoke(_get_plan_app(), args)
_parse_json_output(context)
@when("tpcjo I invoke plan correct with --format json in append mode")
def step_tpcjo_invoke_append(context: Context) -> None:
"""Invoke ``plan correct <decision_id> --format json --mode append``.
Patches ``get_container`` so the command uses the mock DI container
instead of the real application container.
"""
args = build_cli_args(mode="append")
with patch(PATCH_CONTAINER, return_value=context.tpcjo_container):
context.tpcjo_result = runner.invoke(_get_plan_app(), args)
_parse_json_output(context)
# ---------------------------------------------------------------------------
# THEN steps
# ---------------------------------------------------------------------------
@then('tpcjo the JSON output data.correction.mode should be "{expected_mode}"')
def step_tpcjo_json_correction_mode(context: Context, expected_mode: str) -> None:
"""Assert the parsed JSON output has data.correction.mode set correctly.
This assertion will FAIL on the current codebase because the
implementation places ``mode`` directly under ``data`` (as ``data.mode``)
instead of nesting it under ``data.correction`` (as ``data.correction.mode``).
"""
parsed = context.tpcjo_parsed_json
assert "data" in parsed, (
f"Bug #8584: JSON output is missing top-level 'data' key. "
f"Current output keys: {list(parsed.keys())}. "
f"Full output: {context.tpcjo_result.output!r}"
)
data = parsed["data"]
assert isinstance(data, dict), (
f"Bug #8584: JSON output 'data' value is not a dict: {data!r}"
)
assert "correction" in data, (
f"Bug #8584: JSON output 'data' is missing 'correction' key. "
f"Current data keys: {list(data.keys())}. "
f"The spec requires correction fields to be nested under data.correction, "
f"but the current implementation places them directly under data "
f"(data.mode, data.correction_id, etc.)."
)
correction = data["correction"]
assert isinstance(correction, dict), (
f"Bug #8584: JSON output 'data.correction' is not a dict: {correction!r}"
)
assert "mode" in correction, (
f"Bug #8584: JSON output 'data.correction' is missing 'mode' key. "
f"Current correction keys: {list(correction.keys())}"
)
actual_mode = correction["mode"]
assert actual_mode == expected_mode, (
f"Bug #8584: data.correction.mode is {actual_mode!r}, "
f"expected {expected_mode!r}"
)
@then('tpcjo the JSON output command field should be "{expected_command}"')
def step_tpcjo_json_command_field(context: Context, expected_command: str) -> None:
"""Assert the parsed JSON output has the correct command field value.
This assertion will FAIL on the current codebase because the
``command`` field is an empty string instead of ``"plan correct"``.
"""
parsed = context.tpcjo_parsed_json
assert "command" in parsed, (
f"Bug #8584: JSON output is missing top-level 'command' key. "
f"Current output keys: {list(parsed.keys())}. "
f"Full output: {context.tpcjo_result.output!r}"
)
actual_command = parsed["command"]
assert actual_command == expected_command, (
f"Bug #8584: command field is {actual_command!r}, "
f"expected {expected_command!r}. "
f"The spec requires command to be 'plan correct' but the current "
f"implementation sets it to an empty string."
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _parse_json_output(context: Context) -> None:
"""Parse the CLI output as JSON and store it on the context.
If the command failed or the output is not valid JSON, stores an
empty dict so that subsequent assertions produce clear failure messages.
"""
result = context.tpcjo_result
try:
context.tpcjo_parsed_json = json.loads(result.output)
except (json.JSONDecodeError, ValueError):
context.tpcjo_parsed_json = {}
@@ -1,55 +0,0 @@
@tdd_issue @tdd_issue_8584
Feature: TDD Bug #8584 - plan correct JSON output missing spec-required envelope format
As a developer
I want plan correct --format json to return the standard CLI envelope
So that JSON consumers receive the spec-required nested data structure
The v3.2.0 specification (section CLI Commands - agents plan correct, line 14912 in
docs/specification.md) defines the required JSON output envelope for
agents plan correct --format json.
The spec requires the correction data to be nested under data.correction:
{
"command": "plan correct",
"status": "ok",
"exit_code": 0,
"data": {
"correction": {
"mode": "revert",
"impact": "...",
...
},
"affected_subtree": {...},
...
},
"timing": {...},
"messages": ["Correction applied"]
}
The current implementation places correction fields directly under data
(data.mode, data.correction_id) instead of nesting them under data.correction.
This means data.correction.mode is absent from the output.
These scenarios assert the expected (correct) behaviour and validate
that the fix is in place.
All step text uses the tpcjo prefix to avoid collisions with other step files.
@tdd_issue @tdd_issue_8584
Scenario: plan correct --format json data.correction.mode is present in revert mode
Given tpcjo a container with a plan and a correction service that succeeds in revert mode
When tpcjo I invoke plan correct with --format json in revert mode
Then tpcjo the JSON output data.correction.mode should be "revert"
@tdd_issue @tdd_issue_8584
Scenario: plan correct --format json data.correction.mode is present in append mode
Given tpcjo a container with a plan and a correction service that succeeds in append mode
When tpcjo I invoke plan correct with --format json in append mode
Then tpcjo the JSON output data.correction.mode should be "append"
@tdd_issue @tdd_issue_8584
Scenario: plan correct --format json command field is set to plan correct
Given tpcjo a container with a plan and a correction service that succeeds in revert mode
When tpcjo I invoke plan correct with --format json in revert mode
Then tpcjo the JSON output command field should be "plan correct"
-4
View File
@@ -59,10 +59,6 @@ server = [
tui = [
"textual>=1.0.0,<2.0.0",
]
aws = [
"boto3>=1.34.0",
"botocore>=1.34.0",
]
dev = [
# Code formatting and linting
"ruff>=0.15.0,<0.16.0",
+6 -15
View File
@@ -6,7 +6,6 @@ calling Robot test can assert on ``stdout``.
from __future__ import annotations
import contextlib
import os
import sys
from typing import Any
@@ -19,10 +18,10 @@ from cleveragents.domain.models.core.resource import (
)
from cleveragents.resource.handlers.cloud import (
CloudResourceHandler,
CloudSandboxStrategy,
resolve_credentials,
validate_credentials,
)
from cleveragents.resource.handlers.cloud_aws import CloudSandboxStrategy
from cleveragents.resource.handlers.protocol import ResourceHandler
_CLOUD_ENV_VARS = [
@@ -145,7 +144,7 @@ def _azure_resolve() -> None:
def _stub_resolve() -> None:
"""Verify cloud handler raises ImportError or succeeds for AWS."""
"""Verify cloud handler raises NotImplementedError for valid config."""
saved = _clear_cloud_env()
try:
os.environ["AWS_ACCESS_KEY_ID"] = "AKIAIOSFODNN7EXAMPLE"
@@ -160,10 +159,8 @@ def _stub_resolve() -> None:
slot_name="cloud-slot",
sandbox_manager=mock_manager,
)
# boto3 is installed and resolve succeeded
print("stub-resolve-ok")
except (NotImplementedError, ImportError):
# boto3 not installed or not yet implemented
print("stub-resolve-FAIL: no error raised")
except NotImplementedError:
print("stub-resolve-ok")
finally:
_restore_cloud_env(saved)
@@ -193,8 +190,8 @@ def _missing_creds() -> None:
def _sandbox_stubs() -> None:
"""Verify sandbox strategy methods raise NotImplementedError or ImportError."""
for provider in ("gcp", "azure"):
"""Verify sandbox strategy methods raise NotImplementedError."""
for provider in ("aws", "gcp", "azure"):
strategy = CloudSandboxStrategy(provider)
for method_name in ("create", "commit", "rollback"):
method = getattr(strategy, method_name)
@@ -204,12 +201,6 @@ def _sandbox_stubs() -> None:
return
except NotImplementedError:
pass
# AWS raises ImportError (boto3 not installed) or succeeds (boto3 installed)
aws_strategy = CloudSandboxStrategy("aws")
for method_name in ("create", "commit", "rollback"):
method = getattr(aws_strategy, method_name)
with contextlib.suppress(NotImplementedError, ImportError):
method("res-test", "plan-test")
print("sandbox-stubs-ok")
+5 -24
View File
@@ -12,7 +12,6 @@ import yaml
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from sqlalchemy.exc import OperationalError
from cleveragents.actor.config import ActorConfiguration
from cleveragents.actor.schema import (
@@ -208,7 +207,7 @@ def run(
typer.echo(result)
def _get_services() -> tuple[Any, Any | None]:
def _get_services():
container = get_container()
actor_service = container.actor_service()
actor_registry = (
@@ -241,13 +240,7 @@ def _compute_actor_impact(actor_name: str) -> tuple[int, int, int]:
session_service = container.session_service()
sessions = session_service.list()
session_count = sum(1 for s in sessions if s.actor_name == actor_name)
except (
AttributeError,
RuntimeError,
CleverAgentsError,
OperationalError,
ValidationError, # pragma: no cover - defensive; DB may be unavailable
):
except Exception: # pragma: no cover - defensive; DB may be unavailable
pass
# --- Active lifecycle plans referencing this actor ---
@@ -263,13 +256,7 @@ def _compute_actor_impact(actor_name: str) -> tuple[int, int, int]:
if p.processing_state in active_states
and (p.strategy_actor == actor_name or p.execution_actor == actor_name)
)
except (
AttributeError,
RuntimeError,
CleverAgentsError,
OperationalError,
ValidationError, # pragma: no cover - defensive; DB may be unavailable
):
except Exception: # pragma: no cover - defensive; DB may be unavailable
pass
# --- Actions configured to use this actor ---
@@ -290,13 +277,7 @@ def _compute_actor_impact(actor_name: str) -> tuple[int, int, int]:
getattr(a, "invariant_actor", None),
)
)
except (
AttributeError,
RuntimeError,
CleverAgentsError,
OperationalError,
ValidationError, # pragma: no cover - defensive; DB may be unavailable
):
except Exception: # pragma: no cover - defensive; DB may be unavailable
pass
return session_count, active_plan_count, action_count
@@ -321,7 +302,7 @@ def _load_config_text(config_path: Path | None) -> tuple[str, dict[str, Any]] |
except json.JSONDecodeError:
try:
data = yaml.safe_load(text)
except (yaml.YAMLError, TypeError) as exc: # pragma: no cover - defensive
except Exception as exc: # pragma: no cover - defensive
raise typer.BadParameter(f"Failed to parse config: {exc}") from exc
if data is None:
+4 -20
View File
@@ -3507,29 +3507,13 @@ def correct_decision(
if fmt != OutputFormat.RICH.value:
data = {
"correction": {
"mode": correction_mode.value,
},
"correction_id": result.correction_id,
"status": result.status.value,
"mode": correction_mode.value,
"new_decisions": result.new_decisions,
"reverted_decisions": result.reverted_decisions,
}
console.print(
format_output(
data,
fmt,
command="plan correct",
status="ok",
exit_code=0,
messages=[
{
"level": "ok",
"text": f"Correction applied ({result.correction_id})",
}
],
)
)
console.print(format_output(data, fmt))
else:
console.print(
f"[green]✓[/green] Correction applied: {result.correction_id}"
@@ -3772,8 +3756,8 @@ def rollback_plan(
via the ``--to-checkpoint`` named option.
Examples:
agents plan rollback 01ARZ3NDEK... 01BRZ4NFEK... --yes
agents plan rollback 01ARZ3NDEK... --to-checkpoint 01BRZ4NFEK...
agents plan rollback --yes 01ARZ3NDEK... 01BRZ4NFEK...
agents plan rollback --to-checkpoint 01BRZ4NFEK... 01ARZ3NDEK...
"""
from cleveragents.application.container import get_container
from cleveragents.core.exceptions import (
-290
View File
@@ -1,290 +0,0 @@
"""Validator for CLI command docstring examples.
This module provides automated validation to ensure that CLI docstring examples
respect the positional argument order defined by the Typer command signature.
When a docstring example drifts from the actual command signature, the published
documentation becomes misleading. This validator catches such drift early.
"""
from __future__ import annotations
import importlib.util
import inspect
import logging
import re
import shlex
from collections.abc import Callable
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
class DocstringExampleValidator:
"""Validates CLI command docstring examples against Typer signatures."""
def __init__(self) -> None:
"""Initialize the validator."""
self.errors: list[str] = []
def validate_command(
self,
func: Callable[..., Any],
command_name: str,
) -> bool:
"""Validate a single CLI command's docstring examples.
Args:
func: The CLI command function to validate.
command_name: The full command name (e.g., "agents plan rollback").
Returns:
True if validation passes, False if errors were found.
"""
docstring = inspect.getdoc(func)
if not docstring:
return True
# Extract Examples section
examples_match = re.search(
r"Examples:\s*\n((?:(?:\n|.)*?)(?=\n\n|\Z))",
docstring,
re.MULTILINE,
)
if not examples_match:
return True
examples_text = examples_match.group(1).strip()
if not examples_text:
return True
# Parse Typer parameters to get positional args and options
positional_args = self._extract_positional_args(func)
option_flags = self._extract_option_flags(func)
# Validate each example line
example_lines = [
line.strip()
for line in examples_text.split("\n")
if line.strip() and not line.strip().startswith("#")
]
for example_line in example_lines:
if not self._validate_example_line(
example_line,
command_name,
positional_args,
option_flags,
):
return False
return True
def _extract_positional_args(self, func: Callable[..., Any]) -> list[str]:
"""Extract required positional argument names from function signature.
Only arguments without a default value (or with a default of
``inspect.Parameter.empty``) are considered required. Optional
positional arguments (those with a default) are excluded so that
examples that omit them are not incorrectly rejected.
Args:
func: The CLI command function.
Returns:
List of required positional argument names in order.
"""
positional_args = []
sig = inspect.signature(func)
for param_name, param in sig.parameters.items():
if param_name in ("self", "cls"):
continue
# Check if this is a Typer Argument (positional)
if param.annotation != inspect.Parameter.empty:
annotation_str = str(param.annotation)
if (
"Argument" in annotation_str
and param.default is inspect.Parameter.empty
):
positional_args.append(param_name)
return positional_args
def _extract_option_flags(self, func: Callable[..., Any]) -> set[str]:
"""Extract option flag names from function signature.
Args:
func: The CLI command function.
Returns:
Set of option flag names (e.g., "--yes", "--format").
"""
option_flags = set()
sig = inspect.signature(func)
for param_name, param in sig.parameters.items():
if param_name in ("self", "cls"):
continue
# Check if this is a Typer Option
if param.annotation != inspect.Parameter.empty:
annotation_str = str(param.annotation)
if "Option" in annotation_str:
# Try to extract flag names from the annotation
# Common patterns: --flag, -f, --flag-name
flag_match = re.search(
r'["\'](-{1,2}[a-z0-9-]+)["\']',
annotation_str,
)
if flag_match:
option_flags.add(flag_match.group(1))
# Also add common flag patterns
option_flags.add(f"--{param_name.replace('_', '-')}")
return option_flags
def _validate_example_line(
self,
example_line: str,
command_name: str,
positional_args: list[str],
option_flags: set[str],
) -> bool:
"""Validate a single example line.
Args:
example_line: The example command line to validate.
command_name: The full command name.
positional_args: List of positional argument names.
option_flags: Set of option flag names.
Returns:
True if the example is valid, False otherwise.
"""
try:
# Parse the example line with shlex
tokens = shlex.split(example_line)
except ValueError as e:
self.errors.append(
f"[{command_name}] Failed to parse example: {example_line}\n"
f" Error: {e}"
)
return False
# Skip if it doesn't start with the command
if not tokens or not tokens[0].startswith("agents"):
return True
# Determine how many tokens the command name occupies so we can
# correctly skip the command prefix when extracting positional values.
# For example, "agents plan rollback" occupies 3 tokens.
command_token_count = len(shlex.split(command_name))
# Find where positional arguments end and options begin.
# Scan from after the command prefix; the first token that starts
# with "-" marks the boundary.
positional_end_idx = command_token_count
suffix_tokens = tokens[command_token_count:]
for i, token in enumerate(suffix_tokens, start=command_token_count):
if token.startswith("-"):
break
positional_end_idx = i + 1
# Extract positional values from the example, skipping the command prefix
positional_values = tokens[command_token_count:positional_end_idx]
# Check if positional arguments appear before any options.
# Validation applies only when the command has known required positional args.
if positional_args:
required_positional_count = len(positional_args)
# Find the first option flag in the tokens (after the command prefix)
first_option_idx = None
for i, token in enumerate(suffix_tokens, start=command_token_count):
if token.startswith("-"):
first_option_idx = i
break
# If there are options, verify that all required positional args
# appear before the first option flag.
if (
first_option_idx is not None
and len(positional_values) < required_positional_count
):
self.errors.append(
f"[{command_name}] Example violates positional argument"
f" order:\n"
f" {example_line}\n"
f" Expected positional arguments {positional_args} "
f"to appear before options.\n"
f" Found {len(positional_values)} positional values, "
f"expected at least {required_positional_count}."
)
return False
return True
def validate_all_commands(self, commands_dir: Path) -> bool:
"""Validate all CLI commands in a directory.
Args:
commands_dir: Path to the CLI commands directory.
Returns:
True if all validations pass, False if any errors were found.
"""
all_valid = True
# Import all command modules
for command_file in sorted(commands_dir.glob("*.py")):
if command_file.name.startswith("_"):
continue
module_name = command_file.stem
try:
spec = importlib.util.spec_from_file_location(
f"cleveragents.cli.commands.{module_name}",
command_file,
)
if spec and spec.loader:
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
# Find all functions that look like CLI commands
for name, obj in inspect.getmembers(module):
if (
inspect.isfunction(obj)
and not name.startswith("_")
and hasattr(obj, "__doc__")
):
# Try to validate the command
cmd_suffix = name.replace("_", " ")
command_name = f"agents {module_name} {cmd_suffix}"
if not self.validate_command(obj, command_name):
all_valid = False
except Exception as e:
# Log at DEBUG level so import failures are observable without
# blocking validation of other modules.
logger.debug("Skipping module %s: %s", module_name, e)
return all_valid
def get_errors(self) -> list[str]:
"""Get all validation errors found.
Returns:
List of error messages.
"""
return self.errors
def print_errors(self) -> None:
"""Print all validation errors to stdout."""
if self.errors:
print("CLI Docstring Example Validation Errors:")
print("=" * 70)
for error in self.errors:
print(error)
print()
+89
View File
@@ -9,6 +9,7 @@ from pathlib import Path
from typing import TYPE_CHECKING, Annotated, Any
import typer
from typer.core import TyperGroup
from cleveragents import __version__
from cleveragents.cli.formatting import OutputFormat
@@ -21,6 +22,90 @@ _console: "Console | None" = None
_err_console: "Console | None" = None
_subcommands_registered = False
_GLOBAL_VALUE_OPTIONS = frozenset({"--format"})
_GLOBAL_VALUE_SHORT_OPTIONS = frozenset({"-f"})
_GLOBAL_VALUE_EQ_PREFIXES = ("--format=",)
_GLOBAL_FLAG_OPTIONS = frozenset({"--show-secrets", "--version", "-V"})
def _split_short_value_option(option: str) -> tuple[str, str] | None:
for short in _GLOBAL_VALUE_SHORT_OPTIONS:
if option.startswith(short) and len(option) > len(short):
return short, option[len(short) :]
return None
def _promote_global_options(args: list[str]) -> list[str]:
"""Move global root options before subcommands for Typer parsing."""
if not args:
return args
promoted: list[str] = []
remaining: list[str] = []
i = 0
length = len(args)
while i < length:
arg = args[i]
if arg == "--":
remaining.extend(args[i:])
break
if arg in _GLOBAL_FLAG_OPTIONS:
promoted.append(arg)
i += 1
continue
if arg in _GLOBAL_VALUE_OPTIONS:
if i + 1 < length:
promoted.extend([arg, args[i + 1]])
i += 2
else:
remaining.append(arg)
i += 1
continue
eq_prefix = next(
(prefix for prefix in _GLOBAL_VALUE_EQ_PREFIXES if arg.startswith(prefix)),
None,
)
if eq_prefix is not None and len(arg) > len(eq_prefix):
promoted.extend([eq_prefix.rstrip("="), arg[len(eq_prefix) :]])
i += 1
continue
split_short = _split_short_value_option(arg)
if split_short is not None:
short_opt, value = split_short
promoted.extend([short_opt, value])
i += 1
continue
if arg in _GLOBAL_VALUE_SHORT_OPTIONS:
if i + 1 < length:
promoted.extend([arg, args[i + 1]])
i += 2
else:
remaining.append(arg)
i += 1
continue
remaining.append(arg)
i += 1
return promoted + remaining
class CleverAgentsTyperGroup(TyperGroup):
"""Typer group that promotes global options before subcommand parsing."""
def parse_args(self, ctx: Any, args: list[str]) -> list[str]:
promoted_args = _promote_global_options(list(args))
return super().parse_args(ctx, promoted_args)
__all__ = [
"_print_basic_help",
"app",
@@ -69,6 +154,7 @@ app: Any = typer.Typer(
pretty_exceptions_enable=True,
pretty_exceptions_show_locals=False,
rich_markup_mode="rich",
cls=CleverAgentsTyperGroup,
)
@@ -740,6 +826,9 @@ def main(args: list[str] | None = None) -> int:
if not args:
args = ["--help"]
# Global CLI options are promoted by CleverAgentsTyperGroup.parse_args,
# so we avoid reprocessing here to prevent double handling.
# Fast paths for lightweight flags to avoid heavy imports when using the
# real Typer app. If the app is monkeypatched (e.g., tests), fall back to
# normal execution so patched call paths are exercised.
@@ -452,50 +452,17 @@ class SqliteChangeSetStore:
self,
plan_id: str,
) -> list[SpecChangeSet]:
"""Return all ChangeSets associated with *plan_id*.
Returns one ``SpecChangeSet`` per distinct changeset, preserving
per-changeset granularity instead of collapsing every entry into a
single container. Entry ordering is by timestamp within each
changeset.
"""
"""Return all ChangeSets associated with *plan_id*."""
if not plan_id:
return []
session = self._entry_repo._session()
try:
rows = (
session.query(ChangeSetEntryModel)
.filter_by(plan_id=plan_id)
.order_by(ChangeSetEntryModel.timestamp)
.all()
)
except (OperationalError, SQLAlchemyDatabaseError) as exc:
raise DatabaseError(f"Failed to get entries for plan: {exc}") from exc
if not rows:
entries = self._entry_repo.get_entries_for_plan(plan_id)
if not entries:
return []
# Group raw DB rows by their original changeset_id so each
# SpecChangeSet represents one logical chunk of work.
groups: dict[str, list[ChangeSetEntryModel]] = {}
for row in rows:
cs_id = cast(str, row.changeset_id)
groups.setdefault(cs_id, []).append(row)
result: list[SpecChangeSet] = []
for cs_id in sorted(groups):
group_rows = groups[cs_id]
entries = [self._entry_repo._to_domain(r) for r in group_rows]
plan_id_from_entries = entries[0].plan_id if entries else plan_id
result.append(
SpecChangeSet(
changeset_id=cs_id,
plan_id=plan_id_from_entries or plan_id,
entries=entries,
)
)
return result
return [
SpecChangeSet(plan_id=plan_id, entries=entries),
]
def summarize(
self,
+297 -153
View File
@@ -5,27 +5,23 @@ resource types. The handler supports a hierarchical type model where
provider-specific types (``aws-account``, ``aws-vpc``, etc.) inherit
from generic ``cloud-*`` base types.
This handler validates configuration, resolves credentials, and -- when
``boto3`` is installed -- executes real AWS SDK operations to discover
and resolve cloud resources.
AWS-specific SDK logic lives in
:mod:`cleveragents.resource.handlers.cloud_aws`. Provider credential
specifications live in
:mod:`cleveragents.resource.handlers.cloud_providers`.
This handler validates configuration and resolves credentials from
environment variables and profile names but does **not** execute any
cloud SDK operations -- actual execution raises
:exc:`NotImplementedError`.
Cloud resource types are registered as built-in types at bootstrap and
use ``sandbox_strategy = "none"`` because cloud sandbox isolation is
implemented via resource tagging (tag-based isolation strategy).
not yet implemented.
## Provider Detection
The handler extracts the cloud provider from the resource type name:
- ``aws-*`` or ``aws`` -> AWS provider
- ``gcp-*`` or ``gcp`` -> GCP provider
- ``azure-*`` or ``azure`` -> Azure provider
- ``cloud-*`` -> generic (no provider-specific validation)
- ``aws-*`` or ``aws`` AWS provider
- ``gcp-*`` or ``gcp`` GCP provider
- ``azure-*`` or ``azure`` Azure provider
- ``cloud-*`` generic (no provider-specific validation)
## Credential Resolution
@@ -35,12 +31,12 @@ Credentials are resolved in this priority order:
2. Environment variables
3. Profile names (AWS only -- ``AWS_PROFILE``)
Credential values are never logged; the existing
:mod:`cleveragents.shared.redaction` patterns handle masking.
No cloud SDK dependencies are required. Credential values are never
logged; the existing :mod:`cleveragents.shared.redaction` patterns
handle masking.
Based on:
- Issue #343: Cloud Infrastructure Resources
- Issue #1021: AWS SDK integration
- implementation_plan.md group M7.post-resource-cloud
"""
@@ -49,22 +45,12 @@ from __future__ import annotations
import hashlib
import logging
import os
from dataclasses import dataclass, field
from typing import Any
from cleveragents.domain.models.core.resource import Resource
from cleveragents.infrastructure.sandbox.manager import SandboxManager
from cleveragents.resource.handlers._base import EMPTY_CONTENT_HASH
from cleveragents.resource.handlers.cloud_aws import (
_BOTO3_AVAILABLE,
_build_aws_session,
build_child_resources,
discover_aws_resources,
)
from cleveragents.resource.handlers.cloud_providers import (
_PROVIDER_PREFIXES,
CLOUD_PROVIDERS,
_CredentialField,
)
from cleveragents.resource.handlers.protocol import (
Content,
DeleteResult,
@@ -77,6 +63,180 @@ from cleveragents.tool.context import BoundResource
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Cloud credential field definitions
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class _CredentialField:
"""Metadata for a single credential or configuration field."""
name: str
env_var: str
required: bool = True
sensitive: bool = False
description: str = ""
@dataclass(frozen=True)
class CloudProviderSpec:
"""Specification of a cloud provider's credential and config fields."""
provider: str
description: str
credential_fields: tuple[_CredentialField, ...]
metadata_fields: tuple[_CredentialField, ...] = ()
profile_env_var: str | None = None
required_fields: frozenset[str] = field(default_factory=frozenset)
@property
def all_fields(self) -> tuple[_CredentialField, ...]:
"""Return all fields (credential + metadata)."""
return self.credential_fields + self.metadata_fields
# ---------------------------------------------------------------------------
# Provider specifications
# ---------------------------------------------------------------------------
AWS_SPEC = CloudProviderSpec(
provider="aws",
description="Amazon Web Services cloud resource.",
credential_fields=(
_CredentialField(
name="access-key-id",
env_var="AWS_ACCESS_KEY_ID",
required=True,
sensitive=True,
description="AWS access key ID.",
),
_CredentialField(
name="secret-access-key",
env_var="AWS_SECRET_ACCESS_KEY",
required=True,
sensitive=True,
description="AWS secret access key.",
),
_CredentialField(
name="session-token",
env_var="AWS_SESSION_TOKEN",
required=False,
sensitive=True,
description="AWS session token (optional, for temporary credentials).",
),
),
metadata_fields=(
_CredentialField(
name="region",
env_var="AWS_REGION",
required=False,
sensitive=False,
description="AWS region (e.g. us-east-1).",
),
_CredentialField(
name="profile",
env_var="AWS_PROFILE",
required=False,
sensitive=False,
description="AWS profile name from ~/.aws/credentials.",
),
),
profile_env_var="AWS_PROFILE",
required_fields=frozenset({"access-key-id", "secret-access-key"}),
)
GCP_SPEC = CloudProviderSpec(
provider="gcp",
description="Google Cloud Platform resource.",
credential_fields=(
_CredentialField(
name="service-account-json-path",
env_var="GOOGLE_APPLICATION_CREDENTIALS",
required=True,
sensitive=True,
description="Path to GCP service account JSON key file.",
),
),
metadata_fields=(
_CredentialField(
name="project-id",
env_var="GCLOUD_PROJECT",
required=True,
sensitive=False,
description="GCP project ID.",
),
_CredentialField(
name="region",
env_var="GCP_REGION",
required=False,
sensitive=False,
description="GCP region (e.g. us-central1).",
),
),
required_fields=frozenset({"service-account-json-path", "project-id"}),
)
AZURE_SPEC = CloudProviderSpec(
provider="azure",
description="Microsoft Azure cloud resource.",
credential_fields=(
_CredentialField(
name="subscription-id",
env_var="AZURE_SUBSCRIPTION_ID",
required=True,
sensitive=True,
description="Azure subscription ID.",
),
_CredentialField(
name="tenant-id",
env_var="AZURE_TENANT_ID",
required=True,
sensitive=True,
description="Azure Active Directory tenant ID.",
),
_CredentialField(
name="client-id",
env_var="AZURE_CLIENT_ID",
required=True,
sensitive=True,
description="Azure service principal client ID.",
),
_CredentialField(
name="client-secret",
env_var="AZURE_CLIENT_SECRET",
required=True,
sensitive=True,
description="Azure service principal client secret.",
),
),
metadata_fields=(
_CredentialField(
name="region",
env_var="AZURE_REGION",
required=False,
sensitive=False,
description="Azure region (e.g. eastus).",
),
),
required_fields=frozenset(
{"subscription-id", "tenant-id", "client-id", "client-secret"}
),
)
#: Mapping from provider name to its specification.
CLOUD_PROVIDERS: dict[str, CloudProviderSpec] = {
"aws": AWS_SPEC,
"gcp": GCP_SPEC,
"azure": AZURE_SPEC,
}
#: Provider prefixes for hierarchical type name detection.
_PROVIDER_PREFIXES: tuple[str, ...] = ("aws-", "gcp-", "azure-")
def extract_provider(type_name: str) -> str | None:
"""Extract the cloud provider key from a resource type name.
@@ -101,6 +261,11 @@ def extract_provider(type_name: str) -> str | None:
return None
# ---------------------------------------------------------------------------
# Credential resolution
# ---------------------------------------------------------------------------
def _safe_value_repr(field_def: _CredentialField, value: str | None) -> str:
"""Return a safe representation of a credential value for logging."""
if value is None:
@@ -205,12 +370,18 @@ def validate_credentials(
return errors
# ---------------------------------------------------------------------------
# CloudResourceHandler
# ---------------------------------------------------------------------------
class CloudResourceHandler:
"""Handler for cloud infrastructure resource types.
Validates cloud resource configuration, resolves credentials, and --
when ``boto3`` is installed -- executes real AWS SDK operations to
discover and resolve cloud resources.
Validates cloud resource configuration and resolves credentials
from environment variables and profile names. Actual cloud
operations are **not** implemented -- calling :meth:`resolve`
raises :exc:`NotImplementedError` after validation.
Supports hierarchical type names (``aws-account``, ``aws-vpc``,
etc.) in addition to flat provider names (``aws``, ``gcp``,
@@ -231,33 +402,29 @@ class CloudResourceHandler:
sandbox_manager: SandboxManager,
access: str = "read_only",
) -> BoundResource:
"""Resolve a cloud resource into a :class:`BoundResource`.
"""Validate cloud resource configuration and raise.
For AWS resources, builds a boto3 session and returns a
:class:`BoundResource` with the resource ARN as ``sandbox_path``.
For GCP and Azure, raises :exc:`NotImplementedError`.
Performs full credential validation and resolution, then
raises :exc:`NotImplementedError` because cloud sandbox
execution is not yet implemented.
Args:
resource: A cloud resource instance.
plan_id: The plan requesting the sandbox.
slot_name: Name of the tool resource slot being filled.
sandbox_manager: The sandbox lifecycle manager (unused for
cloud resources).
access: Access mode (``read_only`` or ``read_write``).
Returns:
A :class:`BoundResource` with ``sandbox_path`` set to the
resource ARN or location.
sandbox_manager: The sandbox lifecycle manager (unused).
access: Access mode (unused).
Raises:
ValueError: If required credentials are missing.
ImportError: If ``boto3`` is not installed (AWS only).
NotImplementedError: For GCP/Azure or generic ``cloud-*`` types.
ValueError: If required credentials are missing or the
resource type is not a supported cloud provider.
NotImplementedError: Always, after successful validation.
"""
type_name = resource.resource_type_name
provider = extract_provider(type_name)
if provider is None:
# Generic cloud-* base type — no provider-specific validation.
raise NotImplementedError(
f"Cloud resource execution for generic type '{type_name}' "
f"is not yet implemented. Resource '{resource.resource_id}' "
@@ -272,13 +439,26 @@ class CloudResourceHandler:
f"Supported: {', '.join(sorted(CLOUD_PROVIDERS))}."
)
# Resolve credentials
resolved = resolve_credentials(provider, dict(resource.properties))
# Log resolution (with redaction)
for field_def in CLOUD_PROVIDERS[provider].all_fields:
safe = _safe_value_repr(field_def, resolved.get(field_def.name))
logger.debug("Cloud credential %s/%s = %s", provider, field_def.name, safe)
logger.debug(
"Cloud credential %s/%s = %s",
provider,
field_def.name,
safe,
)
is_account_type = type_name in ("aws", "gcp", "azure", "aws-account")
# Validate credentials (only for account-level types that carry creds)
is_account_type = type_name in (
"aws",
"gcp",
"azure",
"aws-account",
)
if is_account_type:
errors = validate_credentials(provider, resolved)
if errors:
@@ -287,86 +467,15 @@ class CloudResourceHandler:
f"(provider={provider}): " + "; ".join(errors)
)
if provider == "aws":
return self._resolve_aws(
resource=resource,
plan_id=plan_id,
slot_name=slot_name,
resolved=resolved,
access=access,
)
raise NotImplementedError(
f"Cloud resource execution for '{type_name}' "
f"(provider={provider}) is not yet implemented. "
f"Resource '{resource.resource_id}' "
f"(plan={plan_id}, slot={slot_name}) validated successfully "
f"but SDK integration for {provider} is pending."
f"but sandbox provisioning for cloud resources is pending."
)
def _resolve_aws(
self,
*,
resource: Resource,
plan_id: str,
slot_name: str,
resolved: dict[str, str | None],
access: str,
) -> BoundResource:
"""Resolve an AWS resource using boto3.
Args:
resource: The AWS resource to resolve.
plan_id: The plan ID (used for tagging).
slot_name: The tool slot name.
resolved: Resolved credential dict.
access: Access mode.
Returns:
A :class:`BoundResource` for the AWS resource.
Raises:
ImportError: If ``boto3`` is not installed.
ValueError: If AWS credentials are invalid or the API call fails.
"""
if not _BOTO3_AVAILABLE:
raise ImportError(
"boto3 is required for AWS resource operations. "
"Install it with: pip install cleveragents[aws]"
)
session = _build_aws_session(resolved)
location = resource.location or ""
type_name = resource.resource_type_name
is_account_type = type_name in ("aws", "aws-account")
if is_account_type:
try:
sts = session.client(
"sts",
region_name=resolved.get("region") or "us-east-1",
)
identity = sts.get_caller_identity()
account_id = identity.get("Account", "")
location = location or f"arn:aws:iam::{account_id}:root"
logger.debug(
"AWS STS identity verified for account %s (plan=%s)",
account_id,
plan_id,
)
except Exception as exc:
raise ValueError(
f"AWS credential verification failed for resource "
f"'{resource.resource_id}' (plan={plan_id}): {exc}"
) from exc
return BoundResource(
slot_name=slot_name,
resource_id=resource.resource_id,
resource_type=type_name,
sandbox_path=location or None,
access=access,
)
# -- CRUD stubs (required by ResourceHandler protocol) -----------------
def read(self, *, resource: Resource, path: str = "") -> Content:
"""Not supported by the cloud handler."""
@@ -389,41 +498,10 @@ class CloudResourceHandler:
raise NotImplementedError("Cloud handler does not support diff()")
def discover_children(self, *, resource: Resource) -> list[Resource]:
"""Discover child AWS resources using the boto3 SDK.
"""Not supported for cloud resources."""
raise NotImplementedError("Cloud handler does not support discover_children()")
For AWS resource types that have a discovery mapping (VPCs,
subnets, instances, etc.), this method queries the AWS API and
returns a list of child :class:`Resource` objects.
Args:
resource: The parent AWS resource.
Returns:
List of discovered child :class:`Resource` objects.
Raises:
ImportError: If ``boto3`` is not installed.
NotImplementedError: For non-AWS providers.
"""
type_name = resource.resource_type_name
provider = extract_provider(type_name)
if provider != "aws":
raise NotImplementedError(
f"Cloud handler discover_children() is only implemented "
f"for aws resources (got provider={provider!r})."
)
if not _BOTO3_AVAILABLE:
raise ImportError(
"boto3 is required for AWS resource discovery. "
"Install it with: pip install cleveragents[aws]"
)
resolved = resolve_credentials("aws", dict(resource.properties))
session = _build_aws_session(resolved)
discovered = discover_aws_resources(resource, session)
return build_child_resources(resource, discovered)
# -- Lifecycle stubs (issue #836) --------------------------------------
def create_sandbox(
self,
@@ -476,6 +554,10 @@ class CloudResourceHandler:
) -> str:
"""Compute content hash for cloud resources.
Cloud resources do not have local content. Returns an identity
hash based on the resource type name and location (ARN, project
ID, or subscription ID).
Returns :data:`EMPTY_CONTENT_HASH` if no location is set.
"""
if not resource.location:
@@ -486,3 +568,65 @@ class CloudResourceHandler:
h.update(b"\0")
h.update(resource.location.encode("utf-8"))
return h.hexdigest()
# ---------------------------------------------------------------------------
# Stubbed sandbox strategies
# ---------------------------------------------------------------------------
class CloudSandboxStrategy:
"""Stubbed sandbox strategy for cloud resources.
Validates configuration but raises :exc:`NotImplementedError` for
all lifecycle operations (create, commit, rollback). This is the
expected behaviour per the issue specification.
"""
def __init__(self, provider: str) -> None:
self._provider = provider
def validate(self, properties: dict[str, Any]) -> list[str]:
"""Validate cloud resource configuration.
Args:
properties: Resource properties dict.
Returns:
List of validation error messages (empty if valid).
"""
resolved = resolve_credentials(self._provider, properties)
return validate_credentials(self._provider, resolved)
def create(self, resource_id: str, plan_id: str) -> None:
"""Create a cloud sandbox (not implemented).
Raises:
NotImplementedError: Always.
"""
raise NotImplementedError(
f"Cloud sandbox creation for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
def commit(self, resource_id: str, plan_id: str) -> None:
"""Commit a cloud sandbox (not implemented).
Raises:
NotImplementedError: Always.
"""
raise NotImplementedError(
f"Cloud sandbox commit for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
def rollback(self, resource_id: str, plan_id: str) -> None:
"""Rollback a cloud sandbox (not implemented).
Raises:
NotImplementedError: Always.
"""
raise NotImplementedError(
f"Cloud sandbox rollback for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
@@ -1,523 +0,0 @@
"""AWS SDK integration for CloudResourceHandler.
Provides AWS-specific session building, resource discovery, and sandbox
strategy implementation using ``boto3`` as an optional dependency.
This module is imported by :mod:`cleveragents.resource.handlers.cloud`
when AWS provider operations are requested. It is intentionally
separated from the generic cloud handler to keep both files under the
500-line limit and to maintain clean module boundaries.
## Optional Dependency
``boto3`` is an optional dependency. If it is not installed, all
functions in this module raise :exc:`ImportError` with a helpful
installation message.
## Sandbox Strategy
Cloud sandbox isolation uses a **tag-based** strategy: a unique
``CleverAgents:PlanId`` tag is applied to all resources created or
modified during a plan. On rollback, tagged resources are deleted or
reverted. On commit, the tag is removed.
Based on:
- Issue #1021: AWS SDK integration
"""
from __future__ import annotations
import importlib
import logging
import types
from typing import Any
from cleveragents.domain.models.core.resource import (
PhysVirt,
Resource,
ResourceCapabilities,
)
from cleveragents.resource.handlers._base import _derive_child_id
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Optional boto3 import
# ---------------------------------------------------------------------------
_BOTO3_AVAILABLE = False
boto3: types.ModuleType | None = None
botocore: types.ModuleType | None = None
try:
boto3 = importlib.import_module("boto3")
botocore = importlib.import_module("botocore.exceptions")
_BOTO3_AVAILABLE = True
except ImportError:
pass
# ---------------------------------------------------------------------------
# AWS resource type -> boto3 service/describe mapping
# ---------------------------------------------------------------------------
#: Maps AWS resource type names to (service_name, describe_method, id_key, arn_prefix).
#: Used by :func:`discover_aws_resources` to enumerate child resources.
_AWS_RESOURCE_MAP: dict[str, tuple[str, str, str, str]] = {
"aws-vpc": ("ec2", "describe_vpcs", "VpcId", "vpc"),
"aws-subnet": ("ec2", "describe_subnets", "SubnetId", "subnet"),
"aws-instance": ("ec2", "describe_instances", "InstanceId", "instance"),
"aws-security-group": (
"ec2",
"describe_security_groups",
"GroupId",
"security-group",
),
"aws-s3-bucket": ("s3", "list_buckets", "Name", "s3"),
"aws-iam-role": ("iam", "list_roles", "RoleName", "iam-role"),
"aws-rds-instance": (
"rds",
"describe_db_instances",
"DBInstanceIdentifier",
"rds",
),
"aws-ecs-cluster": ("ecs", "list_clusters", "clusterArns", "ecs-cluster"),
"aws-lambda-function": (
"lambda",
"list_functions",
"FunctionName",
"lambda",
),
"aws-eks-cluster": ("eks", "list_clusters", "clusters", "eks-cluster"),
}
# ---------------------------------------------------------------------------
# AWS boto3 session factory
# ---------------------------------------------------------------------------
def _build_aws_session(resolved: dict[str, str | None]) -> Any:
"""Build a boto3 Session from resolved credentials.
Args:
resolved: Resolved credential dict from
:func:`~cleveragents.resource.handlers.cloud.resolve_credentials`.
Returns:
A ``boto3.Session`` instance.
Raises:
ImportError: If ``boto3`` is not installed.
"""
if not _BOTO3_AVAILABLE or boto3 is None:
raise ImportError(
"boto3 is required for AWS resource operations. "
"Install it with: pip install cleveragents[aws]"
)
kwargs: dict[str, str] = {}
access_key = resolved.get("access-key-id")
if access_key is not None:
kwargs["aws_access_key_id"] = access_key
secret_key = resolved.get("secret-access-key")
if secret_key is not None:
kwargs["aws_secret_access_key"] = secret_key
session_token = resolved.get("session-token")
if session_token is not None:
kwargs["aws_session_token"] = session_token
region = resolved.get("region")
if region is not None:
kwargs["region_name"] = region
profile = resolved.get("profile")
if profile is not None:
kwargs["profile_name"] = profile
# boto3 is confirmed non-None by the guard above; use attribute access
return boto3.Session(**kwargs) # type: ignore[union-attr]
# ---------------------------------------------------------------------------
# AWS resource discovery
# ---------------------------------------------------------------------------
def discover_aws_resources(
resource: Resource,
session: Any,
) -> list[dict[str, Any]]:
"""Discover AWS child resources using the boto3 SDK.
Queries the AWS API to enumerate resources of the type specified by
``resource.resource_type_name``. Returns a list of resource metadata
dicts, each containing at minimum ``id``, ``type``, and ``arn`` keys.
Args:
resource: The parent AWS resource (account or container type).
session: A ``boto3.Session`` instance.
Returns:
List of discovered resource metadata dicts. Empty list if the
resource type is not mapped or the API call returns no results.
Raises:
botocore.exceptions.ClientError: If the AWS API returns a client
error (e.g. permission denied, invalid credentials).
botocore.exceptions.BotoCoreError: If a lower-level botocore
error occurs (e.g. network failure, endpoint resolution).
"""
type_name = resource.resource_type_name
mapping = _AWS_RESOURCE_MAP.get(type_name)
if mapping is None:
logger.debug(
"No AWS discovery mapping for resource type '%s'; skipping.", type_name
)
return []
service_name, method_name, id_key, arn_prefix = mapping
region = resource.properties.get("region", "us-east-1") or "us-east-1"
try:
client = session.client(service_name, region_name=region)
method = getattr(client, method_name)
response = method()
except Exception as exc:
logger.warning("AWS API error during discovery of '%s': %s", type_name, exc)
return []
results: list[dict[str, Any]] = []
# Handle S3 list_buckets (flat list)
if service_name == "s3" and method_name == "list_buckets":
for bucket in response.get("Buckets", []):
name = bucket.get("Name", "")
results.append(
{
"id": name,
"type": type_name,
"arn": f"arn:aws:s3:::{name}",
"name": name,
"metadata": bucket,
}
)
return results
# Handle ECS list_clusters (returns ARN list)
if service_name == "ecs" and method_name == "list_clusters":
for arn in response.get("clusterArns", []):
cluster_name = arn.split("/")[-1]
results.append(
{
"id": cluster_name,
"type": type_name,
"arn": arn,
"name": cluster_name,
"metadata": {"clusterArn": arn},
}
)
return results
# Handle EKS list_clusters (returns name list)
if service_name == "eks" and method_name == "list_clusters":
for name in response.get("clusters", []):
results.append(
{
"id": name,
"type": type_name,
"arn": f"arn:aws:eks:{region}::cluster/{name}",
"name": name,
"metadata": {},
}
)
return results
# Handle IAM list_roles
if service_name == "iam" and method_name == "list_roles":
for role in response.get("Roles", []):
role_name = role.get("RoleName", "")
results.append(
{
"id": role_name,
"type": type_name,
"arn": role.get("Arn", ""),
"name": role_name,
"metadata": role,
}
)
return results
# Handle Lambda list_functions
if service_name == "lambda" and method_name == "list_functions":
for fn in response.get("Functions", []):
fn_name = fn.get("FunctionName", "")
results.append(
{
"id": fn_name,
"type": type_name,
"arn": fn.get("FunctionArn", ""),
"name": fn_name,
"metadata": fn,
}
)
return results
# Handle RDS describe_db_instances
if service_name == "rds" and method_name == "describe_db_instances":
for db in response.get("DBInstances", []):
db_id = db.get("DBInstanceIdentifier", "")
results.append(
{
"id": db_id,
"type": type_name,
"arn": db.get("DBInstanceArn", ""),
"name": db_id,
"metadata": db,
}
)
return results
# Handle EC2 describe_instances (nested Reservations → Instances structure)
if service_name == "ec2" and method_name == "describe_instances":
account_id = resource.properties.get("account-id", "")
for reservation in response.get("Reservations", []):
for inst in reservation.get("Instances", []):
inst_id = inst.get("InstanceId", "")
results.append(
{
"id": inst_id,
"type": type_name,
"arn": f"arn:aws:ec2:{region}:{account_id}:instance/{inst_id}",
"name": inst_id,
"metadata": inst,
}
)
return results
# Generic EC2-style: describe_* returns a list under a plural key
# e.g. describe_vpcs -> Vpcs, describe_subnets -> Subnets
for key, value in response.items():
if key in ("ResponseMetadata",):
continue
if isinstance(value, list):
for item in value:
if isinstance(item, dict):
resource_id = item.get(id_key, "")
# Build a best-effort ARN
account_id = resource.properties.get("account-id", "")
arn = f"arn:aws:{arn_prefix}:{region}:{account_id}:{resource_id}"
results.append(
{
"id": resource_id,
"type": type_name,
"arn": arn,
"name": resource_id,
"metadata": item,
}
)
break
return results
def build_child_resources(
parent: Resource,
discovered: list[dict[str, Any]],
) -> list[Resource]:
"""Convert raw discovery dicts into :class:`Resource` objects.
Args:
parent: The parent resource whose children are being built.
discovered: List of dicts from :func:`discover_aws_resources`.
Returns:
List of :class:`Resource` objects for the discovered children.
"""
children: list[Resource] = []
for item in discovered:
child_id = _derive_child_id(parent.resource_id, item["id"])
child = Resource(
resource_id=child_id,
resource_type_name=item["type"],
classification=PhysVirt.PHYSICAL,
location=item.get("arn") or None,
properties=item.get("metadata", {}),
capabilities=ResourceCapabilities(
readable=True,
writable=False,
sandboxable=False,
checkpointable=False,
),
)
children.append(child)
return children
# ---------------------------------------------------------------------------
# Cloud sandbox strategy (tag-based isolation)
# ---------------------------------------------------------------------------
#: Tag key applied to all resources created/modified during a plan.
_PLAN_TAG_KEY = "CleverAgents:PlanId"
#: Tag value prefix.
_PLAN_TAG_PREFIX = "ca-plan-"
class CloudSandboxStrategy:
"""Tag-based sandbox strategy for AWS cloud resources.
Applies a ``CleverAgents:PlanId`` tag to all resources created or
modified during a plan. On rollback, tagged resources are deleted
or reverted. On commit, the tag is removed.
For non-AWS providers, all lifecycle operations raise
:exc:`NotImplementedError`.
Validates configuration but raises :exc:`NotImplementedError` for
GCP/Azure lifecycle operations (not yet implemented).
"""
def __init__(self, provider: str) -> None:
self._provider = provider
def validate(self, properties: dict[str, Any]) -> list[str]:
"""Validate cloud resource configuration.
Args:
properties: Resource properties dict.
Returns:
List of validation error messages (empty if valid).
"""
# Deferred import to avoid circular dependency at module level.
from cleveragents.resource.handlers.cloud import (
resolve_credentials,
validate_credentials,
)
resolved = resolve_credentials(self._provider, properties)
return validate_credentials(self._provider, resolved)
def create(self, resource_id: str, plan_id: str) -> None:
"""Create a cloud sandbox by tagging the resource.
For AWS resources, applies the ``CleverAgents:PlanId`` tag to
mark the resource as part of this plan's sandbox.
For non-AWS providers, raises :exc:`NotImplementedError`.
Args:
resource_id: The cloud resource identifier (ARN or ID).
plan_id: The plan ID to use as the tag value.
Raises:
ValueError: If ``plan_id`` is empty or blank.
ImportError: If ``boto3`` is not installed (AWS only).
NotImplementedError: For non-AWS providers.
"""
if not plan_id or not plan_id.strip():
raise ValueError(f"Invalid plan_id '{plan_id}' for AWS sandbox creation.")
if self._provider != "aws":
raise NotImplementedError(
f"Cloud sandbox creation for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
if not _BOTO3_AVAILABLE:
raise ImportError(
"boto3 is required for AWS sandbox operations. "
"Install it with: pip install cleveragents[aws]"
)
tag_value = f"{_PLAN_TAG_PREFIX}{plan_id}"
logger.info(
"AWS sandbox: tagging resource '%s' with %s=%s (plan=%s)",
resource_id,
_PLAN_TAG_KEY,
tag_value,
plan_id,
)
# Tag application is deferred to the actual resource operation;
# here we record the intent and validate the plan ID format.
def commit(self, resource_id: str, plan_id: str) -> None:
"""Commit a cloud sandbox by removing the plan tag.
For AWS resources, removes the ``CleverAgents:PlanId`` tag to
finalise the sandbox and make changes permanent.
For non-AWS providers, raises :exc:`NotImplementedError`.
Args:
resource_id: The cloud resource identifier (ARN or ID).
plan_id: The plan ID whose tag should be removed.
Raises:
ValueError: If ``plan_id`` is empty or blank.
ImportError: If ``boto3`` is not installed (AWS only).
NotImplementedError: For non-AWS providers.
"""
if not plan_id or not plan_id.strip():
raise ValueError(f"Invalid plan_id '{plan_id}' for AWS sandbox commit.")
if self._provider != "aws":
raise NotImplementedError(
f"Cloud sandbox commit for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
if not _BOTO3_AVAILABLE:
raise ImportError(
"boto3 is required for AWS sandbox operations. "
"Install it with: pip install cleveragents[aws]"
)
logger.info(
"AWS sandbox: removing tag %s from resource '%s' (plan=%s)",
_PLAN_TAG_KEY,
resource_id,
plan_id,
)
def rollback(self, resource_id: str, plan_id: str) -> None:
"""Rollback a cloud sandbox by reverting tagged resources.
For AWS resources, identifies all resources tagged with
``CleverAgents:PlanId=<plan_id>`` and reverts or deletes them.
For non-AWS providers, raises :exc:`NotImplementedError`.
Args:
resource_id: The cloud resource identifier (ARN or ID).
plan_id: The plan ID whose tagged resources should be
reverted.
Raises:
ValueError: If ``plan_id`` is empty or blank.
ImportError: If ``boto3`` is not installed (AWS only).
NotImplementedError: For non-AWS providers.
"""
if not plan_id or not plan_id.strip():
raise ValueError(f"Invalid plan_id '{plan_id}' for AWS sandbox rollback.")
if self._provider != "aws":
raise NotImplementedError(
f"Cloud sandbox rollback for provider '{self._provider}' "
f"is not yet implemented (resource={resource_id}, plan={plan_id})."
)
if not _BOTO3_AVAILABLE:
raise ImportError(
"boto3 is required for AWS sandbox operations. "
"Install it with: pip install cleveragents[aws]"
)
logger.info(
"AWS sandbox: rolling back resources tagged %s=%s%s (resource=%s)",
_PLAN_TAG_KEY,
_PLAN_TAG_PREFIX,
plan_id,
resource_id,
)
@@ -1,181 +0,0 @@
"""Cloud provider specifications for CleverAgents.
Defines credential field metadata and provider specifications for
supported cloud providers (AWS, GCP, Azure).
This module is imported by :mod:`cleveragents.resource.handlers.cloud`
and provides the data-only definitions that describe each provider's
required and optional credential fields.
Based on:
- Issue #343: Cloud Infrastructure Resources
- Issue #1021: AWS SDK integration
"""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass(frozen=True)
class _CredentialField:
"""Metadata for a single credential or configuration field."""
name: str
env_var: str
required: bool = True
sensitive: bool = False
description: str = ""
@dataclass(frozen=True)
class CloudProviderSpec:
"""Specification of a cloud provider's credential and config fields."""
provider: str
description: str
credential_fields: tuple[_CredentialField, ...]
metadata_fields: tuple[_CredentialField, ...] = ()
profile_env_var: str | None = None
required_fields: frozenset[str] = field(default_factory=frozenset)
@property
def all_fields(self) -> tuple[_CredentialField, ...]:
"""Return all fields (credential + metadata)."""
return self.credential_fields + self.metadata_fields
AWS_SPEC = CloudProviderSpec(
provider="aws",
description="Amazon Web Services cloud resource.",
credential_fields=(
_CredentialField(
name="access-key-id",
env_var="AWS_ACCESS_KEY_ID",
required=True,
sensitive=True,
description="AWS access key ID.",
),
_CredentialField(
name="secret-access-key",
env_var="AWS_SECRET_ACCESS_KEY",
required=True,
sensitive=True,
description="AWS secret access key.",
),
_CredentialField(
name="session-token",
env_var="AWS_SESSION_TOKEN",
required=False,
sensitive=True,
description="AWS session token (optional, for temporary credentials).",
),
),
metadata_fields=(
_CredentialField(
name="region",
env_var="AWS_REGION",
required=False,
sensitive=False,
description="AWS region (e.g. us-east-1).",
),
_CredentialField(
name="profile",
env_var="AWS_PROFILE",
required=False,
sensitive=False,
description="AWS profile name from ~/.aws/credentials.",
),
),
profile_env_var="AWS_PROFILE",
required_fields=frozenset({"access-key-id", "secret-access-key"}),
)
GCP_SPEC = CloudProviderSpec(
provider="gcp",
description="Google Cloud Platform resource.",
credential_fields=(
_CredentialField(
name="service-account-json-path",
env_var="GOOGLE_APPLICATION_CREDENTIALS",
required=True,
sensitive=True,
description="Path to GCP service account JSON key file.",
),
),
metadata_fields=(
_CredentialField(
name="project-id",
env_var="GCLOUD_PROJECT",
required=True,
sensitive=False,
description="GCP project ID.",
),
_CredentialField(
name="region",
env_var="GCP_REGION",
required=False,
sensitive=False,
description="GCP region (e.g. us-central1).",
),
),
required_fields=frozenset({"service-account-json-path", "project-id"}),
)
AZURE_SPEC = CloudProviderSpec(
provider="azure",
description="Microsoft Azure cloud resource.",
credential_fields=(
_CredentialField(
name="subscription-id",
env_var="AZURE_SUBSCRIPTION_ID",
required=True,
sensitive=True,
description="Azure subscription ID.",
),
_CredentialField(
name="tenant-id",
env_var="AZURE_TENANT_ID",
required=True,
sensitive=True,
description="Azure Active Directory tenant ID.",
),
_CredentialField(
name="client-id",
env_var="AZURE_CLIENT_ID",
required=True,
sensitive=True,
description="Azure service principal client ID.",
),
_CredentialField(
name="client-secret",
env_var="AZURE_CLIENT_SECRET",
required=True,
sensitive=True,
description="Azure service principal client secret.",
),
),
metadata_fields=(
_CredentialField(
name="region",
env_var="AZURE_REGION",
required=False,
sensitive=False,
description="Azure region (e.g. eastus).",
),
),
required_fields=frozenset(
{"subscription-id", "tenant-id", "client-id", "client-secret"}
),
)
#: Mapping from provider name to its specification.
CLOUD_PROVIDERS: dict[str, CloudProviderSpec] = {
"aws": AWS_SPEC,
"gcp": GCP_SPEC,
"azure": AZURE_SPEC,
}
#: Provider prefixes for hierarchical type name detection.
_PROVIDER_PREFIXES: tuple[str, ...] = ("aws-", "gcp-", "azure-")
-16
View File
@@ -1,16 +0,0 @@
"""Minimal type stub for boto3."""
from typing import Any
class Session:
def __init__(
self,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
region_name: str | None = None,
profile_name: str | None = None,
**kwargs: Any,
) -> None: ...
def client(self, service_name: str, **kwargs: Any) -> Any: ...
def resource(self, service_name: str, **kwargs: Any) -> Any: ...
-11
View File
@@ -1,11 +0,0 @@
"""Minimal type stub for botocore.exceptions."""
class BotoCoreError(Exception): ...
class ClientError(BotoCoreError):
response: dict[str, object]
operation_name: str
def __init__(
self, error_response: dict[str, object], operation_name: str
) -> None: ...
-11
View File
@@ -1,11 +0,0 @@
"""Minimal type stub for botocore.exceptions."""
class BotoCoreError(Exception): ...
class ClientError(BotoCoreError):
response: dict[str, object]
operation_name: str
def __init__(
self, error_response: dict[str, object], operation_name: str
) -> None: ...