The Semgrep patterns node is a YAML list (not dict), so the previous
implementation's .get() call on patterns would fail with AttributeError.
Fix: properly handle 'patterns' as a list of alternation groups, each
containing pattern-either -> patterns -> pattern-not entries. This
ensures the BDD assertion robustly finds bare raise pattern-nots in
the xml-no-suppressed-exception rule's nested structure.
Issues Closed: #9103
Document HAL 9000's contribution of the broad exception suppression
Semgrep guard (PR #9185, issue #9103) in CONTRIBUTORS.md per the
PR compliance checklist requirement.
Address blocking issues from PR review by HAL9001 (review #7080):
1. Add success_codes=[0, 1] to noxfile.py semgrep invocation so
the lint session runs in proper audit mode without failing CI on
the ~337 existing violations during phased rollout.
2. Strengthen BDD bare re-raise assertion: replace weak string search
for 'raise' with structured parsing of YAML pattern-not entries to
specifically verify bare raise patterns exist within the rule's
pattern-not configurations.
3. Add CHANGELOG.md entry under [Unreleased] documenting the Semgrep
guard implementation and its audit-mode migration strategy.
Issues Closed: #9103
Add 12 new Behave BDD scenarios to features/security_scan_hooks.feature
to automatically validate the Semgrep broad exception suppression rules
and their escape hatch behavior, as suggested by reviewer HAL9001.
New scenarios verify:
- python-no-suppressed-exception rule exists and targets src/
- python-no-suppress-exception rule exists and targets src/
- Both rules document the nosemgrep escape hatch in their messages
- Both rules document the error-propagation: allow annotation
- python-no-suppressed-exception has pattern-not for bare re-raise
- python-no-suppressed-exception has pattern-not for exception chaining
- Nox lint session integrates Semgrep and references .semgrep.yml
All 25 scenarios pass (13 original + 12 new).
ISSUES CLOSED: #9103
Fix broken escape hatch mechanism and address reviewer feedback:
- Replace non-functional comment-based pattern-not clauses with Semgrep's native # nosemgrep mechanism for the escape hatch. Semgrep strips comments from the AST so pattern-not clauses matching inline comments never fire; # nosemgrep is the only reliable per-line suppression mechanism.
- Require both # nosemgrep: <rule-id> AND # error-propagation: allow on the same line: the former is the actual suppression, the latter is the mandatory human-readable audit annotation.
- Add raise $EXC from $CAUSE pattern-not entries for both Exception and BaseException variants to prevent false positives on legitimate exception chaining (raise ServiceError from e).
- Switch nox -s lint Semgrep invocation to audit mode (no --error) for the phased rollout: the codebase has ~337 existing suppressions that must be triaged before enforcement mode is enabled. A comment in noxfile.py documents the migration path and references #9103.
- Update CONTRIBUTING.md examples and enforcement description to reflect the dual-comment requirement.
Fixed two critical issues with the Semgrep rules for broad exception suppression:
1. **Escape hatch mechanism**: Replaced the broken `# error-propagation: allow` comment-based escape hatch with Semgrep's native `# nosemgrep` comment support. Semgrep strips comments from the AST, so pattern-not clauses looking for inline comments never match. The native `# nosemgrep` comment is properly supported by Semgrep and provides a reliable override mechanism.
2. **Missing exception chaining pattern**: Added `raise $EXC from $CAUSE` pattern-not entries for both `Exception` and `BaseException` variants. This prevents false positives when legitimate exception chaining is used (e.g., `raise ServiceError("context") from e`), which is an allowed pattern per CONTRIBUTING.md.
Updated both `python-no-suppressed-exception` and `python-no-suppress-exception` rules with these fixes. The escape hatch now works reliably and exception chaining is properly recognized as an allowed pattern.
ISSUES CLOSED: #9103
Added two new Semgrep rules to enforce the CONTRIBUTING.md guideline against broad exception suppression:
- python-no-suppressed-exception: Detects except Exception/BaseException blocks without re-raising
- python-no-suppress-exception: Detects contextlib.suppress(Exception/BaseException) usage
Both rules support an escape hatch annotation '# error-propagation: allow' for documented recovery logic.
Integrated Semgrep into the nox lint session to run alongside Ruff checks.
Updated CONTRIBUTING.md to document the escape hatch policy and automated enforcement.
Pre-commit hook already configured to run these rules.
ISSUES CLOSED: #9103
Expand multi-exception tuples in _compute_actor_impact() to one
exception type per line, satisfying ruff format's layout rules.
Collapse single-argument @when() decorator to one line where it fits.
ISSUES CLOSED: #8567
Replace `except _defensive_exc:` with literal exception tuples in all three
except clauses of _compute_actor_impact(). Uses multi-line tuple formatting
to satisfy both ruff B030 (no variable in except clause) and E501 (line length).
ISSUES CLOSED: #8567
- Add return type annotation tuple[Any, Any | None] to _get_services()
- Fix _load_config_text() to catch yaml.YAMLError and TypeError instead
of ValueError/AttributeError around yaml.safe_load(), preserving the
user-friendly typer.BadParameter message for malformed YAML input
- Fix _compute_actor_impact() defensive guards to also catch
CleverAgentsError, OperationalError, and ValidationError in addition
to AttributeError/RuntimeError, keeping actor removal resilient when
the database layer is unavailable
- Update CHANGELOG.md with entry under Changed section for issue #8567
- Add features/actor_exception_handling.feature with four Behave scenarios
covering the exception handling contract changes
- Add features/steps/actor_exception_handling_steps.py with step definitions
ISSUES CLOSED: #8567
The generic EC2 fallback iterated over Reservations dicts and called
item.get("InstanceId", "") on each Reservation, which always returned ""
because Reservations have "ReservationId"/"Instances" — not "InstanceId".
Every result had id="" and a malformed ARN.
Add a dedicated handler before the generic fallback that unpacks
response["Reservations"] -> reservation["Instances"] -> InstanceId,
mirroring the pattern used for s3/ecs/eks/iam/lambda/rds.
Also add BDD scenario, step definitions, and mock helper support for
describe_instances returning N instances with non-empty IDs.
ISSUES CLOSED: #1021
Remove duplicate except ImportError blocks in cloud_resources_steps.py
(B025 violations in step_sandbox_create, step_sandbox_commit,
step_sandbox_rollback). Apply ruff format to 5 files flagged by the
format check.
- Extract AWS-specific logic into cloud_aws.py and cloud_providers.py to keep all files under the 500-line limit (cloud.py: 490 lines, cloud_aws.py: 498 lines, cloud_providers.py: 181 lines)
- Fix sandbox test regression: change cloud_resources.feature sandbox create scenario from "aws" to "gcp" provider (AWS no longer raises NotImplementedError)
- Add ImportError handling to step_sandbox_create/commit/rollback in cloud_resources_steps.py
- Remove all 9 type: ignore comments from production code using proper type narrowing and boto3/botocore type stubs in typings/
- Move plan_id validation before logger.info() in all three sandbox methods (fail-fast principle)
- Fix exception suppression: discover_aws_resources() now propagates exceptions instead of catching bare Exception
- Move deferred imports (PhysVirt, ResourceCapabilities, _derive_child_id) to module level in cloud_aws.py
- Split cloud_aws_sdk_steps.py (755 lines) into focused modules: cloud_aws_helpers.py, cloud_aws_session_steps.py, cloud_aws_discover_steps.py, cloud_aws_sandbox_steps.py
- Add CHANGELOG.md entry for AWS SDK integration feature
- Update robot/helper_cloud_resources.py to handle ImportError/ NotImplementedError for AWS sandbox operations
ISSUES CLOSED: #1021
Implements real AWS SDK integration for CloudResourceHandler using boto3
as an optional dependency. Key changes:
- Add boto3/botocore as optional [aws] dependency in pyproject.toml
- Implement CloudResourceHandler.resolve() for AWS: builds boto3 session,
verifies credentials via STS get_caller_identity for account-level types,
and returns a BoundResource with the resource ARN as sandbox_path
- Implement discover_aws_resources() to enumerate VPCs, subnets, instances,
S3 buckets, IAM roles, RDS instances, ECS clusters, Lambda functions,
and EKS clusters via the AWS API
- Implement CloudResourceHandler.discover_children() for AWS resource types
using the new discovery function
- Implement CloudSandboxStrategy.create/commit/rollback for AWS using a
tag-based isolation strategy (CleverAgents:PlanId tag)
- GCP and Azure providers still raise NotImplementedError (pending)
- boto3 is optional: handler raises ImportError with helpful install message
when boto3 is not installed
- Credentials are never logged (existing redaction infrastructure preserved)
- Update cloud_resources.feature to reflect new AWS behavior
- Add comprehensive cloud_aws_sdk.feature with 47 BDD scenarios covering
all new code paths with mocked boto3
Closes#1021
The plan correct --format json output already nests correction fields
under data.correction and sets command to "plan correct" via
format_output, so the three BDD scenarios now pass against the current
implementation. Remove the @tdd_expected_fail inversion tag so CI
reports them as passing, not as unexpected-pass failures.
Updated CHANGELOG.md and CONTRIBUTORS.md.
ISSUES CLOSED: #8584
Deferred the import of ``cleveragents.cli.commands.plan`` from module
level to a lazy helper ``_get_plan_app()``. The module triggers a
~100 s import chain (application container, all services, repositories,
etc.) when loaded cold. Loading it at step-definition import time
added that cost to every behave worker process before any scenario ran,
causing the unit_tests CI job to time out.
By deferring to the first step execution the module is already cached
in ``sys.modules`` (loaded by the many other step files that import it
at module level), so the actual cost at execution time is near-zero.
The three @tdd_expected_fail scenarios still fail with AssertionError
(data.correction.mode absent, command field empty) and the TDD
inversion hook correctly converts those failures to passes in CI.
ISSUES CLOSED: #8584
Add @tdd_expected_fail BDD scenarios that prove the plan correct --format json
command outputs a flat dict instead of the spec-required nested envelope structure.
The scenarios assert data.correction.mode is present (currently absent) and that
the command field is "plan correct" (currently empty string).
ISSUES CLOSED: #8584
Previously get_for_plan() collapsed all ChangeEntry rows from a plan into
a single SpecChangeSet, losing the ability to distinguish between
separate changeset scopes. This restores per-changeset grouping by
querying raw DB rows and partitioning them by changeset_id before
constructing individual SpecChangeSet objects.
Replace five # type: ignore comments with setattr() calls (for module
attribute monkey-patching) and an Any return type (for _get_memory_engines),
eliminating all type suppressions per project zero-tolerance policy.
Fix ruff format violations: collapse @when decorator to one line, reformat
raise...from split, and reformat assert message wrapping.
ISSUES CLOSED: #7566
Add MEMORY_ENGINES_LOCK to engine_cache.py and wrap the check-and-set
in UnitOfWork.engine with it to prevent concurrent threads from creating
duplicate in-memory SQLite engines. Also fix cache-hit bug where
self._engine was never assigned on a cache hit.
Closes#7566
`_evaluate_dod` called `self._commit_plan(plan)` which does not exist,
causing an AttributeError at runtime for any plan with DoD criteria and
a Pyright error during typecheck. The correct public method is
`self.commit_plan(plan)` (defined at line 792), which persists the
updated plan when a UnitOfWork is wired and is a no-op in in-memory
mode — matching the pattern used in `apply_plan` and `execute_plan`.
ISSUES CLOSED: #7927
The test scenario 'validation_summary is populated after DoD evaluation'
was expecting 'total' and 'required_passed' keys which are not added by
the DoD evaluation. These keys are only present when the plan goes through
the Execute phase validation gate. The DoD evaluation only adds
'dod_evaluated', 'dod_all_passed', and 'dod' keys.
Updated the test to check for the 'dod' key instead, which correctly
reflects the structure of the validation_summary after DoD evaluation.
ISSUES CLOSED: #7927
Update integration test helpers to use definition_of_done values that
satisfy the new TextMatchEvaluator gate (criterion text must appear as
a substring of a plan argument key or value). Also fix _evaluate_dod
to merge DoD metadata into plan.validation_summary without overwriting
Execute-phase validation counts, preserving the coverage/tool-validation
gate used by apply_with_validation_gate. Update CONTRIBUTORS.md.
Fixes: wf02, wf04, wf06, wf07 integration test suites.
ISSUES CLOSED: #7927
Before transitioning to the Apply phase, PlanLifecycleService.apply_plan
now evaluates the plan's definition_of_done criteria using DoDEvaluator
(TextMatchEvaluator). If any required criteria fail, a DoDGatingError is
raised and the plan remains in Execute/COMPLETE state. The evaluation
result is stored in plan.validation_summary with dod_evaluated=True and
dod_all_passed reflecting the outcome. Plans with no DoD text skip
evaluation and proceed normally.
Adds DoDGatingError exception class and _evaluate_dod() helper method.
Adds BDD feature file and step definitions for DoD gating scenarios.
ISSUES CLOSED: #7927
The typing.cast import was placed after third-party imports (behave),
violating ruff/isort PEP 8 ordering enforced by CI lint. Move it into
the stdlib block before the blank-line separator from third-party.
Fixes CI / lint failure on this branch. Closes#8588
Fixes a race condition (TOCTOU) in ActorLoader.list_actors() where the namespace
filter was applied outside the threading lock, creating a window for concurrent
dictionary modifications. Moving the filter inside the with self._lock: block
ensures atomic reads and filtering.
- Move namespace filtering inside the RLock in list_actors()
- Add concurrency BDD test (threading.Barrier) for list_actors + clear race condition
- Update CHANGELOG.md with fix entry (closes#8588)
- Update CONTRIBUTORS.md with contribution details
ISSUES CLOSED: #8588
- Add missing @tdd_issue companion tag to @tdd_issue_7623 scenario;
before_scenario hook raised ValueError causing hook_error on every run
- Apply ruff format to validation_pipeline.py and
validation_pipeline_concurrency_steps.py (2 files reformatted)
- Add use_step_matcher("parse") reset at end of concurrency steps
to match convention in validation_pipeline_steps.py
ISSUES CLOSED: #7623
Address all 4 remaining blocking issues from PR #7811 review:
- Delete duplicate _validation_pipeline_mock.py in features/steps/
(the inline MockValidationExecutor is kept where it belongs)
- Replace # type: ignore[return-value] with cast(dict[str, Any], ...)
to satisfy the zero-type-suppressions policy for test additions
- Add @tdd_issue_7623 regression tag on concurrency scenario
- Remove unreachable dead-code RuntimeError guard in
_install_thread_local_streams() (constructors never return None)
All CI lint checks now pass. Source files remain within line limits.
The relative import from ._validation_pipeline_mock causes behave-parallel's
module loader to create globals without '__name__', raising a KeyError during
load_step_modules(). Restoring the inline definition resolves the crash.
Closes#7811
Updated docs/specification.md to document the correct JSON/YAML output structure
for the project delete command, replacing the legacy deletion_summary object with
the deleted, success, and deleted_at fields that match the actual implementation
introduced in PR #6639. Added BDD feature file and step definitions to validate
the output format.
ISSUES CLOSED: #7872
HAL9000
controller-ci-rerun
freemo
HAL9001
pr-merge-worker