Compare commits

..

14 Commits

Author SHA1 Message Date
HAL9000 8a3ead1046 feat: add structured conflict reporting for three-way merge (PR #11000)
Introduce ConflictMarkerRegion dataclass to capture detailed conflict
information (line ranges, ours snippet) during git three-way merges.
Updates MergeResult.conflicts and FileMergeOutcome.conflicts with
structured lists of ConflictMarkerRegion instead of raw tuples.
2026-05-16 10:44:24 +00:00
freemo 23d73e7fb2 chore: Fix ruff format violations in db_url_sanitisation_steps.py
CI / lint (push) Successful in 56s
CI / typecheck (push) Successful in 1m36s
CI / security (push) Successful in 1m37s
CI / helm (push) Successful in 51s
CI / push-validation (push) Successful in 48s
CI / build (push) Successful in 1m26s
CI / quality (push) Successful in 1m29s
CI / benchmark-regression (push) Failing after 1m47s
CI / e2e_tests (push) Successful in 1m48s
CI / integration_tests (push) Successful in 4m19s
CI / unit_tests (push) Successful in 6m8s
CI / docker (push) Successful in 1m23s
CI / coverage (push) Successful in 10m30s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (push) Successful in 1h26m6s
CI / helm (pull_request) Successful in 47s
CI / push-validation (pull_request) Successful in 47s
CI / build (pull_request) Successful in 1m34s
CI / lint (pull_request) Successful in 2m18s
CI / quality (pull_request) Successful in 2m18s
CI / typecheck (pull_request) Successful in 2m30s
CI / security (pull_request) Successful in 2m36s
CI / integration_tests (pull_request) Successful in 5m23s
CI / unit_tests (pull_request) Successful in 7m48s
CI / docker (pull_request) Successful in 1m34s
CI / coverage (pull_request) Successful in 10m45s
CI / status-check (pull_request) Successful in 3s
Fix formatting issues detected by CI lint check:
- Simplify multi-line decorator arguments to single line
- Simplify multi-line assertion error messages to single line

This resolves the format --check failure blocking CI.
2026-05-16 06:04:01 +00:00
HAL9000 86f96f299e fix(cli): Mask database URL credentials in agents info CLI output (#8395) 2026-05-16 06:04:01 +00:00
HAL9000 5c5309f35d fix(ci): add missing effective set count step definition
CI / lint (push) Successful in 50s
CI / benchmark-regression (push) Failing after 46s
CI / typecheck (push) Successful in 1m21s
CI / security (push) Successful in 1m20s
CI / quality (push) Successful in 1m3s
CI / build (push) Successful in 35s
CI / helm (push) Successful in 32s
CI / integration_tests (push) Successful in 3m22s
CI / push-validation (push) Successful in 33s
CI / e2e_tests (push) Successful in 55s
CI / unit_tests (push) Successful in 5m49s
CI / docker (push) Successful in 2m1s
CI / coverage (push) Successful in 13m13s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Failing after 24m24s
The Behave scenario at line 671 of consolidated_domain_models.feature
asserts 'the effective set should have {count:d} invariants' but no
step handler existed, causing UndefinedStepError and CI failure.

Adds the missing step: @then('the effective set should have {count:d} invariants')
to step_invariant_models_steps.py, mirroring the existing invariant-set count pattern.
2026-05-16 04:14:17 +00:00
HAL9000 f4cea72248 fix(ci): update all merge_invariants callers to use 4-param signature
The PR #11143 adds action_invariants as a 4th parameter to
merge_invariants() and InvariantSet.merge(), but two call sites
were not updated:

- benchmarks/invariant_merge_bench.py: 5 calls with 3 positional args
- robot/helper_m3_e2e_verification.py: 2 calls using keyword args

All call sites now pass action_invariants=[] for backward-compatible
empty-action behavior.
2026-05-16 04:14:17 +00:00
HAL9000 761622f746 fix(invariant): pass action_name in list_invariants when scope is ACTION
Fixes list_invariants(effective=True) to forward action_name to
get_effective_invariants when scope is ACTION, ensuring action-scoped
invariants are included in effective invariant lists.

Also applies ruff formatting to the return statement in get_effective_invariants.

Addresses reviewer observation about list_invariants gap.

Refs: #9126
2026-05-16 04:14:17 +00:00
freemo bf52a9c648 fix(invariant): restore ACTION scope in merge_invariants and InvariantSet.merge precedence chain
The 4-tier invariant precedence chain (plan > action > project > global) was
broken at the domain layer — merge_invariants() and InvariantSet.merge() only
accepted 3 parameters (plan, project, global), silently dropping all action-
scoped invariants. Added action_invariants as a fourth parameter with proper
backward compatibility (default to empty list). Updated module docstrings,
InvariantScope docstring, and InvariantService.get_effective_invariants() to
reflect the correct precedence chain. Added comprehensive BDD test scenarios
covering four-tier merge precedence, action-before-project ordering, and effective
invariant computation with all four scopes.

ISSUES CLOSED: #9126
2026-05-16 04:14:17 +00:00
CoreRasurae 97c1007bb5 feat(events): wire domain services to emit missing EventBus events
CI / push-validation (pull_request) Successful in 48s
CI / build (pull_request) Successful in 1m17s
CI / helm (pull_request) Successful in 1m0s
CI / lint (pull_request) Successful in 1m54s
CI / quality (pull_request) Successful in 1m55s
CI / security (pull_request) Successful in 2m5s
CI / typecheck (pull_request) Successful in 2m4s
CI / integration_tests (pull_request) Successful in 5m1s
CI / unit_tests (pull_request) Successful in 5m10s
CI / docker (pull_request) Successful in 1m51s
CI / coverage (pull_request) Successful in 12m47s
CI / status-check (pull_request) Successful in 8s
CI / benchmark-publish (push) Has started running
CI / benchmark-regression (push) Failing after 1m15s
CI / typecheck (push) Has started running
CI / security (push) Has started running
CI / quality (push) Has started running
CI / integration_tests (push) Has started running
CI / unit_tests (push) Has started running
CI / e2e_tests (push) Has started running
CI / lint (push) Successful in 56s
CI / helm (push) Successful in 53s
CI / push-validation (push) Successful in 54s
CI / build (push) Successful in 1m24s
CI / coverage (push) Blocked by required conditions
CI / docker (push) Blocked by required conditions
CI / status-check (push) Blocked by required conditions
Add TokenAuthMiddleware to emit AUTH_SUCCESS/AUTH_FAILURE with spec-aligned audit details and wire it through the DI container using server.token resolution.

Add Behave and Robot coverage for auth event emission and end-to-end audit persistence, and update audit subscriber producer notes and changelog.

ISSUES CLOSED: #714
2026-05-15 22:19:57 +01:00
hamza.khyari b0b28623a1 Merge pull request 'revert(acms): revert PR #11216 — _resolve_effective_budget reads wrong attribute' (#11228) from bugfix/m5-revert-acms-budget-assembler into master
CI / push-validation (push) Successful in 33s
CI / helm (push) Successful in 44s
CI / build (push) Successful in 1m13s
CI / lint (push) Successful in 1m21s
CI / quality (push) Successful in 1m43s
CI / e2e_tests (push) Successful in 1m46s
CI / typecheck (push) Successful in 1m56s
CI / security (push) Successful in 1m55s
CI / benchmark-regression (push) Failing after 1m30s
CI / integration_tests (push) Successful in 3m54s
CI / unit_tests (push) Successful in 6m50s
CI / docker (push) Successful in 1m29s
CI / coverage (push) Successful in 11m1s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 1h25m33s
CI / push-validation (pull_request) Successful in 48s
CI / helm (pull_request) Successful in 1m3s
CI / lint (pull_request) Successful in 1m48s
CI / build (pull_request) Successful in 1m34s
CI / typecheck (pull_request) Successful in 1m54s
CI / quality (pull_request) Successful in 1m47s
CI / security (pull_request) Successful in 2m2s
CI / integration_tests (pull_request) Failing after 21m0s
CI / unit_tests (pull_request) Failing after 21m2s
CI / coverage (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
Reviewed-on: #11228
Reviewed-by: Rui Hu <rui.hu@cleverthis.com>
2026-05-15 11:47:30 +00:00
hamza.khyari b4351ca78d Revert "Merge pull request 'fix(acms): use project-level hot_max_tokens in execute phase context assembly' (#11216) from bugfix/m5-acms-project-budget-override into master"
CI / push-validation (pull_request) Successful in 41s
CI / helm (pull_request) Successful in 50s
CI / build (pull_request) Successful in 1m13s
CI / lint (pull_request) Successful in 1m34s
CI / typecheck (pull_request) Successful in 1m40s
CI / quality (pull_request) Successful in 1m45s
CI / security (pull_request) Successful in 1m50s
CI / integration_tests (pull_request) Successful in 3m52s
CI / unit_tests (pull_request) Successful in 8m50s
CI / docker (pull_request) Successful in 1m39s
CI / coverage (pull_request) Successful in 12m1s
CI / status-check (pull_request) Successful in 6s
This reverts commit 94622f467c, reversing
changes made to 1196c726f2.
2026-05-15 11:15:41 +00:00
hamza.khyari 94622f467c Merge pull request 'fix(acms): use project-level hot_max_tokens in execute phase context assembly' (#11216) from bugfix/m5-acms-project-budget-override into master
CI / lint (push) Successful in 1m4s
CI / benchmark-publish (push) Has started running
CI / helm (push) Successful in 51s
CI / push-validation (push) Successful in 50s
CI / build (push) Successful in 1m13s
CI / quality (push) Successful in 1m29s
CI / benchmark-regression (push) Failing after 47s
CI / typecheck (push) Successful in 1m48s
CI / e2e_tests (push) Successful in 1m28s
CI / security (push) Successful in 2m7s
CI / integration_tests (push) Successful in 5m35s
CI / unit_tests (push) Successful in 5m39s
CI / docker (push) Successful in 1m38s
CI / coverage (push) Successful in 11m2s
CI / status-check (push) Successful in 2s
Reviewed-on: #11216
Reviewed-by: HAL9001 <hal9001@cleverthis.com>
2026-05-15 10:49:48 +00:00
hamza.khyari 1baa888659 fix(acms): use project-level hot_max_tokens in execute phase context assembly
CI / lint (pull_request) Successful in 57s
CI / typecheck (pull_request) Successful in 1m24s
CI / quality (pull_request) Successful in 1m16s
CI / build (pull_request) Successful in 40s
CI / security (pull_request) Successful in 1m28s
CI / integration_tests (pull_request) Successful in 6m2s
CI / helm (pull_request) Successful in 24s
CI / push-validation (pull_request) Successful in 28s
CI / unit_tests (pull_request) Successful in 7m13s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 11m50s
CI / status-check (pull_request) Successful in 7s
Added _resolve_effective_budget() method to
ACMSExecutePhaseContextAssembler that reads each linked project's
settings.hot_max_tokens and uses the maximum override value as the
pipeline budget instead of the hardcoded global default.

Updated assemble() to use the resolved effective budget for both
CoreContextBudget and ContextRequest.

Added Behave regression scenario with @tdd_issue @tdd_issue_11035 tags
verifying the pipeline receives the project-level budget.

Removed test artifact ANALYSIS.md left over from scenario 20.

Fixes: #11035
Fixes: #11215
2026-05-15 10:27:57 +00:00
HAL9000 1196c726f2 fix(lint): remove broken step file and fix line-length violation
CI / helm (pull_request) Successful in 56s
CI / push-validation (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 1m59s
CI / lint (pull_request) Successful in 2m21s
CI / typecheck (pull_request) Successful in 2m49s
CI / quality (pull_request) Successful in 2m56s
CI / security (pull_request) Successful in 3m21s
CI / integration_tests (pull_request) Successful in 5m36s
CI / unit_tests (pull_request) Successful in 9m7s
CI / docker (pull_request) Successful in 1m49s
CI / coverage (pull_request) Successful in 16m22s
CI / status-check (pull_request) Successful in 4s
CI / build (push) Successful in 1m7s
CI / helm (push) Successful in 39s
CI / push-validation (push) Successful in 35s
CI / lint (push) Successful in 1m36s
CI / typecheck (push) Successful in 1m51s
CI / quality (push) Successful in 1m55s
CI / security (push) Successful in 2m8s
CI / e2e_tests (push) Successful in 1m7s
CI / benchmark-regression (push) Failing after 1m22s
CI / integration_tests (push) Failing after 17m27s
CI / unit_tests (push) Failing after 17m29s
CI / benchmark-publish (push) Successful in 1h37m6s
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
- Removed features/steps/plan_status_json_envelope_steps.py which contained
  invalid Python syntax (dummy content, not referenced by any feature)
- Fixed line-length violation in execute_phase_context_assembler.py
  (CoreContextBudget constructor call exceeded 88-char limit)
2026-05-15 03:41:13 +00:00
HAL9000 655cd7ebc2 fix: persist strategy decisions via DecisionService during strategize (#10813)
The plan tree command reported zero decision nodes after strategize because
PlanExecutor.run_strategize() never persisted strategy decisions as domain
Decision objects. Added _persist_strategy_decisions() to the PlanExecutor,
wired decision_service from the DI container in _get_plan_executor(), and
ensure each strategy decision is recorded with correct DecisionType mapping.

ISSUES CLOSED: #10813
2026-05-15 03:41:13 +00:00
30 changed files with 1224 additions and 274 deletions
+19 -12
View File
@@ -5,6 +5,8 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
Changed `wf10_batch.robot` to be less likely to create files, and
`plan_generation_graph.robot` to give more test answers.
## [Unreleased]
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
requires whole-word closing keywords (avoids false positives like
"prefixes #12"), TDD bug tag discovery now uses exact token matching
@@ -40,6 +42,20 @@ Changed `wf10_batch.robot` to be less likely to create files, and
cleanup conflict with `cli_init_yes_flag_steps.py`); 2 new Behave
scenarios covering multi-line PR description parsing and non-string
`pr_diff` type guard.
- Added `TokenAuthMiddleware` and DI wiring to emit missing auth domain
events (`AUTH_SUCCESS`, `AUTH_FAILURE`) during token checks, including
spec-aligned audit details (`user_identity`, `attempted_identity`,
`ip_address`, `token_prefix`, `failure_reason`) and best-effort publish
behavior. Auth token prefixes now persist to audit logs without
redaction-key collisions, and short tokens are masked (`***...`) to
prevent full token disclosure. Added Behave coverage and Robot
audit-pipeline integration tests for auth event persistence. (#714)
- Added TDD bug-capture E2E tests for bug #1028 — ACMS indexing pipeline not
wired into CLI. Four Robot Framework E2E tests prove ContextTierService starts
empty on every CLI invocation. Tests use ``@tdd_expected_fail`` until the bug
fix is merged. (#1029)
- Added Fix-then-Revalidate orchestration loop for required validations:
bounded retry with configurable limits (0--100 per Safety Profile),
strategy revision escalation via `auto_strategy_revision` float
@@ -52,7 +68,6 @@ Changed `wf10_batch.robot` to be less likely to create files, and
counter, spec-required `validation_summary` and
`final_validation_results` fields on the result model, DI container
## [Unreleased]
- **fix(tui): rename ActorSelectionOverlay._render to _refresh_display (issue #11039)** — `ActorSelectionOverlay._render()` shadows Textual's `Widget._render()` which must return a `Strip`. In textual >=1.0, layout calls `get_content_height()` `self._render()` gets `None` `AttributeError: 'NoneType' object has no attribute 'get_height'`. Renamed the method to `_refresh_display()` and updated all four internal call sites (`show()`, `move_up()`, `move_down()`, `set_search()`) to use the new name.
- **Structural Component Output Validation** (#8164): Replaces exact character matching with structural component checking for output validation. Implements three validators covering plan tree output, decision CLI dicts, and structured session snapshots. The `validate_plan_tree` function validates node dicts for required keys (`decision_id`, `type`, `sequence`, `question`, `children`), ULID format, correct types, and sibling ordering. The `validate_decision_dict` function validates decision CLI output against the `Decision.as_cli_dict()` schema with field presence, type, ULID pattern, confidence range [0..1], and boolean field checks. The `validate_structured_output` function validates the StructuredOutput envelope for `command`, `session_id` (ULID), status membership, `exit_code`, and elements integrity. A unified dispatcher (`validate_structured_component_output`) enables routing by target_type. BDD test coverage added in `features/structural_validation.feature`. [Epic #8137](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8137)
@@ -64,6 +79,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
when separate `provider`/`model` keys are absent. Added validation to reject malformed
combined values with empty provider or model halves.
- **Fixed plan tree reporting zero decision nodes after strategize** (#10813): The `plan tree` command showed no ``decision_id`` fields even though planning completed successfully. Root cause: the PlanExecutor's ``run_strategize()`` method produced strategy decisions via the StrategyActor but never persisted them as domain ``Decision`` objects through the DecisionService wiring. Added ``decision_service`` parameter to the PlanExecutor constructor, wired it from the CLI dependency-injection container in ``_get_plan_executor()``, and added ``_persist_strategy_decisions()`` that converts each strategy decision into a domain Decision with correct type mapping (prompt_definition, strategy_choice, subplan_spawn) so they appear in plan tree output.
- **`task-implementor` posts work-started notification comments** (#11031): Both
the `issue_impl` and `pr_fix` procedures now post an informational "work
started" comment to the Forgejo issue/PR before beginning implementation.
@@ -185,7 +201,8 @@ Changed `wf10_batch.robot` to be less likely to create files, and
were cleaned up in `features/actor_add_update_enforcement.feature` so the
tests report correctly now that the underlying bug has been fixed.
- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
- **Fixed `merge_invariants()` missing ACTION scope — 4-tier precedence restored** (#9126): Updated ``merge_invariants()`` and ``InvariantSet.merge()`` to accept a fourth parameter ``action_invariants`` alongside plan, project, and global tiers. The module docstring, ``InvariantScope`` docstring, and ``InvariantService.get_effective_invariants()`` now all reflect the correct precedence chain: ``plan > action > project > global``. Added ``action_name`` parameter to ``get_effective_invariants()`` so action-scoped invariants are collected and passed through the merge pipeline instead of silently dropped. All docstrings across both files were corrected from ``plan > project > global`` to ``plan > action > project > global``. Comprehensive Behave scenarios added covering four-tier merge precedence, action-before-project ordering, action override of project with same text, and effective invariant computation with all four scopes.
step texts to avoid case-sensitive collisions between different step modules that
prevented all Behave tests from loading. Renamed steps in
`edge_case_plan_steps.py`, `plan_executor_coverage_boost_steps.py`,
@@ -272,16 +289,6 @@ Changed `wf10_batch.robot` to be less likely to create files, and
versions (<3.13.4) cannot be installed even if upstream transitive dependencies have loose
version constraints.
- **Path containment checks replaced `startswith` with semantic path comparison** (#7478, #7801):
Replaced insecure string-prefix path containment (`str.startswith(root + "/")`) with
semantics-aware ``os.path.relpath`` checks in ``tool/path_mapper.py`` and
``application/services/llm_actors.py``. The previous ``startswith`` approach was
vulnerable to sibling-directory prefix-collision attacks where a sandbox root of
``/tmp/sandbox`` would incorrectly allow access to ``/tmp/sandboxmalicious/path``.
All path containment checks now use ``os.path.relpath`` or
``Path.is_relative_to()`` for safe, semantic containment verification (as mandated
by the security specification).
### Fixed
- **Error suppression removed from `reactive_registry_adapter.py`** (#9060): Removed two `try...except Exception:` blocks in `register_registry_agents()` that silently suppressed errors, violating the CONTRIBUTING.md fail-fast policy. Exceptions from `actor_registry.list_actors()` and the route bridge refresh now propagate to the caller instead of being swallowed. Added Behave scenarios verifying RuntimeError, AttributeError, and TypeError propagation.
+3 -2
View File
@@ -8,13 +8,14 @@
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
* Luis Mendes <luis.p.mendes@gmail.com>
* Rui Hu <rui.hu@cleverthis.com>
* HAL 9000 <hal9000@cleverthis.com> has contributed fix for #10813 — wiring DecisionService into PlanExecutor for strategy decision persistence during strategize.
# Details
Below are some of the specific details of various contributions.
* Jeffrey Phillips Freeman has acted as Lead Developer, daily contributor, and Project Owner.
* Jeffrey Phillips Freeman has contributed an implementation for the invariant propagation fix (PR #10881 / issue #9131): added `_propagate_invariant_decisions()` to `SubplanService` to propagate all `invariant_enforced` decisions from parent plans to child plan decision trees during subplan spawn, satisfying the specification requirement for invariant propagation across hierarchical plan execution.
* Jeffrey Phillips Freeman has contributed the invariant merge precedence fix (#9126): restored the missing ACTION scope in ``merge_invariants()`` and ``InvariantSet.merge()``, corrected all module docstrings from ``plan > project > global`` to the spec-compliant ``plan > action > project > global``, and added comprehensive BDD test coverage for four-tier merge precedence.
* Brent E. Edwards has contributed quality assurance, test coverage, and CI pipeline improvements.
* HAL 9000 has contributed automated implementation, bug fixes, and feature development as part of the CleverAgents automation pool.
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
@@ -44,6 +45,6 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
* HAL 9000 has contributed the `ActorSelectionOverlay._render``_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
* HAL 9000 has contributed the path containment security hardening fix (PR #7801 / issue #7478): replaced insecure ``str.startswith(root + "/")`` string-prefix path containment checks with semantic ``os.path.relpath`` comparisons in ``tool/path_mapper.py`` (_is_under) and ``application/services/llm_actors.py`` (_write_to_sandbox), eliminating the sibling-directory prefix-collision path traversal bypass vulnerability.
+5 -5
View File
@@ -74,7 +74,7 @@ class MergeSmallSuite:
def time_merge_small(self) -> None:
"""Benchmark merge with ~13 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeMediumSuite:
@@ -88,7 +88,7 @@ class MergeMediumSuite:
def time_merge_medium(self) -> None:
"""Benchmark merge with ~50 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeLargeSuite:
@@ -102,7 +102,7 @@ class MergeLargeSuite:
def time_merge_large(self) -> None:
"""Benchmark merge with ~250 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeDeduplicationSuite:
@@ -137,7 +137,7 @@ class MergeDeduplicationSuite:
def time_merge_dedup(self) -> None:
"""Benchmark merge with 60 invariants, all duplicates."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class InvariantSetMergeSuite:
@@ -151,7 +151,7 @@ class InvariantSetMergeSuite:
def time_invariant_set_merge(self) -> None:
"""Benchmark InvariantSet.merge()."""
InvariantSet.merge(self.plan, self.project, self.global_invs)
InvariantSet.merge(self.plan, [], self.project, self.global_invs)
class ServiceEffectiveSuite:
+63
View File
@@ -0,0 +1,63 @@
Feature: Auth middleware emits security events
As a platform operator
I want authentication outcomes emitted as domain events
So the audit pipeline captures auth success and failure details
Scenario: Successful authentication emits AUTH_SUCCESS
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I authenticate with token "tok_secret_123" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be true
And the auth event bus should have exactly 1 event
And the latest auth event type should be AUTH_SUCCESS
And the latest auth event should contain detail "user_identity" with value "alice@example.com"
And the latest auth event should contain detail "ip_address" with value "10.0.0.5"
And the latest auth event should contain detail "token_prefix" with value "tok_se..."
Scenario: Short token success masks token_prefix
Given an auth middleware with expected token "short"
And an auth recording event bus
When I authenticate with token "short" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be true
And the latest auth event type should be AUTH_SUCCESS
And the latest auth event should contain detail "token_prefix" with value "***..."
Scenario: Failed authentication emits AUTH_FAILURE
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I authenticate with token "tok_wrong_999" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be false
And the auth event bus should have exactly 1 event
And the latest auth event type should be AUTH_FAILURE
And the latest auth event should contain detail "attempted_identity" with value "alice@example.com"
And the latest auth event should contain detail "failure_reason" with value "invalid_token"
And the latest auth event should contain detail "token_prefix" with value "tok_wr..."
Scenario: Missing configured token emits AUTH_FAILURE
Given an auth middleware with no configured token
And an auth recording event bus
When I authenticate with token "tok_any_111" identity "bob@example.com" and ip "10.0.0.9"
Then the auth result should be false
And the latest auth event type should be AUTH_FAILURE
And the latest auth event should contain detail "failure_reason" with value "token_not_configured"
Scenario: Empty authentication token is rejected before emission
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I try to authenticate with an empty token
Then an auth ValueError should be raised
And the auth event bus should have exactly 0 event
Scenario: Auth middleware events persist through AuditEventSubscriber
Given auth middleware is wired to AuditEventSubscriber with expected token "tok_secret_123"
When I authenticate with token "tok_secret_123" identity "carol@example.com" and ip "10.1.2.3"
And I authenticate with token "tok_bad_123" identity "dave@example.com" and ip "10.1.2.4"
Then the auth result history should be "true,false"
And the audit log should contain 1 auth_success entry from middleware
And the latest auth_success audit entry should contain detail "ip_address" with value "10.1.2.3"
And the latest auth_success audit entry should contain detail "token_prefix" with value "tok_se..."
And the audit log should contain 1 auth_failure entry from middleware
And the latest auth_failure audit entry should contain detail "ip_address" with value "10.1.2.4"
And the latest auth_failure audit entry should contain detail "attempted_identity" with value "dave@example.com"
And the latest auth_failure audit entry should contain detail "failure_reason" with value "invalid_token"
And the latest auth_failure audit entry should contain detail "token_prefix" with value "tok_ba..."
+100 -3
View File
@@ -418,7 +418,7 @@ Feature: Consolidated Domain Models
Scenario: InvariantScope has four values
Then InvariantScope should have values "global, project, action, plan"
# === Merge Precedence (plan > project > global) ===
# === Merge Precedence (plan > action > project > global) ===
Scenario: Merge with no duplicates preserves all invariants
@@ -480,6 +480,75 @@ Feature: Consolidated Domain Models
And the merged invariant at index 0 should have text "LOG ALL CHANGES"
Scenario: Merge with all four scopes preserves action tier — Issue #9126
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
And I have action invariants
| text | source |
| Action rule | action1 |
And I have project invariants
| text | source |
| Project rule | proj1 |
And I have global invariants
| text | source |
| Global rule | system |
When I merge the invariants
Then the merged set should have 4 invariants
And plan invariants appear before action invariants in merge
And action invariants appear before project invariants in merge
And the merged invariant at index 0 should have text "Plan rule"
And the merged invariant at index 1 should have text "Action rule"
And the merged invariant at index 2 should have text "Project rule"
And the merged invariant at index 3 should have text "Global rule"
Scenario: Action invariant overrides project with same text
Given I have plan invariants
| text | source |
And I have action invariants
| text | source |
| Log all changes | action1 |
And I have project invariants
| text | source |
| Log all changes | proj1 |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "action"
Scenario: Action invariant is included in de-duplication with plan override
Given I have plan invariants
| text | source |
| Log all changes | plan1 |
And I have action invariants
| text | source |
| Log all changes | action1 |
And I have project invariants
| text | source |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "plan"
Scenario: Inactive action invariants are excluded from merge
Given I have plan invariants with an inactive entry
And I have action invariants
| text | source |
| Active | action1 |
And I have project invariants
| text | source |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "action"
Scenario: Inactive invariants are excluded from merge
Given I have plan invariants with an inactive entry
And I have project invariants
@@ -492,7 +561,7 @@ Feature: Consolidated Domain Models
# === InvariantSet merge class method ===
Scenario: InvariantSet.merge produces correct result
Scenario: InvariantSet.merge produces correct result
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
@@ -505,7 +574,25 @@ Feature: Consolidated Domain Models
When I merge using InvariantSet
Then the invariant set should have 3 invariants
# === Service: Add/List/Remove ===
Scenario: InvariantSet.merge with four-tier precedence — Issue #9126
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
And I have action invariants
| text | source |
| Action rule | action1 |
And I have project invariants
| text | source |
| Project rule | proj1 |
And I have global invariants
| text | source |
| Global rule | system |
When I merge using InvariantSet
Then the invariant set should have 4 invariants
# === Service: Add/List/Remove ===
Scenario: Service add invariant
@@ -574,6 +661,16 @@ Feature: Consolidated Domain Models
And the effective set should contain global invariants last
Scenario: Effective invariants include action scope — Issue #9126
Given an invariant service with invariants at all four scopes
When I get effective invariants for plan "plan1", action "action1", and project "proj1"
Then the effective set should contain plan invariants first
And the effective set should contain action invariants second
And the effective set should contain project invariants after action
And the effective set should contain global invariants last
And the effective set should have 4 invariants
Scenario: Effective invariants de-duplicate across scopes
Given an invariant service with duplicate text across scopes
When I get effective invariants for plan "plan1" and project "proj1"
+53
View File
@@ -0,0 +1,53 @@
Feature: Database URL sanitisation — credentials must never be exposed
As a CleverAgents operator
I want database URLs in CLI output to have their credentials masked
So that running ``agents info`` never leaks passwords or API tokens
@tdd_bug_8395
Scenario Outline: PostgreSQL URL with user and password — all are masked
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| postgresql://user:secret@localhost/mydb | postgresql://***:***@localhost/mydb |
| postgresql://admin:p%40ssw0rd@db.example.com:5432/agents | postgresql://***:***@db.example.com:5432/agents |
| postgresql://readonly:r0ad_only@pg.cluster.internal:5433/production | postgresql://***:***@pg.cluster.internal:5433/production |
Scenario Outline: MySQL URL with credentials — all are masked
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| mysql://app:s3cret@mysql.internal:3306/agents | mysql://***:***@mysql.internal:3306/agents |
| mysql+pymysql://root:toor@localhost/testdb | mysql+pymysql://***:***@localhost/testdb |
Scenario Outline: SQLite URLs — remain unchanged (no credentials)
Given the database url is "<url>"
When I run sanitise_db_url
Then the sanitised database url should be "<expected>"
Examples:
| url | expected |
| sqlite:///data/cleveragents.db | sqlite:///data/cleveragents.db |
| sqlite:////absolute/path/to/db.sqlite | sqlite:////absolute/path/to/db.sqlite |
| memory | memory |
Scenario: Username-only URL — password is still masked
Given the database url is "postgres://deploy@db.example.com/prod"
When I run sanitise_db_url
Then the sanitised database url should be "postgres://***:***@db.example.com/prod"
Scenario: SQLite URL without credentials remains unchanged
Given the database url is "sqlite:///test.db"
When I run sanitise_db_url
Then the sanitised database url should be "sqlite:///test.db"
Scenario: build_info_data returns sanitised database URL
Given a mock settings object with database url "postgresql://admin:supersecret@host.example.com:5432/mydb"
When I call build_info_data
Then the database field in info data should be "postgresql://***:***@host.example.com:5432/mydb"
+2
View File
@@ -9,6 +9,7 @@ from .fake_provider import FakeProviderInfo, FakeProviderRegistry
from .lsp_transport_mock import MockLspTransport, parse_lsp_responses
from .mock_ai_provider import MockAIProvider
from .mock_mcp_transport import MockMCPTransport
from .recording_event_bus import RecordingEventBus
from .transient_fail_audit_service import TransientFailAuditService
# NOTE: tdd_test_helpers is NOT re-exported here because it imports
@@ -22,6 +23,7 @@ __all__ = [
"MockAIProvider",
"MockLspTransport",
"MockMCPTransport",
"RecordingEventBus",
"TransientFailAuditService",
"parse_lsp_responses",
]
+25
View File
@@ -0,0 +1,25 @@
"""Minimal in-memory event bus used for Behave assertions."""
from __future__ import annotations
from collections.abc import Callable
from cleveragents.infrastructure.events.models import DomainEvent
from cleveragents.infrastructure.events.types import EventType
class RecordingEventBus:
"""Minimal in-memory event bus used for Behave assertions."""
def __init__(self) -> None:
self.events: list[DomainEvent] = []
def emit(self, event: DomainEvent) -> None:
self.events.append(event)
def subscribe(
self,
event_type: EventType,
handler: Callable[[DomainEvent], None],
) -> None:
_ = (event_type, handler)
@@ -1,48 +0,0 @@
Feature: Path containment startswith bypass prevention (issue #7478 / PR #7801)
AS a security engineer
I WANT path containment checks to use semantic comparison instead of string prefix matching
SO THAT sibling-directory prefix-collision attacks cannot bypass sandbox isolation
Background:
Given a temporary sandbox directory "/tmp/sandbox"
And a file "safe.txt" with content "safe content"
# ---- PathMapper _is_under prefix collision prevention (#7478) ----
@tdd_issue @tdd_issue_7478
Scenario: PathMapper rejects sibling-prefix path traversal via relpath containment
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
And a sibling directory with a name that is a prefix of the sandbox root
When I check whether host path is safe from prefix collision
Then the prefix collision check should return ``False``
@tdd_issue @tdd_issue_7478
Scenario: PathMapper correctly identifies legitimate child paths
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
When I check whether "/tmp/sandbox/src/main.py" is a host path
Then the host containment result should be true
@tdd_issue @tdd_issue_7478
Scenario: PathMapper correctly identifies root equality
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
When I check whether "/tmp/sandbox" is a host path
Then the host containment result should be true
@tdd_issue @tdd_issue_7478
Scenario: PathMapper rejects sibling-prefix escape path
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
And a file "escape-evil.txt" with content "malicious" at "/tmp/sandbox-escape/escape-evil.txt"
When I check whether "/tmp/sandbox-escape/escape-evil.txt" is a host path
Then the host containment result should be false
@tdd_issue @tdd_issue_7478
Scenario: PathMapper maps root path exactly (no relative component)
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
When I map the host path "/tmp/sandbox" to container
Then the mapped path should be "/workspace"
@tdd_issue @tdd_issue_7478
Scenario: PathMapper maps a child path correctly through relpath
Given I have a PathMapper with host_root "/tmp/sandbox" and container_root "/workspace"
When I map the host path "/tmp/sandbox/src/main.py" to container
Then the mapped path should be "/workspace/src/main.py"
@@ -0,0 +1,165 @@
"""Step definitions for auth_middleware_events.feature."""
from __future__ import annotations
from behave import given, then, when
from behave.runner import Context
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from cleveragents.application.services.audit_event_subscriber import (
AuditEventSubscriber,
)
from cleveragents.application.services.audit_service import AuditService
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
from cleveragents.config.settings import Settings
from cleveragents.infrastructure.database.models import Base
from cleveragents.infrastructure.events.reactive import ReactiveEventBus
from cleveragents.infrastructure.events.types import EventType
from features.mocks.recording_event_bus import RecordingEventBus
def _fresh_audit_service() -> AuditService:
Settings._instance = None
settings = Settings(database_url="sqlite:///:memory:")
engine = create_engine("sqlite:///:memory:", echo=False)
Base.metadata.create_all(engine)
session = sessionmaker(bind=engine)()
return AuditService(settings=settings, session=session)
@given('an auth middleware with expected token "{token}"')
def step_auth_middleware_with_token(context: Context, token: str) -> None:
context.expected_token = token
@given("an auth middleware with no configured token")
def step_auth_middleware_no_token(context: Context) -> None:
context.expected_token = None
@given("an auth recording event bus")
def step_auth_recording_event_bus(context: Context) -> None:
context.event_bus = RecordingEventBus()
context.auth_middleware = TokenAuthMiddleware(
expected_token=context.expected_token,
event_bus=context.event_bus,
)
context.auth_result_history = []
@given('auth middleware is wired to AuditEventSubscriber with expected token "{token}"')
def step_auth_middleware_audit_pipeline(context: Context, token: str) -> None:
context.audit_service = _fresh_audit_service()
context.event_bus = ReactiveEventBus()
context.audit_subscriber = AuditEventSubscriber(
audit_service=context.audit_service,
event_bus=context.event_bus,
)
context.auth_middleware = TokenAuthMiddleware(
expected_token=token,
event_bus=context.event_bus,
)
context.auth_result_history = []
@when('I authenticate with token "{token}" identity "{identity}" and ip "{ip_address}"')
def step_authenticate(
context: Context, token: str, identity: str, ip_address: str
) -> None:
context.auth_result = context.auth_middleware.authenticate(
token,
identity=identity,
ip_address=ip_address,
)
context.auth_result_history.append(context.auth_result)
@when("I try to authenticate with an empty token")
def step_try_authenticate_empty_token(context: Context) -> None:
context.auth_error = None
try:
context.auth_middleware.authenticate("", identity="empty@example.com")
except ValueError as exc:
context.auth_error = exc
@then("the auth result should be true")
def step_auth_result_true(context: Context) -> None:
assert context.auth_result is True
@then("the auth result should be false")
def step_auth_result_false(context: Context) -> None:
assert context.auth_result is False
@then("the auth event bus should have exactly {count:d} event")
def step_auth_event_count(context: Context, count: int) -> None:
assert len(context.event_bus.events) == count, (
f"Expected {count} events, got {len(context.event_bus.events)}"
)
@then("the latest auth event type should be {event_type_name}")
def step_latest_auth_event_type(context: Context, event_type_name: str) -> None:
expected = EventType[event_type_name]
actual = context.event_bus.events[-1].event_type
assert actual == expected, f"Expected event type {expected}, got {actual}"
@then(
'the latest auth event should contain detail "{key}" with value "{expected_value}"'
)
def step_latest_auth_event_detail(
context: Context, key: str, expected_value: str
) -> None:
details = context.event_bus.events[-1].details
assert details.get(key) == expected_value, (
f"Expected {key}={expected_value!r}, got {details.get(key)!r}"
)
@then("an auth ValueError should be raised")
def step_auth_value_error(context: Context) -> None:
assert context.auth_error is not None, (
"Expected ValueError but no exception was raised"
)
assert isinstance(context.auth_error, ValueError), (
f"Expected ValueError, got {type(context.auth_error).__name__}"
)
@then('the auth result history should be "{history_csv}"')
def step_auth_result_history(context: Context, history_csv: str) -> None:
expected = [part.strip().lower() == "true" for part in history_csv.split(",")]
assert context.auth_result_history == expected, (
f"Expected history {expected}, got {context.auth_result_history}"
)
@then("the audit log should contain 1 auth_success entry from middleware")
def step_audit_has_auth_success(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="auth_success")
assert len(entries) == 1, f"Expected 1 auth_success entry, got {len(entries)}"
@then("the audit log should contain 1 auth_failure entry from middleware")
def step_audit_has_auth_failure(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="auth_failure")
assert len(entries) == 1, f"Expected 1 auth_failure entry, got {len(entries)}"
@then(
'the latest {audit_event_type} audit entry should contain detail "{key}" with value "{expected_value}"'
)
def step_latest_audit_entry_detail(
context: Context, audit_event_type: str, key: str, expected_value: str
) -> None:
entries = context.audit_service.list_entries(event_type=audit_event_type)
assert entries, f"Expected at least one {audit_event_type} audit entry"
details = entries[0].details
assert details.get(key) == expected_value, (
f"Expected {audit_event_type}.{key}={expected_value!r}, "
f"got {details.get(key)!r}"
)
-137
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
import io
import json
import os
import shutil
import tempfile
from pathlib import Path
@@ -71,18 +70,6 @@ def step_have_container_exec_module(context: Any) -> None:
# ---------------------------------------------------------------------------
@given('a temporary sandbox directory "{path}"')
def step_given_named_sandbox(context: Any, path: str) -> None:
"""Create the named directory as a temporary sandbox root for path tests."""
os.makedirs(path, exist_ok=True)
if not hasattr(context, "_cleanup_handlers"):
context._cleanup_handlers = []
context._cleanup_handlers.append(
lambda p=path: shutil.rmtree(p, ignore_errors=True)
)
context.sandbox_dir = path
@given(
'I have a PathMapper with host_root "{host_root}" and container_root "{container_root}"'
)
@@ -95,12 +82,6 @@ def step_map_host_to_container(context: Any, path: str) -> None:
context.mapped_path = context.path_mapper.host_to_container(path)
@when('I map the host path "{path}" to container')
def step_map_the_host_to_container(context: Any, path: str) -> None:
"""Alias for step_map_host_to_container (used in path containment security tests)."""
context.mapped_path = context.path_mapper.host_to_container(path)
@when('I map container path "{path}" to host')
def step_map_container_to_host(context: Any, path: str) -> None:
context.mapped_path = context.path_mapper.container_to_host(path)
@@ -113,14 +94,6 @@ def step_check_container_path(context: Any, expected: str) -> None:
)
@then('the mapped path should be "{expected}"')
def step_check_mapped_path(context: Any, expected: str) -> None:
"""Assert that the most recently mapped path equals expected."""
assert context.mapped_path == expected, (
f"Expected {expected!r}, got {context.mapped_path!r}"
)
@then('the host path should be "{expected}"')
def step_check_host_path(context: Any, expected: str) -> None:
assert context.mapped_path == expected, (
@@ -154,116 +127,6 @@ def step_is_not_container_path(context: Any, path: str) -> None:
)
# ---------------------------------------------------------------------------
# PathMapper prefix-collision security tests (#7478)
# ---------------------------------------------------------------------------
@given("a sibling directory with a name that is a prefix of the sandbox root")
def step_sibling_prefix_dir(context: Any) -> None:
"""Create a sibling directory whose name starts with the sandbox root name.
For example, if the sandbox host_root is ``/tmp/sandbox``, this creates
``/tmp/sandbox-escape``. This exercises the startswith prefix-collision
path traversal bypass (issue #7478).
"""
path_mapper = getattr(context, "path_mapper", None)
assert path_mapper is not None, "PathMapper not initialized in context"
host_root: str = path_mapper.host_root
root_parent = str(Path(host_root).parent)
sibling_name = Path(host_root).name + "-escape"
sibling_path = Path(root_parent) / sibling_name
sibling_path.mkdir(parents=True, exist_ok=True)
context._cleanup_handlers.append(
lambda: shutil.rmtree(str(sibling_path), ignore_errors=True)
)
@when("I check whether host path is safe from prefix collision")
def step_check_prefix_collision(context: Any) -> None:
"""Check whether a sibling-prefix path incorrectly passes containment."""
path_mapper = getattr(context, "path_mapper", None)
assert path_mapper is not None, "PathMapper not initialized in context"
host_root: str = path_mapper.host_root
root_parent = str(Path(host_root).parent)
root_name = Path(host_root).name
escape_root = f"{root_name}-escape"
escape_path = os.path.join(root_parent, escape_root, "evil.txt")
context.prefix_collision_result = path_mapper.is_host_path(escape_path)
# Store the escape path for use in the error message
context.escape_path = escape_path
@then("the prefix collision check should return ``False``")
def step_prefix_collision_rejected(context: Any) -> None:
result: bool = getattr(context, "prefix_collision_result", None)
assert result is False, (
f"Expected is_host_path({getattr(context, 'escape_path', 'unknown')!r}) to be False "
f"but got True — prefix collision bypass not prevented!"
)
@then('"{path}" should result be false for host path containment')
def step_host_path_result_false(context: Any, path: str) -> None:
"""Generalized step to assert is_host_path returns False for a given path."""
result = context.path_mapper.is_host_path(path)
assert result is False, (
f"Expected is_host_path({path!r}) to be False but got {result}"
)
@then('"{path}" should result be true for host path containment')
def step_host_path_result_true(context: Any, path: str) -> None:
"""Generalized step to assert is_host_path returns True for a given path."""
result = context.path_mapper.is_host_path(path)
assert result is True, (
f"Expected is_host_path({path!r}) to be True but got {result}"
)
@given(
'a file "{name}" with content "{content}" at "{absolute_path}"'
)
def step_create_file_at_absolute_path(
context: Any, name: str, content: str, absolute_path: str
) -> None:
"""Create a file at an arbitrary absolute path (used for sibling-prefix tests)."""
abs_path = Path(absolute_path)
abs_path.parent.mkdir(parents=True, exist_ok=True)
abs_path.write_text(content)
# ---------------------------------------------------------------------------
# Generic result assertions (for prefix collision test scenarios)
# ---------------------------------------------------------------------------
@when('I check whether "{path}" is a host path')
def step_check_host_path(context: Any, path: str) -> None:
"""Store the result of is_host_path for later assertion."""
path_mapper = getattr(context, "path_mapper", None)
assert path_mapper is not None, "PathMapper not initialized in context"
context._host_path_result = path_mapper.is_host_path(path)
@then("the host containment result should be true")
def step_result_is_true(context: Any) -> None:
"""Assert that the most recent host path containment check returned True."""
result: bool = getattr(context, "_host_path_result", False)
assert result is True, (
f"Expected host path check to return True but got {result}"
)
@then("the host containment result should be false")
def step_result_is_false(context: Any) -> None:
"""Assert that the most recent host path containment check returned False."""
result: bool = getattr(context, "_host_path_result", True)
assert result is False, (
f"Expected host path check to return False but got {result}"
)
# ---------------------------------------------------------------------------
# ContainerConfig
# ---------------------------------------------------------------------------
@@ -0,0 +1,85 @@
"""Step definitions for db_url_sanitisation.feature.
Tests the ``_sanitise_db_url`` helper and its use in ``build_info_data``.
"""
from __future__ import annotations
from pathlib import Path
from tempfile import mkdtemp
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
# ---------------------------------------------------------------------------
# Given
# ---------------------------------------------------------------------------
@given('the database url is "{url}"')
def step_db_url_given(context: Context, url: str) -> None:
"""Store the raw database URL for processing."""
context.raw_db_url = url
@given('a mock settings object with database url "{db_url}"')
def step_mock_settings_with_db_url(context: Context, db_url: str) -> None:
"""Build a full mock Settings that ``build_info_data`` expects."""
tmpdir = mkdtemp()
mock_settings = MagicMock()
mock_settings.database_url = db_url
mock_settings.data_dir = Path(tmpdir)
mock_settings.storage_path = Path(tmpdir)
mock_settings.config_path = Path(tmpdir) / "config.toml"
mock_settings.log_dir = Path(tmpdir) / "logs"
mock_settings.default_automation_profile = "auto"
mock_settings.has_provider_configured = MagicMock(return_value=False)
mock_settings.configured_provider_names = MagicMock(return_value=[])
mock_settings.debug_enabled = False
context.mock_settings = mock_settings
# ---------------------------------------------------------------------------
# When
# ---------------------------------------------------------------------------
@when("I run sanitise_db_url")
def step_run_sanitise(context: Context) -> None:
"""Call _sanitise_db_url with the stored raw URL."""
from cleveragents.cli.commands.system import _sanitise_db_url
context.sanitised_url = _sanitise_db_url(context.raw_db_url)
@when("I call build_info_data")
def step_call_build_info(context: Context) -> None:
"""Call ``build_info_data`` with the mock settings."""
from cleveragents.cli.commands.system import build_info_data
with patch(
"cleveragents.config.settings.get_settings",
return_value=context.mock_settings,
):
context.info_data = build_info_data()
# ---------------------------------------------------------------------------
# Then
# ---------------------------------------------------------------------------
@then('the sanitised database url should be "{expected}"')
def step_assert_sanitised_url(context: Context, expected: str) -> None:
"""Verify the sanitised URL matches expectation."""
actual = context.sanitised_url
assert actual == expected, f"Expected '{expected}', got '{actual}'"
@then('the database field in info data should be "{expected}"')
def step_assert_info_db_field(context: Context, expected: str) -> None:
"""Verify the sanitised URL appears correctly in build_info_data output."""
actual = context.info_data["database"]
assert actual == expected, f"Expected database field '{expected}', got '{actual}'"
+98
View File
@@ -135,6 +135,11 @@ def step_plan_invariants(context):
context.plan_invariants = _parse_invariant_table(context, InvariantScope.PLAN)
@given("I have action invariants")
def step_action_invariants(context):
context.action_invariants = _parse_invariant_table(context, InvariantScope.ACTION)
@given("I have project invariants")
def step_project_invariants(context):
context.project_invariants = _parse_invariant_table(context, InvariantScope.PROJECT)
@@ -160,6 +165,7 @@ def step_plan_invariants_inactive(context):
def step_merge(context):
context.merged = merge_invariants(
getattr(context, "plan_invariants", []),
getattr(context, "action_invariants", []),
getattr(context, "project_invariants", []),
getattr(context, "global_invariants", []),
)
@@ -180,6 +186,43 @@ def step_merged_scope(context, idx, scope):
assert context.merged[idx].scope.value == scope
@then("action invariants appear before project invariants in merge")
def step_action_before_project(context):
"""Verify ACTION tier comes before PROJECT tier in the merged result."""
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.merged if i.scope == InvariantScope.PROJECT]
assert len(action_invs) > 0, "No action invariants found in merged result"
assert len(project_invs) > 0, "No project invariants found in merged result"
first_action_idx = context.merged.index(action_invs[0])
first_project_idx = context.merged.index(project_invs[0])
assert first_action_idx < first_project_idx, (
f"Action invariant at index {first_action_idx} "
f"should appear before project invariant at index {first_project_idx}"
)
@then("plan invariants appear before action invariants in merge")
def step_plan_before_action(context):
"""Verify PLAN tier comes before ACTION tier in the merged result."""
plan_invs = [i for i in context.merged if i.scope == InvariantScope.PLAN]
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
assert len(plan_invs) > 0, "No plan invariants found in merged result"
assert len(action_invs) > 0, "No action invariants found in merged result"
first_plan_idx = context.merged.index(plan_invs[0])
first_action_idx = context.merged.index(action_invs[0])
assert first_plan_idx < first_action_idx, (
f"Plan invariant at index {first_plan_idx} "
f"should appear before action invariant at index {first_action_idx}"
)
@then("action invariants are preserved in merge result")
def step_action_invariants_preserved(context):
"""Verify that action-scoped invariants appear in the merged output."""
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
assert len(action_invs) > 0, "Expected action-scoped invariants in merge result"
# ================================================================
# InvariantSet
# ================================================================
@@ -189,6 +232,7 @@ def step_merged_scope(context, idx, scope):
def step_merge_invariant_set(context):
inv_set = InvariantSet.merge(
getattr(context, "plan_invariants", []),
getattr(context, "action_invariants", []),
getattr(context, "project_invariants", []),
getattr(context, "global_invariants", []),
)
@@ -389,6 +433,15 @@ def step_service_all_scopes(context):
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
@given("an invariant service with invariants at all four scopes")
def step_service_all_four_scopes(context):
context.service = InvariantService()
context.service.add_invariant("Global rule", InvariantScope.GLOBAL, "system")
context.service.add_invariant("Project rule", InvariantScope.PROJECT, "proj1")
context.service.add_invariant("Action rule", InvariantScope.ACTION, "action1")
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
@when('I get effective invariants for plan "{plan_id}" and project "{project}"')
def step_effective(context, plan_id, project):
context.effective = context.service.get_effective_invariants(
@@ -396,6 +449,15 @@ def step_effective(context, plan_id, project):
)
@when(
'I get effective invariants for plan "{plan_id}", action "{action}", and project "{project}"'
)
def step_effective_with_action(context, plan_id, action, project):
context.effective = context.service.get_effective_invariants(
plan_id=plan_id, action_name=action, project_name=project
)
@then("the effective set should contain plan invariants first")
def step_effective_plan_first(context):
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
@@ -404,6 +466,35 @@ def step_effective_plan_first(context):
assert first_plan_idx == 0
@then("the effective set should contain action invariants second")
def step_effective_action_second(context):
"""Verify ACTION tier appears after PLAN and before PROJECT."""
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
assert len(plan_invs) > 0, "No plan invariants found"
assert len(action_invs) > 0, "No action invariants found — bug #9126 not fixed!"
assert len(project_invs) > 0, "No project invariants found"
first_action_idx = context.effective.index(action_invs[0])
if plan_invs:
last_plan_idx = context.effective.index(plan_invs[-1])
assert first_action_idx > last_plan_idx
first_project_idx = context.effective.index(project_invs[0])
assert first_project_idx > first_action_idx
@then("the effective set should contain project invariants after action")
def step_effective_project_after_action(context):
"""Verify PROJECT tier appears after ACTION tier."""
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
assert len(action_invs) > 0, "No action invariants found"
assert len(project_invs) > 0, "No project invariants found"
first_proj_idx = context.effective.index(project_invs[0])
last_action_idx = context.effective.index(action_invs[-1])
assert first_proj_idx > last_action_idx
@then("the effective set should contain project invariants second")
def step_effective_project_second(context):
proj_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
@@ -435,6 +526,13 @@ def step_no_duplicates(context):
assert len(texts) == len(set(texts)), f"Duplicates found: {texts}"
@then("the effective set should have {count:d} invariants")
def step_effective_count(context, count):
assert len(context.effective) == count, (
f"Expected {count}, got {len(context.effective)}"
)
# ================================================================
# Enforcement
# ================================================================
+10 -1
View File
@@ -57,7 +57,6 @@ E2E Plan Lifecycle Through Audit Pipeline
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} E2E_OK
User Identity Field Propagates Through Audit Pipeline
[Documentation] DomainEvent.user_identity propagates via AuditEventSubscriber to AuditService DB entry
[Tags] observability audit identity
@@ -87,3 +86,13 @@ User Identity Field Precedence Over Details
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} IDENTITY_PRECEDENCE_OK
Auth Middleware Emits Audit Events
[Documentation] End-to-end: TokenAuthMiddleware emits AUTH_SUCCESS/AUTH_FAILURE through audit pipeline
[Tags] observability audit auth
${result}= Run Process ${PYTHON} ${HELPER} auth_middleware_pipeline
... cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} AUTH_PIPELINE_OK
+100
View File
@@ -23,6 +23,9 @@ from cleveragents.application.services.audit_event_subscriber import ( # noqa:
from cleveragents.application.services.audit_service import ( # noqa: E402
AuditService,
)
from cleveragents.application.services.auth_middleware import ( # noqa: E402
TokenAuthMiddleware,
)
from cleveragents.application.services.plan_lifecycle_service import ( # noqa: E402
PlanLifecycleService,
)
@@ -323,6 +326,102 @@ def user_identity_field_precedence() -> None:
print("IDENTITY_PRECEDENCE_OK")
def auth_middleware_pipeline() -> None:
"""E2E: TokenAuthMiddleware -> EventBus -> AuditEventSubscriber -> DB."""
svc = _make_audit_service()
bus = ReactiveEventBus()
AuditEventSubscriber(audit_service=svc, event_bus=bus)
middleware = TokenAuthMiddleware(expected_token="tok_secret_123", event_bus=bus)
success = middleware.authenticate(
"tok_secret_123",
identity="carol@example.com",
ip_address="10.1.2.3",
)
failure = middleware.authenticate(
"tok_bad_123",
identity="dave@example.com",
ip_address="10.1.2.4",
)
if not success or failure:
print(
"FAIL: unexpected auth boolean results "
f"success={success} failure={failure}",
file=sys.stderr,
)
sys.exit(1)
success_entries = svc.list_entries(event_type="auth_success")
failure_entries = svc.list_entries(event_type="auth_failure")
if len(success_entries) != 1:
print(
f"FAIL: expected 1 auth_success entry, got {len(success_entries)}",
file=sys.stderr,
)
sys.exit(1)
if len(failure_entries) != 1:
print(
f"FAIL: expected 1 auth_failure entry, got {len(failure_entries)}",
file=sys.stderr,
)
sys.exit(1)
s_entry = success_entries[0]
f_entry = failure_entries[0]
s_details = s_entry.details
f_details = f_entry.details
if s_entry.user_identity != "carol@example.com":
print(
f"FAIL: unexpected auth_success identity {s_entry.user_identity!r}",
file=sys.stderr,
)
sys.exit(1)
if f_entry.user_identity != "unauthenticated":
print(
f"FAIL: unexpected auth_failure identity {f_entry.user_identity!r}",
file=sys.stderr,
)
sys.exit(1)
if s_details.get("ip_address") != "10.1.2.3":
print(
f"FAIL: unexpected auth_success ip {s_details.get('ip_address')!r}",
file=sys.stderr,
)
sys.exit(1)
if s_details.get("token_prefix") != "tok_se...":
print(
"FAIL: auth_success token_prefix mismatch",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("ip_address") != "10.1.2.4":
print(
f"FAIL: unexpected auth_failure ip {f_details.get('ip_address')!r}",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("attempted_identity") != "dave@example.com":
print(
"FAIL: auth_failure attempted_identity mismatch",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("failure_reason") != "invalid_token":
print(
f"FAIL: unexpected auth_failure reason {f_details.get('failure_reason')!r}",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("token_prefix") != "tok_ba...":
print(
"FAIL: auth_failure token_prefix mismatch",
file=sys.stderr,
)
sys.exit(1)
print("AUTH_PIPELINE_OK")
# ---------------------------------------------------------------------------
# Dispatch
# ---------------------------------------------------------------------------
@@ -336,6 +435,7 @@ _COMMANDS = {
"user_identity_field": user_identity_field,
"user_identity_details_fallback": user_identity_details_fallback,
"user_identity_field_precedence": user_identity_field_precedence,
"auth_middleware_pipeline": auth_middleware_pipeline,
}
if __name__ == "__main__":
+2
View File
@@ -867,6 +867,7 @@ def invariants_enforced_during_strategize() -> None:
merged = merge_invariants(
plan_invariants=[plan_inv],
action_invariants=[],
project_invariants=[project_inv],
global_invariants=[global_inv],
)
@@ -890,6 +891,7 @@ def invariants_enforced_during_strategize() -> None:
invariant_set = InvariantSet.merge(
plan_invariants=[plan_inv],
action_invariants=[],
project_invariants=[project_inv],
global_invariants=[global_inv],
)
+27
View File
@@ -23,6 +23,7 @@ from cleveragents.application.services.audit_event_subscriber import (
AuditEventSubscriber,
)
from cleveragents.application.services.audit_service import AuditService
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
from cleveragents.application.services.automation_profile_service import (
AutomationProfileService,
)
@@ -410,6 +411,25 @@ def _resolve_auto_reindex() -> bool:
return True
def _resolve_server_token() -> str | None:
"""Resolve ``server.token`` from config, returning ``None`` when unset."""
try:
svc = ConfigService()
resolved = svc.resolve("server.token")
raw = resolved.value
if raw is None:
return None
token = str(raw).strip()
return token if token else None
except Exception as exc:
_logger.warning(
"server_token_resolution_failed",
error_type=type(exc).__name__,
error_message=redact_value(str(exc)),
)
return None
def _build_analyzer_registry() -> AnalyzerRegistry:
"""Build an AnalyzerRegistry with built-in analyzers registered."""
from cleveragents.domain.models.acms.python_analyzer import PythonAnalyzer
@@ -667,6 +687,13 @@ class Container(containers.DeclarativeContainer):
# Event Bus - Singleton (one shared instance for the process)
event_bus = providers.Singleton(ReactiveEventBus)
# Per-request auth middleware with fresh token resolution from config.
auth_middleware = providers.Factory(
TokenAuthMiddleware,
expected_token=providers.Callable(_resolve_server_token),
event_bus=event_bus,
)
# Services - Factory (new instance per request with injected dependencies)
project_service = providers.Factory(
ProjectService,
@@ -37,19 +37,13 @@ _logger = structlog.get_logger(__name__)
SECURITY_EVENT_MAP: dict[EventType, str] = {
EventType.PLAN_APPLIED: "plan_applied",
EventType.PLAN_CANCELLED: "plan_cancelled",
# NOTE: RESOURCE_MODIFIED has no producing service yet — the tool
# execution framework does not emit this event. The subscriber handler
# is intentionally registered so that audit entries are automatically
# created once tool-write event emission is implemented.
# Emitted by ResourceFileWatcher when indexed resources change.
EventType.RESOURCE_MODIFIED: "resource_modified",
EventType.CORRECTION_APPLIED: "correction_applied",
EventType.CONFIG_CHANGED: "config_changed",
EventType.ENTITY_DELETED: "entity_deleted",
EventType.SESSION_CREATED: "session_created",
# NOTE: AUTH_SUCCESS and AUTH_FAILURE have no producing service yet —
# server-mode authentication is not implemented. The subscriber handlers
# are registered so that audit entries are automatically created once
# server auth emits these events.
# Emitted by TokenAuthMiddleware during bearer-token checks.
EventType.AUTH_SUCCESS: "auth_success",
EventType.AUTH_FAILURE: "auth_failure",
}
@@ -0,0 +1,189 @@
"""Authentication middleware that emits audit-friendly auth events.
This service encapsulates bearer-token authentication checks and emits
``AUTH_SUCCESS`` / ``AUTH_FAILURE`` domain events through an injected
``EventBus`` so the audit pipeline can persist login outcomes.
Spec alignment:
- docs/specification.md § Event System
- docs/specification.md § Audit Logging (auth_success / auth_failure)
"""
from __future__ import annotations
import hmac
from typing import TYPE_CHECKING
import structlog
from cleveragents.infrastructure.events.models import DomainEvent
from cleveragents.infrastructure.events.types import EventType
from cleveragents.shared.redaction import redact_value
if TYPE_CHECKING:
from cleveragents.infrastructure.events.protocol import EventBus
_logger = structlog.get_logger(__name__)
_TOKEN_PREFIX_LEN = 6
_SHORT_TOKEN_MASK = "***..."
def _token_prefix(token: str) -> str:
"""Return a safe token prefix for audit details.
Never returns the full token value. Tokens shorter than or equal to the
prefix length are collapsed to a constant mask to avoid full disclosure.
"""
if len(token) <= _TOKEN_PREFIX_LEN:
return _SHORT_TOKEN_MASK
return f"{token[:_TOKEN_PREFIX_LEN]}..."
def _normalize_optional_text(field_name: str, value: str | None) -> str | None:
"""Normalize optional text fields with guard-first validation."""
if value is None:
return None
if not isinstance(value, str):
raise TypeError(f"{field_name} must be a string or None")
normalized = value.strip()
if not normalized:
raise ValueError(f"{field_name} must not be empty when provided")
return normalized
class TokenAuthMiddleware:
"""Authenticate bearer tokens and emit AUTH_* events.
This class is intentionally transport-agnostic. A future server-mode
middleware can delegate authentication decisions to this component and
inherit consistent event emission semantics.
"""
def __init__(
self,
expected_token: str | None,
event_bus: EventBus | None = None,
) -> None:
"""Initialize middleware with an expected token and optional event bus.
Args:
expected_token: Configured bearer token for successful auth.
event_bus: Event bus used for AUTH_SUCCESS/AUTH_FAILURE emission.
"""
self._expected_token = _normalize_optional_text(
"expected_token", expected_token
)
self._event_bus = event_bus
def authenticate(
self,
token: str,
*,
identity: str | None = None,
ip_address: str | None = None,
) -> bool:
"""Authenticate *token* and emit a corresponding auth event.
Args:
token: Bearer token supplied by the caller.
identity: Optional user identity / principal for audit context.
ip_address: Optional client IP address for audit context.
Returns:
``True`` if authentication succeeds, otherwise ``False``.
"""
if not isinstance(token, str):
raise TypeError("token must be a string")
normalized_token = token.strip()
if not normalized_token:
raise ValueError("token must not be empty")
normalized_identity = _normalize_optional_text("identity", identity)
normalized_ip = _normalize_optional_text("ip_address", ip_address)
if self._expected_token is None:
self._emit_auth_failure(
attempted_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
reason="token_not_configured",
)
return False
if hmac.compare_digest(normalized_token, self._expected_token):
self._emit_auth_success(
user_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
)
return True
self._emit_auth_failure(
attempted_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
reason="invalid_token",
)
return False
def _emit_auth_success(
self,
*,
user_identity: str | None,
ip_address: str | None,
token_prefix: str,
) -> None:
"""Emit AUTH_SUCCESS if an event bus is configured."""
if self._event_bus is None:
return
details: dict[str, str] = {
"user_identity": user_identity or "unknown",
"token_prefix": token_prefix,
}
if ip_address is not None:
details["ip_address"] = ip_address
self._emit_event(EventType.AUTH_SUCCESS, details)
def _emit_auth_failure(
self,
*,
attempted_identity: str | None,
ip_address: str | None,
token_prefix: str,
reason: str,
) -> None:
"""Emit AUTH_FAILURE if an event bus is configured."""
if self._event_bus is None:
return
details: dict[str, str] = {
"attempted_identity": attempted_identity or "unknown",
"user_identity": "unauthenticated",
"failure_reason": reason,
"token_prefix": token_prefix,
}
if ip_address is not None:
details["ip_address"] = ip_address
self._emit_event(EventType.AUTH_FAILURE, details)
def _emit_event(self, event_type: EventType, details: dict[str, str]) -> None:
"""Emit one auth event and swallow publish-side failures."""
bus = self._event_bus
if bus is None:
return
try:
bus.emit(
DomainEvent(
event_type=event_type,
details=details,
)
)
except Exception as exc: # pragma: no cover - defensive logging path
_logger.warning(
"auth_event_emit_failed",
event_type=event_type.value,
error_type=type(exc).__name__,
error_message=redact_value(str(exc)),
)
__all__ = ["TokenAuthMiddleware"]
@@ -3,6 +3,7 @@
from __future__ import annotations
import fnmatch
import json
from pathlib import PurePath
from typing import Any, Protocol
@@ -70,6 +71,72 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
return ProjectContextPolicy().resolve_view("execute")
return policy.resolve_view("execute")
def _resolve_hot_max_tokens(self, project_names: list[str]) -> int:
"""Resolve the effective ``hot_max_tokens`` budget for *project_names*.
Looks up each project's :attr:`ContextConfig.hot_max_tokens` from its
stored context configuration. Projects that do not have an explicit
setting are excluded from the aggregation so they do not affect
the computed value.
Returns the **maximum** of all explicitly-set project-level values,
falling back to :attr:`_hot_max_tokens` (the caller-passed global
default) when no project overrides are present.
"""
from typing import cast
candidates: list[int] = []
for namespaced_name in project_names:
row = None
try:
session = self._project_repository._session() # type: ignore[attr-defined]
from cleveragents.infrastructure.database.models import (
NamespacedProjectModel,
)
row = (
session.query(NamespacedProjectModel)
.filter_by(namespaced_name=namespaced_name)
.first()
)
session.close()
except AttributeError:
self._logger.warning(
"hot_max_tokens_session_factory_missing",
project_name=namespaced_name,
)
continue
except Exception:
self._logger.warning(
"hot_max_tokens_lookup_failed",
project_name=namespaced_name,
exc_info=True,
)
if row is not None and row.context_policy_json is not None:
try:
config_dict = json.loads(cast(str, row.context_policy_json))
tokens = config_dict.get("hot_max_tokens")
if tokens is not None and isinstance(tokens, int) and tokens > 0:
candidates.append(tokens)
except (ValueError, TypeError):
self._logger.warning(
"hot_max_tokens_parse_failed",
project_name=namespaced_name,
)
if candidates:
effective = max(candidates)
self._logger.info(
"hot_max_tokens_resolved_from_projects",
project_names=project_names,
project_values=candidates,
effective=effective,
)
return effective
# No project overrides --- use the caller-passed global default.
return self._hot_max_tokens
@staticmethod
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
"""Return whether *path* passes include/exclude path globs.
@@ -226,14 +293,21 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
)
return None
budget = CoreContextBudget(max_tokens=self._hot_max_tokens, reserved_tokens=0)
# Resolve effective hot_max_tokens: project-level overrides take precedence
# over the global default. When multiple projects have explicit values,
# use the maximum so all projects can contribute within their biggest budget.
effective_hot_max_tokens = self._resolve_hot_max_tokens(project_names)
budget = CoreContextBudget(
max_tokens=effective_hot_max_tokens, reserved_tokens=0
)
request = ContextRequest(
query=(
f"Execute-phase context for plan {plan.identity.plan_id} "
f"({', '.join(project_names)})"
),
purpose="llm_execute_phase_prompt",
max_tokens=self._hot_max_tokens,
max_tokens=effective_hot_max_tokens,
)
payload = self._pipeline.assemble(
plan_id=plan.identity.plan_id,
@@ -11,8 +11,8 @@ a dict keyed by invariant ID.
## Merge Precedence
Effective invariants are computed using plan > project > global order.
See ``merge_invariants`` for de-duplication semantics.
Effective invariants are computed using plan > action > project > global
order. See ``merge_invariants`` for de-duplication semantics.
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
"""
@@ -124,6 +124,7 @@ class InvariantService:
if effective:
return self.get_effective_invariants(
plan_id=source_name if scope == InvariantScope.PLAN else None,
action_name=source_name if scope == InvariantScope.ACTION else None,
project_name=source_name if scope == InvariantScope.PROJECT else None,
)
@@ -167,16 +168,22 @@ class InvariantService:
def get_effective_invariants(
self,
plan_id: str | None = None,
action_name: str | None = None,
project_name: str | None = None,
) -> list[Invariant]:
"""Return the merged precedence chain for a plan/project context.
"""Return the merged precedence chain for a plan/action/project context.
Collects active invariants from each scope tier and merges them
using plan > project > global precedence.
using plan > action > project > global precedence.
Args:
plan_id: Optional plan identifier to collect plan-scoped
invariants.
action_name: Optional action name to collect action-scoped
invariants (promoted to plan-level during reconciliation).
When ``None``, the action tier is omitted for backward
compatibility. Pass ``"*"`` to include all action-scoped
invariants regardless of source name.
project_name: Optional project name to collect project-scoped
invariants.
@@ -191,6 +198,13 @@ class InvariantService:
if inv.scope == InvariantScope.PLAN
and (plan_id is None or inv.source_name == plan_id)
]
action_invs = [
inv
for inv in active
if inv.scope == InvariantScope.ACTION
and action_name is not None # Only include when explicitly requested
and (inv.source_name == action_name or action_name == "*")
]
project_invs = [
inv
for inv in active
@@ -199,7 +213,7 @@ class InvariantService:
]
global_invs = [inv for inv in active if inv.scope == InvariantScope.GLOBAL]
return merge_invariants(plan_invs, project_invs, global_invs)
return merge_invariants(plan_invs, action_invs, project_invs, global_invs)
def enforce_invariants(
self,
@@ -499,20 +499,8 @@ class LLMExecuteActor:
path = match.group(1).strip()
content = match.group(2)
full_path = os.path.normpath(os.path.join(sandbox_root, path))
# Path traversal guard: reject paths escaping sandbox.
# Uses ``os.path.relpath`` for semantic containment checks
# instead of string prefix matching (which is vulnerable
# to sibling-directory prefix-collision attacks).
# See issue #7478 — *startswith bypass* in path containment.
try:
rel = os.path.relpath(full_path, sandbox_root)
except (ValueError, TypeError):
logger.warning(
"Rejected path traversal in LLM output",
path=path,
resolved=full_path,
)
continue
rel = os.path.relpath(full_path, sandbox_root)
# Path traversal guard: reject paths escaping sandbox
if rel.startswith(".." + os.sep) or rel == "..":
logger.warning(
"Rejected path traversal in LLM output",
@@ -18,6 +18,7 @@ from dataclasses import dataclass, field
from cleveragents.domain.models.core.plan import SubplanMergeStrategy
from cleveragents.infrastructure.sandbox.merge import (
ConflictMarkerRegion,
GitMergeStrategy,
MergeResult,
SequentialMergeStrategy,
@@ -39,12 +40,14 @@ class FileMergeOutcome:
path: Relative file path.
content: The merged content (may contain conflict markers).
has_conflict: Whether unresolved conflicts exist.
conflicts: Structured detail of each conflict region (line ranges, snippet).
source_subplan_ids: IDs of subplans that touched this file.
"""
path: str
content: str
has_conflict: bool = False
conflicts: list[ConflictMarkerRegion] = field(default_factory=list)
source_subplan_ids: list[str] = field(default_factory=list)
@@ -251,19 +254,26 @@ class SubplanMergeService:
contents: list[str],
subplan_ids: list[str],
) -> FileMergeOutcome:
"""Iteratively three-way merge each subplan's changes."""
"""Iteratively three-way merge each subplan's changes.
Collects structured conflict details from each intermediate merge step
so the caller has full visibility into where conflicts occurred.
"""
current = contents[0]
has_conflict = False
all_conflicts: list[ConflictMarkerRegion] = []
for content in contents[1:]:
result: MergeResult = self._git_merge.merge(base_content, current, content)
current = result.content
if result.has_conflicts:
has_conflict = True
all_conflicts.extend(result.conflicts)
return FileMergeOutcome(
path=path,
content=current,
has_conflict=has_conflict,
conflicts=all_conflicts,
source_subplan_ids=subplan_ids,
)
+37 -1
View File
@@ -17,6 +17,7 @@ from datetime import UTC, datetime
from enum import StrEnum
from pathlib import Path
from typing import Any
from urllib.parse import urlparse, urlunparse
from sqlalchemy import create_engine
from sqlalchemy import inspect as sa_inspect
@@ -82,6 +83,41 @@ def _dep_version(package: str) -> str:
return "not installed"
def _sanitise_db_url(url: str) -> str:
"""Sanitise a database URL by masking credentials.
Parses the URL and replaces username/password components with masked
placeholders so that CLI output never exposes real credentials.
Examples::
>>> _sanitise_db_url("postgresql://user:secret@localhost/mydb")
'postgresql://***:***@localhost/mydb'
>>> _sanitise_db_url("sqlite:///path/to/db.sqlite")
'sqlite:///path/to/db.sqlite'
Args:
url: The raw database URL.
Returns:
The sanitised URL with credentials masked (or unchanged if no
username/password is present).
"""
parsed = urlparse(url)
# If there is no userinfo segment, return as-is (e.g. sqlite URLs)
if not parsed.username and not parsed.password:
return url
# Rebuild the netloc with masked credentials
userinfo = "***:***"
port_part = f":{parsed.port}" if parsed.port else ""
new_netloc = f"{userinfo}@{parsed.hostname}{port_part}"
sanitized = parsed._replace(netloc=new_netloc)
return urlunparse(sanitized)
def build_version_data() -> dict[str, Any]:
"""Assemble structured data for the ``version`` command."""
return {
@@ -145,7 +181,7 @@ def build_info_data() -> dict[str, Any]:
"version": __version__,
"data_dir": str(data_dir),
"config_path": str(config_path),
"database": db_url,
"database": _sanitise_db_url(db_url),
"server_mode": server_mode,
"platform": f"{platform.system()} {platform.release()} ({platform.machine()})",
"automation": settings.default_automation_profile,
@@ -2,9 +2,10 @@
Invariants are natural-language constraints on plan execution, scoped at
global, project, action, or plan level. When an action is used, its
invariants are promoted to plan-level. The runtime precedence chain is:
invariants are promoted to plan-level and participate in the merge.
The runtime precedence chain is:
plan > project > global
plan > action > project > global
They are reconciled by the Invariant Reconciliation Actor at the start
of Strategize and recorded as ``invariant_enforced`` decisions.
@@ -21,8 +22,8 @@ of Strategize and recorded as ``invariant_enforced`` decisions.
## Merge Precedence
When computing the effective set of invariants for a plan, the merge
order is **plan > project > global**. Duplicate texts (case-insensitive)
are de-duplicated, keeping the highest-precedence copy.
order is **plan > action > project > global**. Duplicate texts
(case-insensitive) are de-duplicated, keeping the highest-precedence copy.
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
"""
@@ -39,8 +40,10 @@ from ulid import ULID
class InvariantScope(StrEnum):
"""Scope at which an invariant applies.
Precedence (highest to lowest): PLAN > PROJECT > GLOBAL.
ACTION invariants are promoted to PLAN scope at ``plan use`` time.
Precedence (highest to lowest): PLAN > ACTION > PROJECT > GLOBAL.
ACTION invariants are promoted to plan-level and participate in the
merge during reconciliation, sitting between PLAN and PROJECT in the
precedence chain.
"""
GLOBAL = "global"
@@ -137,10 +140,11 @@ class InvariantSet(BaseModel):
def merge(
cls,
plan_invariants: list[Invariant],
project_invariants: list[Invariant],
global_invariants: list[Invariant],
action_invariants: list[Invariant] | None = None,
project_invariants: list[Invariant] | None = None,
global_invariants: list[Invariant] | None = None,
) -> InvariantSet:
"""Merge invariants respecting plan > project > global precedence.
"""Merge invariants respecting plan > action > project > global precedence.
De-duplicates by text (case-insensitive), keeping the copy from
the highest-precedence tier. Within each tier, source ordering
@@ -148,6 +152,9 @@ class InvariantSet(BaseModel):
Args:
plan_invariants: Plan-level invariants (highest precedence).
action_invariants: Action-scoped invariants (second-highest
precedence; promoted to plan-level when an
action is used).
project_invariants: Project-level invariants.
global_invariants: Global-level invariants (lowest precedence).
@@ -156,7 +163,12 @@ class InvariantSet(BaseModel):
"""
return cls(
invariants=tuple(
merge_invariants(plan_invariants, project_invariants, global_invariants)
merge_invariants(
plan_invariants,
action_invariants or [],
project_invariants or [],
global_invariants or [],
)
)
)
@@ -165,10 +177,11 @@ class InvariantSet(BaseModel):
def merge_invariants(
plan_invariants: list[Invariant],
project_invariants: list[Invariant],
global_invariants: list[Invariant],
action_invariants: list[Invariant] | None = None,
project_invariants: list[Invariant] | None = None,
global_invariants: list[Invariant] | None = None,
) -> list[Invariant]:
"""Merge invariants implementing plan > project > global precedence.
"""Merge invariants implementing plan > action > project > global precedence.
De-duplicates by text (case-insensitive). The first occurrence
(from the highest-precedence tier) wins. Within each tier, the
@@ -176,8 +189,13 @@ def merge_invariants(
Args:
plan_invariants: Plan-level invariants (highest precedence).
project_invariants: Project-level invariants.
action_invariants: Action-scoped invariants (second-highest
precedence; promoted to plan level when an action is used).
Defaults to empty list when ``None``.
project_invariants: Project-level invariants. Defaults to empty
list when ``None``.
global_invariants: Global-level invariants (lowest precedence).
Defaults to empty list when ``None``.
Returns:
A de-duplicated list of invariants in precedence order.
@@ -185,7 +203,12 @@ def merge_invariants(
seen: set[str] = set()
result: list[Invariant] = []
for inv_list in (plan_invariants, project_invariants, global_invariants):
for inv_list in (
plan_invariants,
action_invariants or [],
project_invariants or [],
global_invariants or [],
):
for inv in inv_list:
if not inv.active:
continue
@@ -27,6 +27,7 @@ from cleveragents.infrastructure.sandbox.factory import SandboxFactory
from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox
from cleveragents.infrastructure.sandbox.manager import SandboxManager
from cleveragents.infrastructure.sandbox.merge import (
ConflictMarkerRegion,
GitMergeStrategy,
JsonMergeStrategy,
MergeResult,
@@ -53,6 +54,7 @@ from cleveragents.infrastructure.sandbox.strategy_registry import (
from cleveragents.infrastructure.sandbox.transaction_sandbox import TransactionSandbox
__all__ = [
"ConflictMarkerRegion",
"AtomicCommitError",
"BoundaryCache",
"BuiltInSandboxStrategyAdapter",
@@ -31,6 +31,25 @@ logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class ConflictMarkerRegion:
"""A single conflict region detected in merged content.
Attributes:
start_line: 1-indexed line where the conflict begins (after <<<<<<< marker).
ours_end_line: 1-indexed line of the ======= divider line (0 if not found, meaning malformed conflict).
theirs_start_line: 1-indexed line after ======= or 0 if no ======= found.
theirs_end_line: 1-indexed line of >>>>>>> marker (or 0 if unterminated).
ours_snippet: Last 3 lines from "ours" section as context.
"""
start_line: int
ours_end_line: int = 0
theirs_start_line: int = 0
theirs_end_line: int = 0
ours_snippet: str = ""
@dataclass(frozen=True)
class MergeResult:
"""Outcome of a three-way merge operation.
@@ -41,13 +60,17 @@ class MergeResult:
``has_conflicts`` is ``True``).
has_conflicts: Whether unresolved conflicts exist in ``content``.
conflict_markers: List of ``(start_line, end_line)`` tuples
indicating regions that contain conflict markers.
indicating regions that contain conflict markers. Deprecated
in favour of :attr:`conflicts` for structured detail.
conflicts: Structured detail of each conflict region (line ranges,
ours snippet).
"""
success: bool
content: str
has_conflicts: bool = False
conflict_markers: list[tuple[int, int]] = field(default_factory=list)
conflicts: list[ConflictMarkerRegion] = field(default_factory=list)
# ---------------------------------------------------------------------------
@@ -136,13 +159,18 @@ class GitMergeStrategy:
return SequentialMergeStrategy().merge(base, ours, theirs)
has_conflicts = result.returncode > 0
conflict_markers = self._find_conflict_markers(merged_content)
conflicts = self._find_conflict_markers(merged_content)
conflict_markers = [
(c.start_line, c.theirs_end_line)
for c in conflicts
]
return MergeResult(
success=not has_conflicts,
content=merged_content,
has_conflicts=has_conflicts,
conflict_markers=conflict_markers,
conflicts=conflicts,
)
except FileNotFoundError:
@@ -154,27 +182,65 @@ class GitMergeStrategy:
shutil.rmtree(tmp_dir, ignore_errors=True)
@staticmethod
def _find_conflict_markers(content: str) -> list[tuple[int, int]]:
def _find_conflict_markers(self, content: str) -> list[ConflictMarkerRegion]:
"""Scan *content* for git conflict marker regions.
Args:
content: The merged file content.
Returns:
List of ``(start_line, end_line)`` 1-indexed tuples for each
conflict region.
List of :class:`ConflictMarkerRegion` objects describing each
conflict region with line ranges and context.
"""
markers: list[tuple[int, int]] = []
markers: list[ConflictMarkerRegion] = []
lines = content.splitlines()
start: int | None = None
for i, line in enumerate(lines):
i = 0
while i < len(lines):
line = lines[i]
if line.startswith("<<<<<<<"):
start = i + 1 # 1-indexed
elif line.startswith(">>>>>>>") and start is not None:
markers.append((start, i + 1))
start = None
ours_lines: list[str] = []
ours_end_line: int = 0
theirs_start_line: int = 0
theirs_end_line: int = 0
j = i + 1
while j < len(lines):
if lines[j].startswith("======="):
ours_end_line = j + 1 # 1-indexed
theirs_start_line = j + 2 # line after =======
break
else:
ours_lines.append(lines[j])
j += 1
k = j + 1 if j < len(lines) and lines[j].startswith("=======") else i + 1
while k < len(lines):
if lines[k].startswith(">>>>>>>"):
theirs_end_line = k + 1 # 1-indexed
break
k += 1
ours_snippet = "\n".join(ours_lines[-3:]) if ours_lines else ""
markers.append(
ConflictMarkerRegion(
start_line=start,
ours_end_line=ours_end_line,
theirs_start_line=theirs_start_line,
theirs_end_line=theirs_end_line,
ours_snippet=ours_snippet,
)
)
# If terminator found, set start to None; otherwise the conflict is unterminated
if theirs_end_line:
start = None
i = k + 1
else:
i += 1
return markers
@@ -251,6 +317,7 @@ class JsonMergeStrategy:
content=theirs if theirs else ours,
has_conflicts=True,
conflict_markers=[],
conflicts=[],
)
merged = self._deep_merge(ours_obj, theirs_obj)
+1
View File
@@ -42,6 +42,7 @@ _SENSITIVE_SUBSTRINGS: set[str] = {
# though they contain a sensitive substring (e.g. "token_count").
_FALSE_POSITIVE_KEYS: set[str] = {
"token_count",
"token_prefix",
"token_limit",
"token_usage",
"input_tokens",
+8 -4
View File
@@ -79,12 +79,16 @@ def validate_path(path_str: str, sandbox_root: str | None = None) -> Path:
Raises ``ValueError`` when the resolved path escapes *sandbox_root*
(defaults to the current working directory when not supplied).
"""
root = Path(sandbox_root).resolve() if sandbox_root else Path.cwd().resolve()
root = Path(sandbox_root) if sandbox_root else Path.cwd()
root = root.resolve()
target = (root / path_str).resolve()
if not (target == root or target.is_relative_to(root)):
try:
target.relative_to(root)
except ValueError as exc:
raise ValueError(
f"Path traversal detected: '{path_str}' escapes sandbox root '{root}'"
)
f"Path traversal detected: '{path_str}' escapes sandbox root"
) from exc
return target
+4 -5
View File
@@ -163,19 +163,18 @@ def _normalise(path: str) -> str:
def _is_under(path: str, root: str) -> bool:
"""Return ``True`` if *path* is equal to or a child of *root*.
Uses semantic path containment via ``posixpath.relpath`` instead of
string prefix matching (``str.startswith``). String prefix matching
Uses semantic path containment via posixpath.relpath instead of
string prefix matching (str.startswith). String prefix matching
is vulnerable to sibling-directory prefix-collision attacks where
``/tmp/sandbox`` would incorrectly match ``/tmp/sandboxmalicious/file``.
/tmp/sandbox would incorrectly match /tmp/sandboxmalicious/file.
See issue #7478 — *startswith bypass* in path containment checks.
See issue #7478 — startswith bypass in path containment checks.
"""
if path == root:
return True
try:
relative = posixpath.relpath(path, root)
except (ValueError, TypeError):
# Cross-drive paths on Windows or other edge cases where relpath fails
return False
return not relative.startswith(".." + posixpath.sep) and relative != ".."