Compare commits
15 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 39b06e7324 | |||
| 8a48f2e62b | |||
| c58a5d772b | |||
| 2fff625a02 | |||
| 5c5309f35d | |||
| f4cea72248 | |||
| 761622f746 | |||
| bf52a9c648 | |||
|
97c1007bb5
|
|||
| b0b28623a1 | |||
| b4351ca78d | |||
| 94622f467c | |||
| 1baa888659 | |||
| 1196c726f2 | |||
| 655cd7ebc2 |
+24
-1
@@ -5,6 +5,8 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
`plan_generation_graph.robot` to give more test answers.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
|
||||
requires whole-word closing keywords (avoids false positives like
|
||||
"prefixes #12"), TDD bug tag discovery now uses exact token matching
|
||||
@@ -40,6 +42,20 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
cleanup conflict with `cli_init_yes_flag_steps.py`); 2 new Behave
|
||||
scenarios covering multi-line PR description parsing and non-string
|
||||
`pr_diff` type guard.
|
||||
|
||||
- Added `TokenAuthMiddleware` and DI wiring to emit missing auth domain
|
||||
events (`AUTH_SUCCESS`, `AUTH_FAILURE`) during token checks, including
|
||||
spec-aligned audit details (`user_identity`, `attempted_identity`,
|
||||
`ip_address`, `token_prefix`, `failure_reason`) and best-effort publish
|
||||
behavior. Auth token prefixes now persist to audit logs without
|
||||
redaction-key collisions, and short tokens are masked (`***...`) to
|
||||
prevent full token disclosure. Added Behave coverage and Robot
|
||||
audit-pipeline integration tests for auth event persistence. (#714)
|
||||
|
||||
- Added TDD bug-capture E2E tests for bug #1028 — ACMS indexing pipeline not
|
||||
wired into CLI. Four Robot Framework E2E tests prove ContextTierService starts
|
||||
empty on every CLI invocation. Tests use ``@tdd_expected_fail`` until the bug
|
||||
fix is merged. (#1029)
|
||||
- Added Fix-then-Revalidate orchestration loop for required validations:
|
||||
bounded retry with configurable limits (0--100 per Safety Profile),
|
||||
strategy revision escalation via `auto_strategy_revision` float
|
||||
@@ -53,6 +69,11 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
`final_validation_results` fields on the result model, DI container
|
||||
|
||||
## [Unreleased]
|
||||
### Added
|
||||
|
||||
- **ContextStrategy protocol and StrategyRegistry with entry-point discovery** (#8616, Epic #8505): Implements the `ContextStrategy` Protocol as a standardized interface for context selection algorithms with configurable budget and scope parameters. Adds centralized `StrategyRegistry` supporting registration, lookup by name, listing enabled/all strategies, configuration-driven enable/disable, per-strategy timeout/max-fragment limits, thread-safe concurrent access via RLock, entry-point-based automatic discovery from the ``cleveragents.context_strategies`` group for plugin-style extensibility, and six built-in strategies (`simple-keyword` quality 0.3, `semantic-embedding` quality 0.6, `breadth-depth-navigator` quality 0.85, `arce` quality 0.95, `temporal-archaeology` quality 0.5, `plan-decision-context` quality 0.7). Includes comprehensive BDD test coverage (>= 97%) validating protocol compliance, registry operations, and strategy discovery mechanisms.
|
||||
|
||||
### Fixed
|
||||
- **fix(tui): rename ActorSelectionOverlay._render to _refresh_display (issue #11039)** — `ActorSelectionOverlay._render()` shadows Textual's `Widget._render()` which must return a `Strip`. In textual >=1.0, layout calls `get_content_height()` `self._render()` gets `None` `AttributeError: 'NoneType' object has no attribute 'get_height'`. Renamed the method to `_refresh_display()` and updated all four internal call sites (`show()`, `move_up()`, `move_down()`, `set_search()`) to use the new name.
|
||||
|
||||
- **Structural Component Output Validation** (#8164): Replaces exact character matching with structural component checking for output validation. Implements three validators covering plan tree output, decision CLI dicts, and structured session snapshots. The `validate_plan_tree` function validates node dicts for required keys (`decision_id`, `type`, `sequence`, `question`, `children`), ULID format, correct types, and sibling ordering. The `validate_decision_dict` function validates decision CLI output against the `Decision.as_cli_dict()` schema with field presence, type, ULID pattern, confidence range [0..1], and boolean field checks. The `validate_structured_output` function validates the StructuredOutput envelope for `command`, `session_id` (ULID), status membership, `exit_code`, and elements integrity. A unified dispatcher (`validate_structured_component_output`) enables routing by target_type. BDD test coverage added in `features/structural_validation.feature`. [Epic #8137](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8137)
|
||||
@@ -64,6 +85,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
when separate `provider`/`model` keys are absent. Added validation to reject malformed
|
||||
combined values with empty provider or model halves.
|
||||
|
||||
- **Fixed plan tree reporting zero decision nodes after strategize** (#10813): The `plan tree` command showed no ``decision_id`` fields even though planning completed successfully. Root cause: the PlanExecutor's ``run_strategize()`` method produced strategy decisions via the StrategyActor but never persisted them as domain ``Decision`` objects through the DecisionService wiring. Added ``decision_service`` parameter to the PlanExecutor constructor, wired it from the CLI dependency-injection container in ``_get_plan_executor()``, and added ``_persist_strategy_decisions()`` that converts each strategy decision into a domain Decision with correct type mapping (prompt_definition, strategy_choice, subplan_spawn) so they appear in plan tree output.
|
||||
- **`task-implementor` posts work-started notification comments** (#11031): Both
|
||||
the `issue_impl` and `pr_fix` procedures now post an informational "work
|
||||
started" comment to the Forgejo issue/PR before beginning implementation.
|
||||
@@ -185,7 +207,8 @@ Changed `wf10_batch.robot` to be less likely to create files, and
|
||||
were cleaned up in `features/actor_add_update_enforcement.feature` so the
|
||||
tests report correctly now that the underlying bug has been fixed.
|
||||
|
||||
- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
|
||||
- **Fixed `merge_invariants()` missing ACTION scope — 4-tier precedence restored** (#9126): Updated ``merge_invariants()`` and ``InvariantSet.merge()`` to accept a fourth parameter ``action_invariants`` alongside plan, project, and global tiers. The module docstring, ``InvariantScope`` docstring, and ``InvariantService.get_effective_invariants()`` now all reflect the correct precedence chain: ``plan > action > project > global``. Added ``action_name`` parameter to ``get_effective_invariants()`` so action-scoped invariants are collected and passed through the merge pipeline instead of silently dropped. All docstrings across both files were corrected from ``plan > project > global`` to ``plan > action > project > global``. Comprehensive Behave scenarios added covering four-tier merge precedence, action-before-project ordering, action override of project with same text, and effective invariant computation with all four scopes.
|
||||
|
||||
step texts to avoid case-sensitive collisions between different step modules that
|
||||
prevented all Behave tests from loading. Renamed steps in
|
||||
`edge_case_plan_steps.py`, `plan_executor_coverage_boost_steps.py`,
|
||||
|
||||
+5
-1
@@ -8,17 +8,19 @@
|
||||
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
|
||||
* Luis Mendes <luis.p.mendes@gmail.com>
|
||||
* Rui Hu <rui.hu@cleverthis.com>
|
||||
* HAL 9000 <hal9000@cleverthis.com> has contributed fix for #10813 — wiring DecisionService into PlanExecutor for strategy decision persistence during strategize.
|
||||
|
||||
# Details
|
||||
|
||||
Below are some of the specific details of various contributions.
|
||||
|
||||
* Jeffrey Phillips Freeman has acted as Lead Developer, daily contributor, and Project Owner.
|
||||
* Jeffrey Phillips Freeman has contributed an implementation for the invariant propagation fix (PR #10881 / issue #9131): added `_propagate_invariant_decisions()` to `SubplanService` to propagate all `invariant_enforced` decisions from parent plans to child plan decision trees during subplan spawn, satisfying the specification requirement for invariant propagation across hierarchical plan execution.
|
||||
* Jeffrey Phillips Freeman has contributed the invariant merge precedence fix (#9126): restored the missing ACTION scope in ``merge_invariants()`` and ``InvariantSet.merge()``, corrected all module docstrings from ``plan > project > global`` to the spec-compliant ``plan > action > project > global``, and added comprehensive BDD test coverage for four-tier merge precedence.
|
||||
* Brent E. Edwards has contributed quality assurance, test coverage, and CI pipeline improvements.
|
||||
* HAL 9000 has contributed automated implementation, bug fixes, and feature development as part of the CleverAgents automation pool.
|
||||
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
|
||||
* HAL 9000 has contributed the plan concurrency race-condition fix (#7989): wired `LockService` into the plan lifecycle, guarding `execute_plan()` and `apply_plan()` with plan-level advisory locks and unique per-invocation owner identities to prevent silent concurrent state corruption.
|
||||
|
||||
* HAL 9000 has contributed the bug-hunt-pool-supervisor non-blocking tracking fix (#7875 / PR #7957): updated step 5 to be best-effort and added rule 9 to prevent the automation-tracking-manager call from blocking the main supervisor loop.
|
||||
* Jeffrey Phillips Freeman has contributed the complete AUTO-BUG-POOL to AUTO-BUG-SUP tracking prefix fix across agent-system-specification.md, automation-tracking.md documentation and agent-system-specification.md spec document, replaced with correct `AUTO-BUG-SUP` prefix used by the bug-hunt-pool-supervisor agent (#7875).
|
||||
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
|
||||
@@ -43,6 +45,8 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed the ACMS context path matching fix (PR #10975 / issue #10972): corrects `_path_matches()` and `_matches_pattern()` to properly match absolute fragment paths against relative glob patterns by auto-prefixing with `**/` before calling `PurePath.full_match()`, preventing silent inefficacy of include/exclude filters for absolute paths in fragment metadata.
|
||||
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
|
||||
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
|
||||
* HAL 9000 has contributed the ContextStrategy protocol and StrategyRegistry system (PR, Epic #8505): implements the ``ContextStrategy`` Protocol with type-safe strategy implementations, a centralized ``StrategyRegistry`` supporting registration/lookup/discovery, entry-point-based automatic discovery from the ``cleveragents.context_strategies`` group for plugin-style extensibility, and six built-in strategies (``simple-keyword``, ``semantic-embedding``, ``breadth-depth-navigator``, ``arce``, ``temporal-archaeology``, ``plan-decision-context``) with BDD test coverage of 97%+.
|
||||
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
|
||||
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
|
||||
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
|
||||
* HAL 9000 has contributed the `ActorSelectionOverlay._render` → `_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
|
||||
|
||||
@@ -74,7 +74,7 @@ class MergeSmallSuite:
|
||||
|
||||
def time_merge_small(self) -> None:
|
||||
"""Benchmark merge with ~13 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeMediumSuite:
|
||||
@@ -88,7 +88,7 @@ class MergeMediumSuite:
|
||||
|
||||
def time_merge_medium(self) -> None:
|
||||
"""Benchmark merge with ~50 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeLargeSuite:
|
||||
@@ -102,7 +102,7 @@ class MergeLargeSuite:
|
||||
|
||||
def time_merge_large(self) -> None:
|
||||
"""Benchmark merge with ~250 invariants."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class MergeDeduplicationSuite:
|
||||
@@ -137,7 +137,7 @@ class MergeDeduplicationSuite:
|
||||
|
||||
def time_merge_dedup(self) -> None:
|
||||
"""Benchmark merge with 60 invariants, all duplicates."""
|
||||
merge_invariants(self.plan, self.project, self.global_invs)
|
||||
merge_invariants(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class InvariantSetMergeSuite:
|
||||
@@ -151,7 +151,7 @@ class InvariantSetMergeSuite:
|
||||
|
||||
def time_invariant_set_merge(self) -> None:
|
||||
"""Benchmark InvariantSet.merge()."""
|
||||
InvariantSet.merge(self.plan, self.project, self.global_invs)
|
||||
InvariantSet.merge(self.plan, [], self.project, self.global_invs)
|
||||
|
||||
|
||||
class ServiceEffectiveSuite:
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
Feature: Auth middleware emits security events
|
||||
As a platform operator
|
||||
I want authentication outcomes emitted as domain events
|
||||
So the audit pipeline captures auth success and failure details
|
||||
|
||||
Scenario: Successful authentication emits AUTH_SUCCESS
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_secret_123" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be true
|
||||
And the auth event bus should have exactly 1 event
|
||||
And the latest auth event type should be AUTH_SUCCESS
|
||||
And the latest auth event should contain detail "user_identity" with value "alice@example.com"
|
||||
And the latest auth event should contain detail "ip_address" with value "10.0.0.5"
|
||||
And the latest auth event should contain detail "token_prefix" with value "tok_se..."
|
||||
|
||||
Scenario: Short token success masks token_prefix
|
||||
Given an auth middleware with expected token "short"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "short" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be true
|
||||
And the latest auth event type should be AUTH_SUCCESS
|
||||
And the latest auth event should contain detail "token_prefix" with value "***..."
|
||||
|
||||
Scenario: Failed authentication emits AUTH_FAILURE
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_wrong_999" identity "alice@example.com" and ip "10.0.0.5"
|
||||
Then the auth result should be false
|
||||
And the auth event bus should have exactly 1 event
|
||||
And the latest auth event type should be AUTH_FAILURE
|
||||
And the latest auth event should contain detail "attempted_identity" with value "alice@example.com"
|
||||
And the latest auth event should contain detail "failure_reason" with value "invalid_token"
|
||||
And the latest auth event should contain detail "token_prefix" with value "tok_wr..."
|
||||
|
||||
Scenario: Missing configured token emits AUTH_FAILURE
|
||||
Given an auth middleware with no configured token
|
||||
And an auth recording event bus
|
||||
When I authenticate with token "tok_any_111" identity "bob@example.com" and ip "10.0.0.9"
|
||||
Then the auth result should be false
|
||||
And the latest auth event type should be AUTH_FAILURE
|
||||
And the latest auth event should contain detail "failure_reason" with value "token_not_configured"
|
||||
|
||||
Scenario: Empty authentication token is rejected before emission
|
||||
Given an auth middleware with expected token "tok_secret_123"
|
||||
And an auth recording event bus
|
||||
When I try to authenticate with an empty token
|
||||
Then an auth ValueError should be raised
|
||||
And the auth event bus should have exactly 0 event
|
||||
|
||||
Scenario: Auth middleware events persist through AuditEventSubscriber
|
||||
Given auth middleware is wired to AuditEventSubscriber with expected token "tok_secret_123"
|
||||
When I authenticate with token "tok_secret_123" identity "carol@example.com" and ip "10.1.2.3"
|
||||
And I authenticate with token "tok_bad_123" identity "dave@example.com" and ip "10.1.2.4"
|
||||
Then the auth result history should be "true,false"
|
||||
And the audit log should contain 1 auth_success entry from middleware
|
||||
And the latest auth_success audit entry should contain detail "ip_address" with value "10.1.2.3"
|
||||
And the latest auth_success audit entry should contain detail "token_prefix" with value "tok_se..."
|
||||
And the audit log should contain 1 auth_failure entry from middleware
|
||||
And the latest auth_failure audit entry should contain detail "ip_address" with value "10.1.2.4"
|
||||
And the latest auth_failure audit entry should contain detail "attempted_identity" with value "dave@example.com"
|
||||
And the latest auth_failure audit entry should contain detail "failure_reason" with value "invalid_token"
|
||||
And the latest auth_failure audit entry should contain detail "token_prefix" with value "tok_ba..."
|
||||
@@ -418,7 +418,7 @@ Feature: Consolidated Domain Models
|
||||
Scenario: InvariantScope has four values
|
||||
Then InvariantScope should have values "global, project, action, plan"
|
||||
|
||||
# === Merge Precedence (plan > project > global) ===
|
||||
# === Merge Precedence (plan > action > project > global) ===
|
||||
|
||||
|
||||
Scenario: Merge with no duplicates preserves all invariants
|
||||
@@ -480,6 +480,75 @@ Feature: Consolidated Domain Models
|
||||
And the merged invariant at index 0 should have text "LOG ALL CHANGES"
|
||||
|
||||
|
||||
Scenario: Merge with all four scopes preserves action tier — Issue #9126
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Action rule | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Project rule | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
| Global rule | system |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 4 invariants
|
||||
And plan invariants appear before action invariants in merge
|
||||
And action invariants appear before project invariants in merge
|
||||
And the merged invariant at index 0 should have text "Plan rule"
|
||||
And the merged invariant at index 1 should have text "Action rule"
|
||||
And the merged invariant at index 2 should have text "Project rule"
|
||||
And the merged invariant at index 3 should have text "Global rule"
|
||||
|
||||
|
||||
Scenario: Action invariant overrides project with same text
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Log all changes | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Log all changes | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "action"
|
||||
|
||||
|
||||
Scenario: Action invariant is included in de-duplication with plan override
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Log all changes | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Log all changes | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "plan"
|
||||
|
||||
|
||||
Scenario: Inactive action invariants are excluded from merge
|
||||
Given I have plan invariants with an inactive entry
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Active | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
When I merge the invariants
|
||||
Then the merged set should have 1 invariants
|
||||
And the merged invariant at index 0 should have scope "action"
|
||||
|
||||
|
||||
Scenario: Inactive invariants are excluded from merge
|
||||
Given I have plan invariants with an inactive entry
|
||||
And I have project invariants
|
||||
@@ -492,7 +561,7 @@ Feature: Consolidated Domain Models
|
||||
# === InvariantSet merge class method ===
|
||||
|
||||
|
||||
Scenario: InvariantSet.merge produces correct result
|
||||
Scenario: InvariantSet.merge produces correct result
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
@@ -505,7 +574,25 @@ Feature: Consolidated Domain Models
|
||||
When I merge using InvariantSet
|
||||
Then the invariant set should have 3 invariants
|
||||
|
||||
# === Service: Add/List/Remove ===
|
||||
|
||||
Scenario: InvariantSet.merge with four-tier precedence — Issue #9126
|
||||
Given I have plan invariants
|
||||
| text | source |
|
||||
| Plan rule | plan1 |
|
||||
And I have action invariants
|
||||
| text | source |
|
||||
| Action rule | action1 |
|
||||
And I have project invariants
|
||||
| text | source |
|
||||
| Project rule | proj1 |
|
||||
And I have global invariants
|
||||
| text | source |
|
||||
| Global rule | system |
|
||||
When I merge using InvariantSet
|
||||
Then the invariant set should have 4 invariants
|
||||
|
||||
|
||||
# === Service: Add/List/Remove ===
|
||||
|
||||
|
||||
Scenario: Service add invariant
|
||||
@@ -574,6 +661,16 @@ Feature: Consolidated Domain Models
|
||||
And the effective set should contain global invariants last
|
||||
|
||||
|
||||
Scenario: Effective invariants include action scope — Issue #9126
|
||||
Given an invariant service with invariants at all four scopes
|
||||
When I get effective invariants for plan "plan1", action "action1", and project "proj1"
|
||||
Then the effective set should contain plan invariants first
|
||||
And the effective set should contain action invariants second
|
||||
And the effective set should contain project invariants after action
|
||||
And the effective set should contain global invariants last
|
||||
And the effective set should have 4 invariants
|
||||
|
||||
|
||||
Scenario: Effective invariants de-duplicate across scopes
|
||||
Given an invariant service with duplicate text across scopes
|
||||
When I get effective invariants for plan "plan1" and project "proj1"
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
@phase2 @acms @context_strategies_batch2
|
||||
Feature: Built-in Context Strategies Batch 2 — Advanced Strategies
|
||||
As a CleverAgents developer
|
||||
I want advanced built-in context strategies (ARCE, Temporal Archaeology, Plan Decision)
|
||||
So that the ACMS pipeline can use high-quality strategies for diverse retrieval scenarios
|
||||
|
||||
# ===========================================================================
|
||||
# ARCE Strategy (quality 0.95)
|
||||
# ===========================================================================
|
||||
|
||||
@arce
|
||||
Scenario: ARCE returns correct quality score with all backends
|
||||
Given an empty context strategy registry
|
||||
When I register all 6 built-in strategies in the registry
|
||||
Then the registry should contain "arce"
|
||||
And the arce explain should contain "adaptive"
|
||||
And the 'arce' name should be "arce"
|
||||
|
||||
|
||||
@arce
|
||||
Scenario: ARCE assembles with composite scoring and iterative refinement
|
||||
Given an empty context strategy registry
|
||||
When I register all 6 built-in strategies in the registry
|
||||
Then the entry for "arce" should be marked as builtin
|
||||
And the arce name should be "arce"
|
||||
|
||||
# ===========================================================================
|
||||
# TemporalArchaeologyStrategy (quality 0.5)
|
||||
# ===========================================================================
|
||||
|
||||
@temporal_archaeology
|
||||
Scenario: TemporalArchaeology returns correct quality score
|
||||
Given an empty context strategy registry
|
||||
When I register all 6 built-in strategies in the registry
|
||||
Then the entry for "temporal-archaeology" should be marked as builtin
|
||||
|
||||
|
||||
@temporal_archaeology
|
||||
Scenario: TemporalArchaeology explain mentions historical patterns
|
||||
Given a BackendSet with temporal backend only
|
||||
And a default ContextRequest
|
||||
When I instantiate the "temporal-archaeology" strategy
|
||||
Then the temporal archaeology explain should contain "historical"
|
||||
|
||||
# ===========================================================================
|
||||
# PlanDecisionContextStrategy (quality 0.7)
|
||||
# ===========================================================================
|
||||
|
||||
@plan_decision_context
|
||||
Scenario: PlanDecisionContext returns correct quality score
|
||||
Given an empty context strategy registry
|
||||
When I register all 6 built-in strategies in the registry
|
||||
Then the entry for "plan-decision-context" should be marked as builtin
|
||||
|
||||
|
||||
@plan_decision_context
|
||||
Scenario: PlanDecisionContext explain mentions decision history
|
||||
Given a BackendSet with temporal backend only
|
||||
And a default ContextRequest
|
||||
When I instantiate the "plan-decision-context" strategy
|
||||
Then the plan decision context explain should contain "prior"
|
||||
@@ -0,0 +1,52 @@
|
||||
@phase2 @acms @entry_points
|
||||
Feature: Context Strategy Entry-Point Discovery
|
||||
As a CleverAgents developer
|
||||
I want context strategies to be auto-discovered via Python entry points
|
||||
So that third-party strategies can be loaded without modifying core code
|
||||
|
||||
# ===========================================================================
|
||||
# Entry-point registration verification
|
||||
# ===========================================================================
|
||||
|
||||
@entry_points_registration
|
||||
Scenario: All 6 built-in strategies are discoverable via entry points
|
||||
Given an empty strategy registry
|
||||
When I discover all strategies from the entry point group in the registry
|
||||
Then the registry should contain "simple-keyword"
|
||||
And the registry should contain "semantic-embedding"
|
||||
And the registry should contain "breadth-depth-navigator"
|
||||
And the registry should contain "arce"
|
||||
And the registry should contain "temporal-archaeology"
|
||||
And the registry should contain "plan-decision-context"
|
||||
And the registry should list 6 strategies
|
||||
|
||||
|
||||
@entry_points_registration
|
||||
Scenario: Entry-point strategy is callable and has correct name
|
||||
Given an empty strategy registry
|
||||
When I discover all strategies from the entry point group in the registry
|
||||
Then the 'arce' should be in the registry via entry points
|
||||
And the 'arce' name should be "arce"
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# Non-existent group handling
|
||||
# ===========================================================================
|
||||
|
||||
@entry_points_missing_group
|
||||
Scenario: Discovery returns empty for non-existent entry point group
|
||||
Given an empty strategy registry
|
||||
When I attempt to discover strategies from a non-existent entry point group
|
||||
Then no new strategies should be registered
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# Third-party strategy discovery (mocked / future)
|
||||
# ===========================================================================
|
||||
|
||||
@third_party_entry_point
|
||||
Scenario: Built-in strategies are properly marked as builtins
|
||||
Given an empty strategy registry
|
||||
When I discover all strategies from the entry point group in the registry
|
||||
Then the builtins should include "simple-keyword"
|
||||
And the builtins should include "arce"
|
||||
@@ -9,6 +9,7 @@ from .fake_provider import FakeProviderInfo, FakeProviderRegistry
|
||||
from .lsp_transport_mock import MockLspTransport, parse_lsp_responses
|
||||
from .mock_ai_provider import MockAIProvider
|
||||
from .mock_mcp_transport import MockMCPTransport
|
||||
from .recording_event_bus import RecordingEventBus
|
||||
from .transient_fail_audit_service import TransientFailAuditService
|
||||
|
||||
# NOTE: tdd_test_helpers is NOT re-exported here because it imports
|
||||
@@ -22,6 +23,7 @@ __all__ = [
|
||||
"MockAIProvider",
|
||||
"MockLspTransport",
|
||||
"MockMCPTransport",
|
||||
"RecordingEventBus",
|
||||
"TransientFailAuditService",
|
||||
"parse_lsp_responses",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Minimal in-memory event bus used for Behave assertions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
|
||||
|
||||
class RecordingEventBus:
|
||||
"""Minimal in-memory event bus used for Behave assertions."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.events: list[DomainEvent] = []
|
||||
|
||||
def emit(self, event: DomainEvent) -> None:
|
||||
self.events.append(event)
|
||||
|
||||
def subscribe(
|
||||
self,
|
||||
event_type: EventType,
|
||||
handler: Callable[[DomainEvent], None],
|
||||
) -> None:
|
||||
_ = (event_type, handler)
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Step definitions for auth_middleware_events.feature."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from cleveragents.application.services.audit_event_subscriber import (
|
||||
AuditEventSubscriber,
|
||||
)
|
||||
from cleveragents.application.services.audit_service import AuditService
|
||||
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.infrastructure.database.models import Base
|
||||
from cleveragents.infrastructure.events.reactive import ReactiveEventBus
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
from features.mocks.recording_event_bus import RecordingEventBus
|
||||
|
||||
|
||||
def _fresh_audit_service() -> AuditService:
|
||||
Settings._instance = None
|
||||
settings = Settings(database_url="sqlite:///:memory:")
|
||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
Base.metadata.create_all(engine)
|
||||
session = sessionmaker(bind=engine)()
|
||||
return AuditService(settings=settings, session=session)
|
||||
|
||||
|
||||
@given('an auth middleware with expected token "{token}"')
|
||||
def step_auth_middleware_with_token(context: Context, token: str) -> None:
|
||||
context.expected_token = token
|
||||
|
||||
|
||||
@given("an auth middleware with no configured token")
|
||||
def step_auth_middleware_no_token(context: Context) -> None:
|
||||
context.expected_token = None
|
||||
|
||||
|
||||
@given("an auth recording event bus")
|
||||
def step_auth_recording_event_bus(context: Context) -> None:
|
||||
context.event_bus = RecordingEventBus()
|
||||
context.auth_middleware = TokenAuthMiddleware(
|
||||
expected_token=context.expected_token,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_result_history = []
|
||||
|
||||
|
||||
@given('auth middleware is wired to AuditEventSubscriber with expected token "{token}"')
|
||||
def step_auth_middleware_audit_pipeline(context: Context, token: str) -> None:
|
||||
context.audit_service = _fresh_audit_service()
|
||||
context.event_bus = ReactiveEventBus()
|
||||
context.audit_subscriber = AuditEventSubscriber(
|
||||
audit_service=context.audit_service,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_middleware = TokenAuthMiddleware(
|
||||
expected_token=token,
|
||||
event_bus=context.event_bus,
|
||||
)
|
||||
context.auth_result_history = []
|
||||
|
||||
|
||||
@when('I authenticate with token "{token}" identity "{identity}" and ip "{ip_address}"')
|
||||
def step_authenticate(
|
||||
context: Context, token: str, identity: str, ip_address: str
|
||||
) -> None:
|
||||
context.auth_result = context.auth_middleware.authenticate(
|
||||
token,
|
||||
identity=identity,
|
||||
ip_address=ip_address,
|
||||
)
|
||||
context.auth_result_history.append(context.auth_result)
|
||||
|
||||
|
||||
@when("I try to authenticate with an empty token")
|
||||
def step_try_authenticate_empty_token(context: Context) -> None:
|
||||
context.auth_error = None
|
||||
try:
|
||||
context.auth_middleware.authenticate("", identity="empty@example.com")
|
||||
except ValueError as exc:
|
||||
context.auth_error = exc
|
||||
|
||||
|
||||
@then("the auth result should be true")
|
||||
def step_auth_result_true(context: Context) -> None:
|
||||
assert context.auth_result is True
|
||||
|
||||
|
||||
@then("the auth result should be false")
|
||||
def step_auth_result_false(context: Context) -> None:
|
||||
assert context.auth_result is False
|
||||
|
||||
|
||||
@then("the auth event bus should have exactly {count:d} event")
|
||||
def step_auth_event_count(context: Context, count: int) -> None:
|
||||
assert len(context.event_bus.events) == count, (
|
||||
f"Expected {count} events, got {len(context.event_bus.events)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the latest auth event type should be {event_type_name}")
|
||||
def step_latest_auth_event_type(context: Context, event_type_name: str) -> None:
|
||||
expected = EventType[event_type_name]
|
||||
actual = context.event_bus.events[-1].event_type
|
||||
assert actual == expected, f"Expected event type {expected}, got {actual}"
|
||||
|
||||
|
||||
@then(
|
||||
'the latest auth event should contain detail "{key}" with value "{expected_value}"'
|
||||
)
|
||||
def step_latest_auth_event_detail(
|
||||
context: Context, key: str, expected_value: str
|
||||
) -> None:
|
||||
details = context.event_bus.events[-1].details
|
||||
assert details.get(key) == expected_value, (
|
||||
f"Expected {key}={expected_value!r}, got {details.get(key)!r}"
|
||||
)
|
||||
|
||||
|
||||
@then("an auth ValueError should be raised")
|
||||
def step_auth_value_error(context: Context) -> None:
|
||||
assert context.auth_error is not None, (
|
||||
"Expected ValueError but no exception was raised"
|
||||
)
|
||||
assert isinstance(context.auth_error, ValueError), (
|
||||
f"Expected ValueError, got {type(context.auth_error).__name__}"
|
||||
)
|
||||
|
||||
|
||||
@then('the auth result history should be "{history_csv}"')
|
||||
def step_auth_result_history(context: Context, history_csv: str) -> None:
|
||||
expected = [part.strip().lower() == "true" for part in history_csv.split(",")]
|
||||
assert context.auth_result_history == expected, (
|
||||
f"Expected history {expected}, got {context.auth_result_history}"
|
||||
)
|
||||
|
||||
|
||||
@then("the audit log should contain 1 auth_success entry from middleware")
|
||||
def step_audit_has_auth_success(context: Context) -> None:
|
||||
entries = context.audit_service.list_entries(event_type="auth_success")
|
||||
assert len(entries) == 1, f"Expected 1 auth_success entry, got {len(entries)}"
|
||||
|
||||
|
||||
@then("the audit log should contain 1 auth_failure entry from middleware")
|
||||
def step_audit_has_auth_failure(context: Context) -> None:
|
||||
entries = context.audit_service.list_entries(event_type="auth_failure")
|
||||
assert len(entries) == 1, f"Expected 1 auth_failure entry, got {len(entries)}"
|
||||
|
||||
|
||||
@then(
|
||||
'the latest {audit_event_type} audit entry should contain detail "{key}" with value "{expected_value}"'
|
||||
)
|
||||
def step_latest_audit_entry_detail(
|
||||
context: Context, audit_event_type: str, key: str, expected_value: str
|
||||
) -> None:
|
||||
entries = context.audit_service.list_entries(event_type=audit_event_type)
|
||||
assert entries, f"Expected at least one {audit_event_type} audit entry"
|
||||
details = entries[0].details
|
||||
assert details.get(key) == expected_value, (
|
||||
f"Expected {audit_event_type}.{key}={expected_value!r}, "
|
||||
f"got {details.get(key)!r}"
|
||||
)
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Behave step implementations for context strategy registry batch 2 and entry points."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.strategy_registry import StrategyRegistry
|
||||
from cleveragents.domain.models.acms.strategy import StrategyConfig
|
||||
from cleveragents.domain.models.acms.strategy_stubs import (
|
||||
ARCEStrategy,
|
||||
BUILTIN_STRATEGY_CLASSES,
|
||||
PlanDecisionContextStrategy,
|
||||
TemporalArchaeologyStrategy,
|
||||
)
|
||||
|
||||
__all__: list[str] = []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _builtin_dict() -> dict[str, object]:
|
||||
"""Return all built-in strategy instances keyed by name."""
|
||||
return {cls().name: cls() for cls in BUILTIN_STRATEGY_CLASSES}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps — batch 2 strategies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("an empty context strategy registry")
|
||||
def step_given_empty_registry(context: Context) -> None:
|
||||
"""Create a fresh ``StrategyRegistry`` with no strategies."""
|
||||
context.registry = StrategyRegistry()
|
||||
|
||||
|
||||
@given('I instantiate the "{name}" strategy')
|
||||
def step_given_instantiate_strategy(context: Context, name: str) -> None:
|
||||
"""Instantiate a single strategy for batch 2 scenarios."""
|
||||
strategy_map: dict[str, object] = {
|
||||
"arce": ARCEStrategy(),
|
||||
"temporal-archaeology": TemporalArchaeologyStrategy(),
|
||||
"plan-decision-context": PlanDecisionContextStrategy(),
|
||||
}
|
||||
if name not in strategy_map:
|
||||
raise ValueError(f"Unknown strategy name: {name}")
|
||||
context.strategy = strategy_map[name]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps — batch 2 strategies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I register all 6 built-in strategies in the registry")
|
||||
def step_when_register_all_6(context: Context) -> None:
|
||||
"""Register all six built-in strategies from strategy_stubs."""
|
||||
for cls in BUILTIN_STRATEGY_CLASSES:
|
||||
inst = cls()
|
||||
context.registry.register(inst, config=StrategyConfig(enabled=True), is_builtin=True)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps — entry points discovery
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I discover all strategies from the entry point group in the registry")
|
||||
def step_when_discover_via_entry_points(context: Context) -> None:
|
||||
"""Trigger actual entry-point discovery via importlib.metadata."""
|
||||
discovered = context.registry.discover_from_entry_points()
|
||||
context.entry_points_discovered = discovered
|
||||
|
||||
|
||||
@when(
|
||||
"I attempt to discover strategies from a non-existent entry point group"
|
||||
)
|
||||
def step_when_discover_nonexistent_group(context: Context) -> None:
|
||||
"""Try to discover from a fake entry-point group — should find nothing."""
|
||||
discovered = context.registry.discover_from_entry_points(
|
||||
group="cleveragents.nonexistent.group"
|
||||
)
|
||||
context.entry_points_discovered = discovered
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps — batch 2 + entry points
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the "arce" should be in the registry via entry points')
|
||||
def step_then_arce_via_entry_points(context: Context) -> None:
|
||||
"""Verify ARCE was registered through entry-point discovery."""
|
||||
assert context.registry.is_registered("arce"), (
|
||||
"ARCE strategy was not registered via entry points"
|
||||
)
|
||||
|
||||
|
||||
@then("no new strategies should be registered")
|
||||
def step_then_no_new_strategies(context: Context) -> None:
|
||||
"""Verify no side effects occurred during discovery."""
|
||||
assert context.entry_points_discovered == 0, (
|
||||
f"Expected 0 discovered strategies, got {context.entry_points_discovered}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Additional Then steps — batch 2 scenarios
|
||||
# The following step implementations cover scenarios from:
|
||||
# features/context_strategies_batch2.feature
|
||||
# features/entry_point_discovery.feature
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the registry should contain "{name}"')
|
||||
def step_then_registry_contains(context: Context, name: str) -> None:
|
||||
"""Verify the registry contains a strategy by name."""
|
||||
assert context.registry.is_registered(name), (
|
||||
f"'{name}' not in registry. Registered: {context.registry.list_all()}"
|
||||
)
|
||||
|
||||
|
||||
@then('the entry for "{name}" should be marked as builtin')
|
||||
def step_then_entry_for_marked_builtin(context: Context, name: str) -> None:
|
||||
"""Verify a named strategy entry is marked as builtin."""
|
||||
entry = context.registry.get_entry(name)
|
||||
assert entry.is_builtin, (
|
||||
f"Entry '{name}' is not marked as builtin. "
|
||||
f"Is registered: {context.registry.is_registered(name)}"
|
||||
)
|
||||
|
||||
|
||||
@then("the builtin list should include the following strategies")
|
||||
def step_then_builtin_list_include(context: Context, table: object) -> None:
|
||||
"""Verify a list of strategy names are marked as builtin."""
|
||||
builtins = context.registry.list_builtin()
|
||||
for row in table.dict: # type: ignore[union-attr]
|
||||
# Iterate over each column (all should map to the same name field)
|
||||
for _key, value in row.items():
|
||||
assert value in builtins, (
|
||||
f"Expected '{value}' in builtin list. Got: {builtins}"
|
||||
)
|
||||
|
||||
|
||||
##############################################################################
|
||||
# --- Named strategy explain steps (batch 2) ---
|
||||
###############################################################################
|
||||
|
||||
|
||||
@then("the arce explain should contain \"{text}\"")
|
||||
def step_then_arce_explain_contains(context: Context, text: str) -> None:
|
||||
"""Verify ARCE strategy explain mentions the given keyword."""
|
||||
strategy = context.registry.get("arce")
|
||||
explanation = strategy.explain()
|
||||
assert text in explanation, (
|
||||
f"Expected '{text}' in ARCE explain.\nGot:\n{explanation}"
|
||||
)
|
||||
|
||||
|
||||
@then("the arce name should be \"{name}\"")
|
||||
def step_then_arce_name(context: Context, name: str) -> None:
|
||||
"""Verify the ARCE strategy has the expected name."""
|
||||
strategy = context.registry.get("arce")
|
||||
assert strategy.name == name, (
|
||||
f"Expected ARCE name '{name}', got '{strategy.name}'"
|
||||
)
|
||||
|
||||
|
||||
##############################################################################
|
||||
# --- Named strategy explain steps — temporal archaeology ---
|
||||
###############################################################################
|
||||
|
||||
|
||||
@then("the temporal archaeology explain should contain \"{text}\"")
|
||||
def step_then_temporal_explain_contains(context: Context, text: str) -> None:
|
||||
"""Verify TemporalArchaeology strategy explain mentions the given keyword."""
|
||||
strategy = context.registry.get("temporal-archaeology")
|
||||
explanation = strategy.explain()
|
||||
assert text in explanation, (
|
||||
f"Expected '{text}' in temporal archaeology explain.\nGot:\n{explanation}"
|
||||
)
|
||||
|
||||
|
||||
##############################################################################
|
||||
# --- Named strategy explain steps — plan decision context ---
|
||||
###############################################################################
|
||||
|
||||
|
||||
@then("the plan decision context explain should contain \"{text}\"")
|
||||
def step_then_plan_dec_explain_contains(context: Context, text: str) -> None:
|
||||
"""Verify PlanDecisionContextStrategy explain mentions the given keyword."""
|
||||
strategy = context.registry.get("plan-decision-context")
|
||||
explanation = strategy.explain()
|
||||
assert text in explanation, (
|
||||
f"Expected '{text}' in plan decision context explain.\nGot:\n{explanation}"
|
||||
)
|
||||
|
||||
|
||||
##############################################################################
|
||||
# --- Named strategy name steps with quoted single quotes ---
|
||||
################################################################################
|
||||
|
||||
|
||||
@then("the 'arce' name should be \"{name}\"")
|
||||
def step_then_arce_quoted_name(context: Context, name: str) -> None:
|
||||
"""Verify ARCE (quoted key) has the expected name.
|
||||
|
||||
This step handles the Gherkin pattern: the 'arce' name should be "arce"
|
||||
where single quotes delimit the strategy identifier.
|
||||
"""
|
||||
strategy = context.registry.get("arce")
|
||||
assert strategy.name == name, (
|
||||
f"Expected 'arce' name '{name}', got '{strategy.name}'"
|
||||
)
|
||||
|
||||
|
||||
##############################################################################
|
||||
# --- Builtins should include individual names ---
|
||||
###############################################################################
|
||||
|
||||
|
||||
@then("the builtins should include \"{name}\"")
|
||||
def step_then_builtin_include(context: Context, name: str) -> None:
|
||||
"""Verify a named strategy exists in the builtin list."""
|
||||
builtins = context.registry.list_builtin()
|
||||
assert name in builtins, (
|
||||
f"Expected '{name}' in builtin list. Got: {builtins}"
|
||||
)
|
||||
@@ -135,6 +135,11 @@ def step_plan_invariants(context):
|
||||
context.plan_invariants = _parse_invariant_table(context, InvariantScope.PLAN)
|
||||
|
||||
|
||||
@given("I have action invariants")
|
||||
def step_action_invariants(context):
|
||||
context.action_invariants = _parse_invariant_table(context, InvariantScope.ACTION)
|
||||
|
||||
|
||||
@given("I have project invariants")
|
||||
def step_project_invariants(context):
|
||||
context.project_invariants = _parse_invariant_table(context, InvariantScope.PROJECT)
|
||||
@@ -160,6 +165,7 @@ def step_plan_invariants_inactive(context):
|
||||
def step_merge(context):
|
||||
context.merged = merge_invariants(
|
||||
getattr(context, "plan_invariants", []),
|
||||
getattr(context, "action_invariants", []),
|
||||
getattr(context, "project_invariants", []),
|
||||
getattr(context, "global_invariants", []),
|
||||
)
|
||||
@@ -180,6 +186,43 @@ def step_merged_scope(context, idx, scope):
|
||||
assert context.merged[idx].scope.value == scope
|
||||
|
||||
|
||||
@then("action invariants appear before project invariants in merge")
|
||||
def step_action_before_project(context):
|
||||
"""Verify ACTION tier comes before PROJECT tier in the merged result."""
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.merged if i.scope == InvariantScope.PROJECT]
|
||||
assert len(action_invs) > 0, "No action invariants found in merged result"
|
||||
assert len(project_invs) > 0, "No project invariants found in merged result"
|
||||
first_action_idx = context.merged.index(action_invs[0])
|
||||
first_project_idx = context.merged.index(project_invs[0])
|
||||
assert first_action_idx < first_project_idx, (
|
||||
f"Action invariant at index {first_action_idx} "
|
||||
f"should appear before project invariant at index {first_project_idx}"
|
||||
)
|
||||
|
||||
|
||||
@then("plan invariants appear before action invariants in merge")
|
||||
def step_plan_before_action(context):
|
||||
"""Verify PLAN tier comes before ACTION tier in the merged result."""
|
||||
plan_invs = [i for i in context.merged if i.scope == InvariantScope.PLAN]
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
assert len(plan_invs) > 0, "No plan invariants found in merged result"
|
||||
assert len(action_invs) > 0, "No action invariants found in merged result"
|
||||
first_plan_idx = context.merged.index(plan_invs[0])
|
||||
first_action_idx = context.merged.index(action_invs[0])
|
||||
assert first_plan_idx < first_action_idx, (
|
||||
f"Plan invariant at index {first_plan_idx} "
|
||||
f"should appear before action invariant at index {first_action_idx}"
|
||||
)
|
||||
|
||||
|
||||
@then("action invariants are preserved in merge result")
|
||||
def step_action_invariants_preserved(context):
|
||||
"""Verify that action-scoped invariants appear in the merged output."""
|
||||
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
|
||||
assert len(action_invs) > 0, "Expected action-scoped invariants in merge result"
|
||||
|
||||
|
||||
# ================================================================
|
||||
# InvariantSet
|
||||
# ================================================================
|
||||
@@ -189,6 +232,7 @@ def step_merged_scope(context, idx, scope):
|
||||
def step_merge_invariant_set(context):
|
||||
inv_set = InvariantSet.merge(
|
||||
getattr(context, "plan_invariants", []),
|
||||
getattr(context, "action_invariants", []),
|
||||
getattr(context, "project_invariants", []),
|
||||
getattr(context, "global_invariants", []),
|
||||
)
|
||||
@@ -389,6 +433,15 @@ def step_service_all_scopes(context):
|
||||
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
|
||||
|
||||
|
||||
@given("an invariant service with invariants at all four scopes")
|
||||
def step_service_all_four_scopes(context):
|
||||
context.service = InvariantService()
|
||||
context.service.add_invariant("Global rule", InvariantScope.GLOBAL, "system")
|
||||
context.service.add_invariant("Project rule", InvariantScope.PROJECT, "proj1")
|
||||
context.service.add_invariant("Action rule", InvariantScope.ACTION, "action1")
|
||||
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
|
||||
|
||||
|
||||
@when('I get effective invariants for plan "{plan_id}" and project "{project}"')
|
||||
def step_effective(context, plan_id, project):
|
||||
context.effective = context.service.get_effective_invariants(
|
||||
@@ -396,6 +449,15 @@ def step_effective(context, plan_id, project):
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I get effective invariants for plan "{plan_id}", action "{action}", and project "{project}"'
|
||||
)
|
||||
def step_effective_with_action(context, plan_id, action, project):
|
||||
context.effective = context.service.get_effective_invariants(
|
||||
plan_id=plan_id, action_name=action, project_name=project
|
||||
)
|
||||
|
||||
|
||||
@then("the effective set should contain plan invariants first")
|
||||
def step_effective_plan_first(context):
|
||||
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
|
||||
@@ -404,6 +466,35 @@ def step_effective_plan_first(context):
|
||||
assert first_plan_idx == 0
|
||||
|
||||
|
||||
@then("the effective set should contain action invariants second")
|
||||
def step_effective_action_second(context):
|
||||
"""Verify ACTION tier appears after PLAN and before PROJECT."""
|
||||
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
|
||||
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
assert len(plan_invs) > 0, "No plan invariants found"
|
||||
assert len(action_invs) > 0, "No action invariants found — bug #9126 not fixed!"
|
||||
assert len(project_invs) > 0, "No project invariants found"
|
||||
first_action_idx = context.effective.index(action_invs[0])
|
||||
if plan_invs:
|
||||
last_plan_idx = context.effective.index(plan_invs[-1])
|
||||
assert first_action_idx > last_plan_idx
|
||||
first_project_idx = context.effective.index(project_invs[0])
|
||||
assert first_project_idx > first_action_idx
|
||||
|
||||
|
||||
@then("the effective set should contain project invariants after action")
|
||||
def step_effective_project_after_action(context):
|
||||
"""Verify PROJECT tier appears after ACTION tier."""
|
||||
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
|
||||
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
assert len(action_invs) > 0, "No action invariants found"
|
||||
assert len(project_invs) > 0, "No project invariants found"
|
||||
first_proj_idx = context.effective.index(project_invs[0])
|
||||
last_action_idx = context.effective.index(action_invs[-1])
|
||||
assert first_proj_idx > last_action_idx
|
||||
|
||||
|
||||
@then("the effective set should contain project invariants second")
|
||||
def step_effective_project_second(context):
|
||||
proj_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
|
||||
@@ -435,6 +526,13 @@ def step_no_duplicates(context):
|
||||
assert len(texts) == len(set(texts)), f"Duplicates found: {texts}"
|
||||
|
||||
|
||||
@then("the effective set should have {count:d} invariants")
|
||||
def step_effective_count(context, count):
|
||||
assert len(context.effective) == count, (
|
||||
f"Expected {count}, got {len(context.effective)}"
|
||||
)
|
||||
|
||||
|
||||
# ================================================================
|
||||
# Enforcement
|
||||
# ================================================================
|
||||
|
||||
@@ -107,6 +107,14 @@ Issues = "https://git.cleverthis.com/cleveragents/core/issues"
|
||||
cleveragents = "cleveragents.cli:main"
|
||||
agents = "cleveragents.cli:main"
|
||||
|
||||
[project.entry-points."cleveragents.context_strategies"]
|
||||
simple-keyword = "cleveragents.domain.models.acms.strategy_stubs:SimpleKeywordStrategy"
|
||||
semantic-embedding = "cleveragents.domain.models.acms.strategy_stubs:SemanticEmbeddingStrategy"
|
||||
breadth-depth-navigator = "cleveragents.domain.models.acms.strategy_stubs:BreadthDepthNavigatorStrategy"
|
||||
arce = "cleveragents.domain.models.acms.strategy_stubs:ARCEStrategy"
|
||||
temporal-archaeology = "cleveragents.domain.models.acms.strategy_stubs:TemporalArchaeologyStrategy"
|
||||
plan-decision-context = "cleveragents.domain.models.acms.strategy_stubs:PlanDecisionContextStrategy"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/cleveragents"]
|
||||
include = [
|
||||
|
||||
@@ -57,7 +57,6 @@ E2E Plan Lifecycle Through Audit Pipeline
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} E2E_OK
|
||||
|
||||
User Identity Field Propagates Through Audit Pipeline
|
||||
[Documentation] DomainEvent.user_identity propagates via AuditEventSubscriber to AuditService DB entry
|
||||
[Tags] observability audit identity
|
||||
@@ -87,3 +86,13 @@ User Identity Field Precedence Over Details
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} IDENTITY_PRECEDENCE_OK
|
||||
|
||||
Auth Middleware Emits Audit Events
|
||||
[Documentation] End-to-end: TokenAuthMiddleware emits AUTH_SUCCESS/AUTH_FAILURE through audit pipeline
|
||||
[Tags] observability audit auth
|
||||
${result}= Run Process ${PYTHON} ${HELPER} auth_middleware_pipeline
|
||||
... cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} AUTH_PIPELINE_OK
|
||||
|
||||
@@ -23,6 +23,9 @@ from cleveragents.application.services.audit_event_subscriber import ( # noqa:
|
||||
from cleveragents.application.services.audit_service import ( # noqa: E402
|
||||
AuditService,
|
||||
)
|
||||
from cleveragents.application.services.auth_middleware import ( # noqa: E402
|
||||
TokenAuthMiddleware,
|
||||
)
|
||||
from cleveragents.application.services.plan_lifecycle_service import ( # noqa: E402
|
||||
PlanLifecycleService,
|
||||
)
|
||||
@@ -323,6 +326,102 @@ def user_identity_field_precedence() -> None:
|
||||
print("IDENTITY_PRECEDENCE_OK")
|
||||
|
||||
|
||||
def auth_middleware_pipeline() -> None:
|
||||
"""E2E: TokenAuthMiddleware -> EventBus -> AuditEventSubscriber -> DB."""
|
||||
svc = _make_audit_service()
|
||||
bus = ReactiveEventBus()
|
||||
AuditEventSubscriber(audit_service=svc, event_bus=bus)
|
||||
middleware = TokenAuthMiddleware(expected_token="tok_secret_123", event_bus=bus)
|
||||
|
||||
success = middleware.authenticate(
|
||||
"tok_secret_123",
|
||||
identity="carol@example.com",
|
||||
ip_address="10.1.2.3",
|
||||
)
|
||||
failure = middleware.authenticate(
|
||||
"tok_bad_123",
|
||||
identity="dave@example.com",
|
||||
ip_address="10.1.2.4",
|
||||
)
|
||||
if not success or failure:
|
||||
print(
|
||||
"FAIL: unexpected auth boolean results "
|
||||
f"success={success} failure={failure}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
success_entries = svc.list_entries(event_type="auth_success")
|
||||
failure_entries = svc.list_entries(event_type="auth_failure")
|
||||
if len(success_entries) != 1:
|
||||
print(
|
||||
f"FAIL: expected 1 auth_success entry, got {len(success_entries)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if len(failure_entries) != 1:
|
||||
print(
|
||||
f"FAIL: expected 1 auth_failure entry, got {len(failure_entries)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
s_entry = success_entries[0]
|
||||
f_entry = failure_entries[0]
|
||||
s_details = s_entry.details
|
||||
f_details = f_entry.details
|
||||
if s_entry.user_identity != "carol@example.com":
|
||||
print(
|
||||
f"FAIL: unexpected auth_success identity {s_entry.user_identity!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_entry.user_identity != "unauthenticated":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure identity {f_entry.user_identity!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if s_details.get("ip_address") != "10.1.2.3":
|
||||
print(
|
||||
f"FAIL: unexpected auth_success ip {s_details.get('ip_address')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if s_details.get("token_prefix") != "tok_se...":
|
||||
print(
|
||||
"FAIL: auth_success token_prefix mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("ip_address") != "10.1.2.4":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure ip {f_details.get('ip_address')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("attempted_identity") != "dave@example.com":
|
||||
print(
|
||||
"FAIL: auth_failure attempted_identity mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("failure_reason") != "invalid_token":
|
||||
print(
|
||||
f"FAIL: unexpected auth_failure reason {f_details.get('failure_reason')!r}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
if f_details.get("token_prefix") != "tok_ba...":
|
||||
print(
|
||||
"FAIL: auth_failure token_prefix mismatch",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print("AUTH_PIPELINE_OK")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dispatch
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -336,6 +435,7 @@ _COMMANDS = {
|
||||
"user_identity_field": user_identity_field,
|
||||
"user_identity_details_fallback": user_identity_details_fallback,
|
||||
"user_identity_field_precedence": user_identity_field_precedence,
|
||||
"auth_middleware_pipeline": auth_middleware_pipeline,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -867,6 +867,7 @@ def invariants_enforced_during_strategize() -> None:
|
||||
|
||||
merged = merge_invariants(
|
||||
plan_invariants=[plan_inv],
|
||||
action_invariants=[],
|
||||
project_invariants=[project_inv],
|
||||
global_invariants=[global_inv],
|
||||
)
|
||||
@@ -890,6 +891,7 @@ def invariants_enforced_during_strategize() -> None:
|
||||
|
||||
invariant_set = InvariantSet.merge(
|
||||
plan_invariants=[plan_inv],
|
||||
action_invariants=[],
|
||||
project_invariants=[project_inv],
|
||||
global_invariants=[global_inv],
|
||||
)
|
||||
|
||||
@@ -23,6 +23,7 @@ from cleveragents.application.services.audit_event_subscriber import (
|
||||
AuditEventSubscriber,
|
||||
)
|
||||
from cleveragents.application.services.audit_service import AuditService
|
||||
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
|
||||
from cleveragents.application.services.automation_profile_service import (
|
||||
AutomationProfileService,
|
||||
)
|
||||
@@ -410,6 +411,25 @@ def _resolve_auto_reindex() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _resolve_server_token() -> str | None:
|
||||
"""Resolve ``server.token`` from config, returning ``None`` when unset."""
|
||||
try:
|
||||
svc = ConfigService()
|
||||
resolved = svc.resolve("server.token")
|
||||
raw = resolved.value
|
||||
if raw is None:
|
||||
return None
|
||||
token = str(raw).strip()
|
||||
return token if token else None
|
||||
except Exception as exc:
|
||||
_logger.warning(
|
||||
"server_token_resolution_failed",
|
||||
error_type=type(exc).__name__,
|
||||
error_message=redact_value(str(exc)),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _build_analyzer_registry() -> AnalyzerRegistry:
|
||||
"""Build an AnalyzerRegistry with built-in analyzers registered."""
|
||||
from cleveragents.domain.models.acms.python_analyzer import PythonAnalyzer
|
||||
@@ -667,6 +687,13 @@ class Container(containers.DeclarativeContainer):
|
||||
# Event Bus - Singleton (one shared instance for the process)
|
||||
event_bus = providers.Singleton(ReactiveEventBus)
|
||||
|
||||
# Per-request auth middleware with fresh token resolution from config.
|
||||
auth_middleware = providers.Factory(
|
||||
TokenAuthMiddleware,
|
||||
expected_token=providers.Callable(_resolve_server_token),
|
||||
event_bus=event_bus,
|
||||
)
|
||||
|
||||
# Services - Factory (new instance per request with injected dependencies)
|
||||
project_service = providers.Factory(
|
||||
ProjectService,
|
||||
|
||||
@@ -37,19 +37,13 @@ _logger = structlog.get_logger(__name__)
|
||||
SECURITY_EVENT_MAP: dict[EventType, str] = {
|
||||
EventType.PLAN_APPLIED: "plan_applied",
|
||||
EventType.PLAN_CANCELLED: "plan_cancelled",
|
||||
# NOTE: RESOURCE_MODIFIED has no producing service yet — the tool
|
||||
# execution framework does not emit this event. The subscriber handler
|
||||
# is intentionally registered so that audit entries are automatically
|
||||
# created once tool-write event emission is implemented.
|
||||
# Emitted by ResourceFileWatcher when indexed resources change.
|
||||
EventType.RESOURCE_MODIFIED: "resource_modified",
|
||||
EventType.CORRECTION_APPLIED: "correction_applied",
|
||||
EventType.CONFIG_CHANGED: "config_changed",
|
||||
EventType.ENTITY_DELETED: "entity_deleted",
|
||||
EventType.SESSION_CREATED: "session_created",
|
||||
# NOTE: AUTH_SUCCESS and AUTH_FAILURE have no producing service yet —
|
||||
# server-mode authentication is not implemented. The subscriber handlers
|
||||
# are registered so that audit entries are automatically created once
|
||||
# server auth emits these events.
|
||||
# Emitted by TokenAuthMiddleware during bearer-token checks.
|
||||
EventType.AUTH_SUCCESS: "auth_success",
|
||||
EventType.AUTH_FAILURE: "auth_failure",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
"""Authentication middleware that emits audit-friendly auth events.
|
||||
|
||||
This service encapsulates bearer-token authentication checks and emits
|
||||
``AUTH_SUCCESS`` / ``AUTH_FAILURE`` domain events through an injected
|
||||
``EventBus`` so the audit pipeline can persist login outcomes.
|
||||
|
||||
Spec alignment:
|
||||
- docs/specification.md § Event System
|
||||
- docs/specification.md § Audit Logging (auth_success / auth_failure)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import structlog
|
||||
|
||||
from cleveragents.infrastructure.events.models import DomainEvent
|
||||
from cleveragents.infrastructure.events.types import EventType
|
||||
from cleveragents.shared.redaction import redact_value
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cleveragents.infrastructure.events.protocol import EventBus
|
||||
|
||||
_logger = structlog.get_logger(__name__)
|
||||
_TOKEN_PREFIX_LEN = 6
|
||||
_SHORT_TOKEN_MASK = "***..."
|
||||
|
||||
|
||||
def _token_prefix(token: str) -> str:
|
||||
"""Return a safe token prefix for audit details.
|
||||
|
||||
Never returns the full token value. Tokens shorter than or equal to the
|
||||
prefix length are collapsed to a constant mask to avoid full disclosure.
|
||||
"""
|
||||
if len(token) <= _TOKEN_PREFIX_LEN:
|
||||
return _SHORT_TOKEN_MASK
|
||||
return f"{token[:_TOKEN_PREFIX_LEN]}..."
|
||||
|
||||
|
||||
def _normalize_optional_text(field_name: str, value: str | None) -> str | None:
|
||||
"""Normalize optional text fields with guard-first validation."""
|
||||
if value is None:
|
||||
return None
|
||||
if not isinstance(value, str):
|
||||
raise TypeError(f"{field_name} must be a string or None")
|
||||
normalized = value.strip()
|
||||
if not normalized:
|
||||
raise ValueError(f"{field_name} must not be empty when provided")
|
||||
return normalized
|
||||
|
||||
|
||||
class TokenAuthMiddleware:
|
||||
"""Authenticate bearer tokens and emit AUTH_* events.
|
||||
|
||||
This class is intentionally transport-agnostic. A future server-mode
|
||||
middleware can delegate authentication decisions to this component and
|
||||
inherit consistent event emission semantics.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
expected_token: str | None,
|
||||
event_bus: EventBus | None = None,
|
||||
) -> None:
|
||||
"""Initialize middleware with an expected token and optional event bus.
|
||||
|
||||
Args:
|
||||
expected_token: Configured bearer token for successful auth.
|
||||
event_bus: Event bus used for AUTH_SUCCESS/AUTH_FAILURE emission.
|
||||
"""
|
||||
self._expected_token = _normalize_optional_text(
|
||||
"expected_token", expected_token
|
||||
)
|
||||
self._event_bus = event_bus
|
||||
|
||||
def authenticate(
|
||||
self,
|
||||
token: str,
|
||||
*,
|
||||
identity: str | None = None,
|
||||
ip_address: str | None = None,
|
||||
) -> bool:
|
||||
"""Authenticate *token* and emit a corresponding auth event.
|
||||
|
||||
Args:
|
||||
token: Bearer token supplied by the caller.
|
||||
identity: Optional user identity / principal for audit context.
|
||||
ip_address: Optional client IP address for audit context.
|
||||
|
||||
Returns:
|
||||
``True`` if authentication succeeds, otherwise ``False``.
|
||||
"""
|
||||
if not isinstance(token, str):
|
||||
raise TypeError("token must be a string")
|
||||
normalized_token = token.strip()
|
||||
if not normalized_token:
|
||||
raise ValueError("token must not be empty")
|
||||
|
||||
normalized_identity = _normalize_optional_text("identity", identity)
|
||||
normalized_ip = _normalize_optional_text("ip_address", ip_address)
|
||||
|
||||
if self._expected_token is None:
|
||||
self._emit_auth_failure(
|
||||
attempted_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
reason="token_not_configured",
|
||||
)
|
||||
return False
|
||||
|
||||
if hmac.compare_digest(normalized_token, self._expected_token):
|
||||
self._emit_auth_success(
|
||||
user_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
)
|
||||
return True
|
||||
|
||||
self._emit_auth_failure(
|
||||
attempted_identity=normalized_identity,
|
||||
ip_address=normalized_ip,
|
||||
token_prefix=_token_prefix(normalized_token),
|
||||
reason="invalid_token",
|
||||
)
|
||||
return False
|
||||
|
||||
def _emit_auth_success(
|
||||
self,
|
||||
*,
|
||||
user_identity: str | None,
|
||||
ip_address: str | None,
|
||||
token_prefix: str,
|
||||
) -> None:
|
||||
"""Emit AUTH_SUCCESS if an event bus is configured."""
|
||||
if self._event_bus is None:
|
||||
return
|
||||
details: dict[str, str] = {
|
||||
"user_identity": user_identity or "unknown",
|
||||
"token_prefix": token_prefix,
|
||||
}
|
||||
if ip_address is not None:
|
||||
details["ip_address"] = ip_address
|
||||
self._emit_event(EventType.AUTH_SUCCESS, details)
|
||||
|
||||
def _emit_auth_failure(
|
||||
self,
|
||||
*,
|
||||
attempted_identity: str | None,
|
||||
ip_address: str | None,
|
||||
token_prefix: str,
|
||||
reason: str,
|
||||
) -> None:
|
||||
"""Emit AUTH_FAILURE if an event bus is configured."""
|
||||
if self._event_bus is None:
|
||||
return
|
||||
details: dict[str, str] = {
|
||||
"attempted_identity": attempted_identity or "unknown",
|
||||
"user_identity": "unauthenticated",
|
||||
"failure_reason": reason,
|
||||
"token_prefix": token_prefix,
|
||||
}
|
||||
if ip_address is not None:
|
||||
details["ip_address"] = ip_address
|
||||
self._emit_event(EventType.AUTH_FAILURE, details)
|
||||
|
||||
def _emit_event(self, event_type: EventType, details: dict[str, str]) -> None:
|
||||
"""Emit one auth event and swallow publish-side failures."""
|
||||
bus = self._event_bus
|
||||
if bus is None:
|
||||
return
|
||||
try:
|
||||
bus.emit(
|
||||
DomainEvent(
|
||||
event_type=event_type,
|
||||
details=details,
|
||||
)
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - defensive logging path
|
||||
_logger.warning(
|
||||
"auth_event_emit_failed",
|
||||
event_type=event_type.value,
|
||||
error_type=type(exc).__name__,
|
||||
error_message=redact_value(str(exc)),
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["TokenAuthMiddleware"]
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import json
|
||||
from pathlib import PurePath
|
||||
from typing import Any, Protocol
|
||||
|
||||
@@ -70,6 +71,72 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
return ProjectContextPolicy().resolve_view("execute")
|
||||
return policy.resolve_view("execute")
|
||||
|
||||
def _resolve_hot_max_tokens(self, project_names: list[str]) -> int:
|
||||
"""Resolve the effective ``hot_max_tokens`` budget for *project_names*.
|
||||
|
||||
Looks up each project's :attr:`ContextConfig.hot_max_tokens` from its
|
||||
stored context configuration. Projects that do not have an explicit
|
||||
setting are excluded from the aggregation so they do not affect
|
||||
the computed value.
|
||||
|
||||
Returns the **maximum** of all explicitly-set project-level values,
|
||||
falling back to :attr:`_hot_max_tokens` (the caller-passed global
|
||||
default) when no project overrides are present.
|
||||
"""
|
||||
from typing import cast
|
||||
|
||||
candidates: list[int] = []
|
||||
for namespaced_name in project_names:
|
||||
row = None
|
||||
try:
|
||||
session = self._project_repository._session() # type: ignore[attr-defined]
|
||||
from cleveragents.infrastructure.database.models import (
|
||||
NamespacedProjectModel,
|
||||
)
|
||||
|
||||
row = (
|
||||
session.query(NamespacedProjectModel)
|
||||
.filter_by(namespaced_name=namespaced_name)
|
||||
.first()
|
||||
)
|
||||
session.close()
|
||||
except AttributeError:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_session_factory_missing",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
continue
|
||||
except Exception:
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_lookup_failed",
|
||||
project_name=namespaced_name,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
if row is not None and row.context_policy_json is not None:
|
||||
try:
|
||||
config_dict = json.loads(cast(str, row.context_policy_json))
|
||||
tokens = config_dict.get("hot_max_tokens")
|
||||
if tokens is not None and isinstance(tokens, int) and tokens > 0:
|
||||
candidates.append(tokens)
|
||||
except (ValueError, TypeError):
|
||||
self._logger.warning(
|
||||
"hot_max_tokens_parse_failed",
|
||||
project_name=namespaced_name,
|
||||
)
|
||||
|
||||
if candidates:
|
||||
effective = max(candidates)
|
||||
self._logger.info(
|
||||
"hot_max_tokens_resolved_from_projects",
|
||||
project_names=project_names,
|
||||
project_values=candidates,
|
||||
effective=effective,
|
||||
)
|
||||
return effective
|
||||
# No project overrides --- use the caller-passed global default.
|
||||
return self._hot_max_tokens
|
||||
|
||||
@staticmethod
|
||||
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
|
||||
"""Return whether *path* passes include/exclude path globs.
|
||||
@@ -226,14 +293,21 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
|
||||
)
|
||||
return None
|
||||
|
||||
budget = CoreContextBudget(max_tokens=self._hot_max_tokens, reserved_tokens=0)
|
||||
# Resolve effective hot_max_tokens: project-level overrides take precedence
|
||||
# over the global default. When multiple projects have explicit values,
|
||||
# use the maximum so all projects can contribute within their biggest budget.
|
||||
effective_hot_max_tokens = self._resolve_hot_max_tokens(project_names)
|
||||
|
||||
budget = CoreContextBudget(
|
||||
max_tokens=effective_hot_max_tokens, reserved_tokens=0
|
||||
)
|
||||
request = ContextRequest(
|
||||
query=(
|
||||
f"Execute-phase context for plan {plan.identity.plan_id} "
|
||||
f"({', '.join(project_names)})"
|
||||
),
|
||||
purpose="llm_execute_phase_prompt",
|
||||
max_tokens=self._hot_max_tokens,
|
||||
max_tokens=effective_hot_max_tokens,
|
||||
)
|
||||
payload = self._pipeline.assemble(
|
||||
plan_id=plan.identity.plan_id,
|
||||
|
||||
@@ -11,8 +11,8 @@ a dict keyed by invariant ID.
|
||||
|
||||
## Merge Precedence
|
||||
|
||||
Effective invariants are computed using plan > project > global order.
|
||||
See ``merge_invariants`` for de-duplication semantics.
|
||||
Effective invariants are computed using plan > action > project > global
|
||||
order. See ``merge_invariants`` for de-duplication semantics.
|
||||
|
||||
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
|
||||
"""
|
||||
@@ -124,6 +124,7 @@ class InvariantService:
|
||||
if effective:
|
||||
return self.get_effective_invariants(
|
||||
plan_id=source_name if scope == InvariantScope.PLAN else None,
|
||||
action_name=source_name if scope == InvariantScope.ACTION else None,
|
||||
project_name=source_name if scope == InvariantScope.PROJECT else None,
|
||||
)
|
||||
|
||||
@@ -167,16 +168,22 @@ class InvariantService:
|
||||
def get_effective_invariants(
|
||||
self,
|
||||
plan_id: str | None = None,
|
||||
action_name: str | None = None,
|
||||
project_name: str | None = None,
|
||||
) -> list[Invariant]:
|
||||
"""Return the merged precedence chain for a plan/project context.
|
||||
"""Return the merged precedence chain for a plan/action/project context.
|
||||
|
||||
Collects active invariants from each scope tier and merges them
|
||||
using plan > project > global precedence.
|
||||
using plan > action > project > global precedence.
|
||||
|
||||
Args:
|
||||
plan_id: Optional plan identifier to collect plan-scoped
|
||||
invariants.
|
||||
action_name: Optional action name to collect action-scoped
|
||||
invariants (promoted to plan-level during reconciliation).
|
||||
When ``None``, the action tier is omitted for backward
|
||||
compatibility. Pass ``"*"`` to include all action-scoped
|
||||
invariants regardless of source name.
|
||||
project_name: Optional project name to collect project-scoped
|
||||
invariants.
|
||||
|
||||
@@ -191,6 +198,13 @@ class InvariantService:
|
||||
if inv.scope == InvariantScope.PLAN
|
||||
and (plan_id is None or inv.source_name == plan_id)
|
||||
]
|
||||
action_invs = [
|
||||
inv
|
||||
for inv in active
|
||||
if inv.scope == InvariantScope.ACTION
|
||||
and action_name is not None # Only include when explicitly requested
|
||||
and (inv.source_name == action_name or action_name == "*")
|
||||
]
|
||||
project_invs = [
|
||||
inv
|
||||
for inv in active
|
||||
@@ -199,7 +213,7 @@ class InvariantService:
|
||||
]
|
||||
global_invs = [inv for inv in active if inv.scope == InvariantScope.GLOBAL]
|
||||
|
||||
return merge_invariants(plan_invs, project_invs, global_invs)
|
||||
return merge_invariants(plan_invs, action_invs, project_invs, global_invs)
|
||||
|
||||
def enforce_invariants(
|
||||
self,
|
||||
|
||||
@@ -497,6 +497,116 @@ class StrategyRegistry:
|
||||
|
||||
return warnings
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Entry-point discovery
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def discover_from_entry_points(
|
||||
self, *, group: str = "cleveragents.context_strategies"
|
||||
) -> int:
|
||||
"""Discover and register strategies from Python entry points.
|
||||
|
||||
Scans the given entry-point group for strategy registrations
|
||||
(see ``pyproject.toml`` ``[project.entry-points]``). Each matching
|
||||
entry point is resolved to a class, instantiated, and registered
|
||||
with the registry as a built-in strategy.
|
||||
|
||||
Security:
|
||||
Only modules under :attr:`DEFAULT_ALLOWED_MODULE_PREFIXES` may be
|
||||
dynamically imported from external packages. Internal built-ins
|
||||
(under ``cleveragents.``) are always permitted.
|
||||
|
||||
Args:
|
||||
group: The entry-point group to scan. Defaults to
|
||||
``"cleveragents.context_strategies"``.
|
||||
|
||||
Returns:
|
||||
Number of strategies discovered and registered.
|
||||
|
||||
Example::
|
||||
|
||||
registry = StrategyRegistry()
|
||||
count = registry.discover_from_entry_points()
|
||||
# count == 6 for the six built-in strategies
|
||||
"""
|
||||
import importlib.metadata as _metadata
|
||||
|
||||
discovered = 0
|
||||
|
||||
try:
|
||||
eps = _metadata.entry_points(group=group)
|
||||
except (ValueError, TypeError): # group doesn't exist
|
||||
logger.debug(
|
||||
"strategy.discovering_no_group",
|
||||
group=group,
|
||||
)
|
||||
return 0
|
||||
|
||||
for ep in sorted(eps, key=lambda e: e.name):
|
||||
name = ep.name
|
||||
# Security (CWE-706): Enforce module allowlist BEFORE loading.
|
||||
# The entry point value is ``"module.path:ClassName"`` — extract
|
||||
# the module portion and validate against _allowed_module_prefixes.
|
||||
ep_value = str(ep.value) # e.g. "pkg.module:StrategyName"
|
||||
if ":" in ep_value:
|
||||
module_name = ep_value.split(":", 1)[0].strip()
|
||||
else:
|
||||
logger.warning(
|
||||
"strategy.discovering_bad_entry_point",
|
||||
name=name,
|
||||
value=ep_value,
|
||||
)
|
||||
continue
|
||||
|
||||
if self._allowed_module_prefixes and not any(
|
||||
module_name.startswith(prefix)
|
||||
for prefix in self._allowed_module_prefixes
|
||||
):
|
||||
logger.warning(
|
||||
"strategy.discovering_blocked",
|
||||
name=name,
|
||||
module=module_name,
|
||||
allowed=self._allowed_module_prefixes,
|
||||
)
|
||||
continue
|
||||
|
||||
try:
|
||||
loaded = ep.load()
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"strategy.discovering_failed",
|
||||
name=name,
|
||||
error=str(exc),
|
||||
)
|
||||
continue
|
||||
|
||||
# Type safety: ``ep.load()`` returns ``Any`` — narrow to a Callable.
|
||||
if not callable(loaded):
|
||||
logger.warning(
|
||||
"strategy.discovering_not_callable",
|
||||
name=name,
|
||||
type=type(loaded).__name__,
|
||||
)
|
||||
continue
|
||||
|
||||
instance = loaded()
|
||||
|
||||
self.register(
|
||||
instance,
|
||||
name=name,
|
||||
config=StrategyConfig(enabled=True),
|
||||
is_builtin=True,
|
||||
)
|
||||
discovered += 1
|
||||
|
||||
if discovered > 0:
|
||||
logger.info(
|
||||
"strategy.discovered_entry_points",
|
||||
count=discovered,
|
||||
group=group,
|
||||
)
|
||||
return discovered
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Removal (for testing / reconfiguration)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -2,9 +2,10 @@
|
||||
|
||||
Invariants are natural-language constraints on plan execution, scoped at
|
||||
global, project, action, or plan level. When an action is used, its
|
||||
invariants are promoted to plan-level. The runtime precedence chain is:
|
||||
invariants are promoted to plan-level and participate in the merge.
|
||||
The runtime precedence chain is:
|
||||
|
||||
plan > project > global
|
||||
plan > action > project > global
|
||||
|
||||
They are reconciled by the Invariant Reconciliation Actor at the start
|
||||
of Strategize and recorded as ``invariant_enforced`` decisions.
|
||||
@@ -21,8 +22,8 @@ of Strategize and recorded as ``invariant_enforced`` decisions.
|
||||
## Merge Precedence
|
||||
|
||||
When computing the effective set of invariants for a plan, the merge
|
||||
order is **plan > project > global**. Duplicate texts (case-insensitive)
|
||||
are de-duplicated, keeping the highest-precedence copy.
|
||||
order is **plan > action > project > global**. Duplicate texts
|
||||
(case-insensitive) are de-duplicated, keeping the highest-precedence copy.
|
||||
|
||||
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
|
||||
"""
|
||||
@@ -39,8 +40,10 @@ from ulid import ULID
|
||||
class InvariantScope(StrEnum):
|
||||
"""Scope at which an invariant applies.
|
||||
|
||||
Precedence (highest to lowest): PLAN > PROJECT > GLOBAL.
|
||||
ACTION invariants are promoted to PLAN scope at ``plan use`` time.
|
||||
Precedence (highest to lowest): PLAN > ACTION > PROJECT > GLOBAL.
|
||||
ACTION invariants are promoted to plan-level and participate in the
|
||||
merge during reconciliation, sitting between PLAN and PROJECT in the
|
||||
precedence chain.
|
||||
"""
|
||||
|
||||
GLOBAL = "global"
|
||||
@@ -137,10 +140,11 @@ class InvariantSet(BaseModel):
|
||||
def merge(
|
||||
cls,
|
||||
plan_invariants: list[Invariant],
|
||||
project_invariants: list[Invariant],
|
||||
global_invariants: list[Invariant],
|
||||
action_invariants: list[Invariant] | None = None,
|
||||
project_invariants: list[Invariant] | None = None,
|
||||
global_invariants: list[Invariant] | None = None,
|
||||
) -> InvariantSet:
|
||||
"""Merge invariants respecting plan > project > global precedence.
|
||||
"""Merge invariants respecting plan > action > project > global precedence.
|
||||
|
||||
De-duplicates by text (case-insensitive), keeping the copy from
|
||||
the highest-precedence tier. Within each tier, source ordering
|
||||
@@ -148,6 +152,9 @@ class InvariantSet(BaseModel):
|
||||
|
||||
Args:
|
||||
plan_invariants: Plan-level invariants (highest precedence).
|
||||
action_invariants: Action-scoped invariants (second-highest
|
||||
precedence; promoted to plan-level when an
|
||||
action is used).
|
||||
project_invariants: Project-level invariants.
|
||||
global_invariants: Global-level invariants (lowest precedence).
|
||||
|
||||
@@ -156,7 +163,12 @@ class InvariantSet(BaseModel):
|
||||
"""
|
||||
return cls(
|
||||
invariants=tuple(
|
||||
merge_invariants(plan_invariants, project_invariants, global_invariants)
|
||||
merge_invariants(
|
||||
plan_invariants,
|
||||
action_invariants or [],
|
||||
project_invariants or [],
|
||||
global_invariants or [],
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -165,10 +177,11 @@ class InvariantSet(BaseModel):
|
||||
|
||||
def merge_invariants(
|
||||
plan_invariants: list[Invariant],
|
||||
project_invariants: list[Invariant],
|
||||
global_invariants: list[Invariant],
|
||||
action_invariants: list[Invariant] | None = None,
|
||||
project_invariants: list[Invariant] | None = None,
|
||||
global_invariants: list[Invariant] | None = None,
|
||||
) -> list[Invariant]:
|
||||
"""Merge invariants implementing plan > project > global precedence.
|
||||
"""Merge invariants implementing plan > action > project > global precedence.
|
||||
|
||||
De-duplicates by text (case-insensitive). The first occurrence
|
||||
(from the highest-precedence tier) wins. Within each tier, the
|
||||
@@ -176,8 +189,13 @@ def merge_invariants(
|
||||
|
||||
Args:
|
||||
plan_invariants: Plan-level invariants (highest precedence).
|
||||
project_invariants: Project-level invariants.
|
||||
action_invariants: Action-scoped invariants (second-highest
|
||||
precedence; promoted to plan level when an action is used).
|
||||
Defaults to empty list when ``None``.
|
||||
project_invariants: Project-level invariants. Defaults to empty
|
||||
list when ``None``.
|
||||
global_invariants: Global-level invariants (lowest precedence).
|
||||
Defaults to empty list when ``None``.
|
||||
|
||||
Returns:
|
||||
A de-duplicated list of invariants in precedence order.
|
||||
@@ -185,7 +203,12 @@ def merge_invariants(
|
||||
seen: set[str] = set()
|
||||
result: list[Invariant] = []
|
||||
|
||||
for inv_list in (plan_invariants, project_invariants, global_invariants):
|
||||
for inv_list in (
|
||||
plan_invariants,
|
||||
action_invariants or [],
|
||||
project_invariants or [],
|
||||
global_invariants or [],
|
||||
):
|
||||
for inv in inv_list:
|
||||
if not inv.active:
|
||||
continue
|
||||
|
||||
@@ -42,6 +42,7 @@ _SENSITIVE_SUBSTRINGS: set[str] = {
|
||||
# though they contain a sensitive substring (e.g. "token_count").
|
||||
_FALSE_POSITIVE_KEYS: set[str] = {
|
||||
"token_count",
|
||||
"token_prefix",
|
||||
"token_limit",
|
||||
"token_usage",
|
||||
"input_tokens",
|
||||
|
||||
Reference in New Issue
Block a user