Compare commits

...

13 Commits

Author SHA1 Message Date
HAL9000 dfa701b354 fix(cli): wrap plan status --format json output in spec-required JSON envelope 2026-05-17 02:54:16 +00:00
HAL9000 f79c18a3ec fix(data-integrity): remove silent argument swap in ValidationAttachmentRepository.attach (#8177)
Fixed a critical data integrity bug where validation_name and resource_id
arguments were being silently swapped based on the fragile heuristic
('/' in resource_id and '/' not in validation_name). This caused silent
data corruption without any error or warning. The conditional swap block
has been removed, ensuring arguments flow directly from caller to the
persistence layer in their declared order.
2026-05-16 08:14:39 +00:00
HAL9000 5c5309f35d fix(ci): add missing effective set count step definition
CI / lint (push) Successful in 50s
CI / benchmark-regression (push) Failing after 46s
CI / typecheck (push) Successful in 1m21s
CI / security (push) Successful in 1m20s
CI / quality (push) Successful in 1m3s
CI / build (push) Successful in 35s
CI / helm (push) Successful in 32s
CI / integration_tests (push) Successful in 3m22s
CI / push-validation (push) Successful in 33s
CI / e2e_tests (push) Successful in 55s
CI / unit_tests (push) Successful in 5m49s
CI / docker (push) Successful in 2m1s
CI / coverage (push) Successful in 13m13s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Failing after 24m24s
The Behave scenario at line 671 of consolidated_domain_models.feature
asserts 'the effective set should have {count:d} invariants' but no
step handler existed, causing UndefinedStepError and CI failure.

Adds the missing step: @then('the effective set should have {count:d} invariants')
to step_invariant_models_steps.py, mirroring the existing invariant-set count pattern.
2026-05-16 04:14:17 +00:00
HAL9000 f4cea72248 fix(ci): update all merge_invariants callers to use 4-param signature
The PR #11143 adds action_invariants as a 4th parameter to
merge_invariants() and InvariantSet.merge(), but two call sites
were not updated:

- benchmarks/invariant_merge_bench.py: 5 calls with 3 positional args
- robot/helper_m3_e2e_verification.py: 2 calls using keyword args

All call sites now pass action_invariants=[] for backward-compatible
empty-action behavior.
2026-05-16 04:14:17 +00:00
HAL9000 761622f746 fix(invariant): pass action_name in list_invariants when scope is ACTION
Fixes list_invariants(effective=True) to forward action_name to
get_effective_invariants when scope is ACTION, ensuring action-scoped
invariants are included in effective invariant lists.

Also applies ruff formatting to the return statement in get_effective_invariants.

Addresses reviewer observation about list_invariants gap.

Refs: #9126
2026-05-16 04:14:17 +00:00
freemo bf52a9c648 fix(invariant): restore ACTION scope in merge_invariants and InvariantSet.merge precedence chain
The 4-tier invariant precedence chain (plan > action > project > global) was
broken at the domain layer — merge_invariants() and InvariantSet.merge() only
accepted 3 parameters (plan, project, global), silently dropping all action-
scoped invariants. Added action_invariants as a fourth parameter with proper
backward compatibility (default to empty list). Updated module docstrings,
InvariantScope docstring, and InvariantService.get_effective_invariants() to
reflect the correct precedence chain. Added comprehensive BDD test scenarios
covering four-tier merge precedence, action-before-project ordering, and effective
invariant computation with all four scopes.

ISSUES CLOSED: #9126
2026-05-16 04:14:17 +00:00
CoreRasurae 97c1007bb5 feat(events): wire domain services to emit missing EventBus events
CI / push-validation (pull_request) Successful in 48s
CI / build (pull_request) Successful in 1m17s
CI / helm (pull_request) Successful in 1m0s
CI / lint (pull_request) Successful in 1m54s
CI / quality (pull_request) Successful in 1m55s
CI / security (pull_request) Successful in 2m5s
CI / typecheck (pull_request) Successful in 2m4s
CI / integration_tests (pull_request) Successful in 5m1s
CI / unit_tests (pull_request) Successful in 5m10s
CI / docker (pull_request) Successful in 1m51s
CI / coverage (pull_request) Successful in 12m47s
CI / status-check (pull_request) Successful in 8s
CI / benchmark-publish (push) Has started running
CI / benchmark-regression (push) Failing after 1m15s
CI / typecheck (push) Has started running
CI / security (push) Has started running
CI / quality (push) Has started running
CI / integration_tests (push) Has started running
CI / unit_tests (push) Has started running
CI / e2e_tests (push) Has started running
CI / lint (push) Successful in 56s
CI / helm (push) Successful in 53s
CI / push-validation (push) Successful in 54s
CI / build (push) Successful in 1m24s
CI / coverage (push) Blocked by required conditions
CI / docker (push) Blocked by required conditions
CI / status-check (push) Blocked by required conditions
Add TokenAuthMiddleware to emit AUTH_SUCCESS/AUTH_FAILURE with spec-aligned audit details and wire it through the DI container using server.token resolution.

Add Behave and Robot coverage for auth event emission and end-to-end audit persistence, and update audit subscriber producer notes and changelog.

ISSUES CLOSED: #714
2026-05-15 22:19:57 +01:00
hamza.khyari b0b28623a1 Merge pull request 'revert(acms): revert PR #11216 — _resolve_effective_budget reads wrong attribute' (#11228) from bugfix/m5-revert-acms-budget-assembler into master
CI / push-validation (push) Successful in 33s
CI / helm (push) Successful in 44s
CI / build (push) Successful in 1m13s
CI / lint (push) Successful in 1m21s
CI / quality (push) Successful in 1m43s
CI / e2e_tests (push) Successful in 1m46s
CI / typecheck (push) Successful in 1m56s
CI / security (push) Successful in 1m55s
CI / benchmark-regression (push) Failing after 1m30s
CI / integration_tests (push) Successful in 3m54s
CI / unit_tests (push) Successful in 6m50s
CI / docker (push) Successful in 1m29s
CI / coverage (push) Successful in 11m1s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 1h25m33s
CI / push-validation (pull_request) Successful in 48s
CI / helm (pull_request) Successful in 1m3s
CI / lint (pull_request) Successful in 1m48s
CI / build (pull_request) Successful in 1m34s
CI / typecheck (pull_request) Successful in 1m54s
CI / quality (pull_request) Successful in 1m47s
CI / security (pull_request) Successful in 2m2s
CI / integration_tests (pull_request) Failing after 21m0s
CI / unit_tests (pull_request) Failing after 21m2s
CI / coverage (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
Reviewed-on: #11228
Reviewed-by: Rui Hu <rui.hu@cleverthis.com>
2026-05-15 11:47:30 +00:00
hamza.khyari b4351ca78d Revert "Merge pull request 'fix(acms): use project-level hot_max_tokens in execute phase context assembly' (#11216) from bugfix/m5-acms-project-budget-override into master"
CI / push-validation (pull_request) Successful in 41s
CI / helm (pull_request) Successful in 50s
CI / build (pull_request) Successful in 1m13s
CI / lint (pull_request) Successful in 1m34s
CI / typecheck (pull_request) Successful in 1m40s
CI / quality (pull_request) Successful in 1m45s
CI / security (pull_request) Successful in 1m50s
CI / integration_tests (pull_request) Successful in 3m52s
CI / unit_tests (pull_request) Successful in 8m50s
CI / docker (pull_request) Successful in 1m39s
CI / coverage (pull_request) Successful in 12m1s
CI / status-check (pull_request) Successful in 6s
This reverts commit 94622f467c, reversing
changes made to 1196c726f2.
2026-05-15 11:15:41 +00:00
hamza.khyari 94622f467c Merge pull request 'fix(acms): use project-level hot_max_tokens in execute phase context assembly' (#11216) from bugfix/m5-acms-project-budget-override into master
CI / lint (push) Successful in 1m4s
CI / benchmark-publish (push) Has started running
CI / helm (push) Successful in 51s
CI / push-validation (push) Successful in 50s
CI / build (push) Successful in 1m13s
CI / quality (push) Successful in 1m29s
CI / benchmark-regression (push) Failing after 47s
CI / typecheck (push) Successful in 1m48s
CI / e2e_tests (push) Successful in 1m28s
CI / security (push) Successful in 2m7s
CI / integration_tests (push) Successful in 5m35s
CI / unit_tests (push) Successful in 5m39s
CI / docker (push) Successful in 1m38s
CI / coverage (push) Successful in 11m2s
CI / status-check (push) Successful in 2s
Reviewed-on: #11216
Reviewed-by: HAL9001 <hal9001@cleverthis.com>
2026-05-15 10:49:48 +00:00
hamza.khyari 1baa888659 fix(acms): use project-level hot_max_tokens in execute phase context assembly
CI / lint (pull_request) Successful in 57s
CI / typecheck (pull_request) Successful in 1m24s
CI / quality (pull_request) Successful in 1m16s
CI / build (pull_request) Successful in 40s
CI / security (pull_request) Successful in 1m28s
CI / integration_tests (pull_request) Successful in 6m2s
CI / helm (pull_request) Successful in 24s
CI / push-validation (pull_request) Successful in 28s
CI / unit_tests (pull_request) Successful in 7m13s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 11m50s
CI / status-check (pull_request) Successful in 7s
Added _resolve_effective_budget() method to
ACMSExecutePhaseContextAssembler that reads each linked project's
settings.hot_max_tokens and uses the maximum override value as the
pipeline budget instead of the hardcoded global default.

Updated assemble() to use the resolved effective budget for both
CoreContextBudget and ContextRequest.

Added Behave regression scenario with @tdd_issue @tdd_issue_11035 tags
verifying the pipeline receives the project-level budget.

Removed test artifact ANALYSIS.md left over from scenario 20.

Fixes: #11035
Fixes: #11215
2026-05-15 10:27:57 +00:00
HAL9000 1196c726f2 fix(lint): remove broken step file and fix line-length violation
CI / helm (pull_request) Successful in 56s
CI / push-validation (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 1m59s
CI / lint (pull_request) Successful in 2m21s
CI / typecheck (pull_request) Successful in 2m49s
CI / quality (pull_request) Successful in 2m56s
CI / security (pull_request) Successful in 3m21s
CI / integration_tests (pull_request) Successful in 5m36s
CI / unit_tests (pull_request) Successful in 9m7s
CI / docker (pull_request) Successful in 1m49s
CI / coverage (pull_request) Successful in 16m22s
CI / status-check (pull_request) Successful in 4s
CI / build (push) Successful in 1m7s
CI / helm (push) Successful in 39s
CI / push-validation (push) Successful in 35s
CI / lint (push) Successful in 1m36s
CI / typecheck (push) Successful in 1m51s
CI / quality (push) Successful in 1m55s
CI / security (push) Successful in 2m8s
CI / e2e_tests (push) Successful in 1m7s
CI / benchmark-regression (push) Failing after 1m22s
CI / integration_tests (push) Failing after 17m27s
CI / unit_tests (push) Failing after 17m29s
CI / benchmark-publish (push) Successful in 1h37m6s
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
- Removed features/steps/plan_status_json_envelope_steps.py which contained
  invalid Python syntax (dummy content, not referenced by any feature)
- Fixed line-length violation in execute_phase_context_assembler.py
  (CoreContextBudget constructor call exceeded 88-char limit)
2026-05-15 03:41:13 +00:00
HAL9000 655cd7ebc2 fix: persist strategy decisions via DecisionService during strategize (#10813)
The plan tree command reported zero decision nodes after strategize because
PlanExecutor.run_strategize() never persisted strategy decisions as domain
Decision objects. Added _persist_strategy_decisions() to the PlanExecutor,
wired decision_service from the DI container in _get_plan_executor(), and
ensure each strategy decision is recorded with correct DecisionType mapping.

ISSUES CLOSED: #10813
2026-05-15 03:41:13 +00:00
20 changed files with 1171 additions and 47 deletions
+19 -2
View File
@@ -5,6 +5,8 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
Changed `wf10_batch.robot` to be less likely to create files, and
`plan_generation_graph.robot` to give more test answers.
## [Unreleased]
- Hardened the TDD bug-fix quality gate for issue #629: PR parsing now
requires whole-word closing keywords (avoids false positives like
"prefixes #12"), TDD bug tag discovery now uses exact token matching
@@ -40,6 +42,20 @@ Changed `wf10_batch.robot` to be less likely to create files, and
cleanup conflict with `cli_init_yes_flag_steps.py`); 2 new Behave
scenarios covering multi-line PR description parsing and non-string
`pr_diff` type guard.
- Added `TokenAuthMiddleware` and DI wiring to emit missing auth domain
events (`AUTH_SUCCESS`, `AUTH_FAILURE`) during token checks, including
spec-aligned audit details (`user_identity`, `attempted_identity`,
`ip_address`, `token_prefix`, `failure_reason`) and best-effort publish
behavior. Auth token prefixes now persist to audit logs without
redaction-key collisions, and short tokens are masked (`***...`) to
prevent full token disclosure. Added Behave coverage and Robot
audit-pipeline integration tests for auth event persistence. (#714)
- Added TDD bug-capture E2E tests for bug #1028 — ACMS indexing pipeline not
wired into CLI. Four Robot Framework E2E tests prove ContextTierService starts
empty on every CLI invocation. Tests use ``@tdd_expected_fail`` until the bug
fix is merged. (#1029)
- Added Fix-then-Revalidate orchestration loop for required validations:
bounded retry with configurable limits (0--100 per Safety Profile),
strategy revision escalation via `auto_strategy_revision` float
@@ -52,7 +68,6 @@ Changed `wf10_batch.robot` to be less likely to create files, and
counter, spec-required `validation_summary` and
`final_validation_results` fields on the result model, DI container
## [Unreleased]
- **fix(tui): rename ActorSelectionOverlay._render to _refresh_display (issue #11039)** — `ActorSelectionOverlay._render()` shadows Textual's `Widget._render()` which must return a `Strip`. In textual >=1.0, layout calls `get_content_height()` `self._render()` gets `None` `AttributeError: 'NoneType' object has no attribute 'get_height'`. Renamed the method to `_refresh_display()` and updated all four internal call sites (`show()`, `move_up()`, `move_down()`, `set_search()`) to use the new name.
- **Structural Component Output Validation** (#8164): Replaces exact character matching with structural component checking for output validation. Implements three validators covering plan tree output, decision CLI dicts, and structured session snapshots. The `validate_plan_tree` function validates node dicts for required keys (`decision_id`, `type`, `sequence`, `question`, `children`), ULID format, correct types, and sibling ordering. The `validate_decision_dict` function validates decision CLI output against the `Decision.as_cli_dict()` schema with field presence, type, ULID pattern, confidence range [0..1], and boolean field checks. The `validate_structured_output` function validates the StructuredOutput envelope for `command`, `session_id` (ULID), status membership, `exit_code`, and elements integrity. A unified dispatcher (`validate_structured_component_output`) enables routing by target_type. BDD test coverage added in `features/structural_validation.feature`. [Epic #8137](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8137)
@@ -64,6 +79,7 @@ Changed `wf10_batch.robot` to be less likely to create files, and
when separate `provider`/`model` keys are absent. Added validation to reject malformed
combined values with empty provider or model halves.
- **Fixed plan tree reporting zero decision nodes after strategize** (#10813): The `plan tree` command showed no ``decision_id`` fields even though planning completed successfully. Root cause: the PlanExecutor's ``run_strategize()`` method produced strategy decisions via the StrategyActor but never persisted them as domain ``Decision`` objects through the DecisionService wiring. Added ``decision_service`` parameter to the PlanExecutor constructor, wired it from the CLI dependency-injection container in ``_get_plan_executor()``, and added ``_persist_strategy_decisions()`` that converts each strategy decision into a domain Decision with correct type mapping (prompt_definition, strategy_choice, subplan_spawn) so they appear in plan tree output.
- **`task-implementor` posts work-started notification comments** (#11031): Both
the `issue_impl` and `pr_fix` procedures now post an informational "work
started" comment to the Forgejo issue/PR before beginning implementation.
@@ -185,7 +201,8 @@ Changed `wf10_batch.robot` to be less likely to create files, and
were cleaned up in `features/actor_add_update_enforcement.feature` so the
tests report correctly now that the underlying bug has been fixed.
- **Resolved Behave AmbiguousStep collisions in step definitions** (#4186): Renamed
- **Fixed `merge_invariants()` missing ACTION scope — 4-tier precedence restored** (#9126): Updated ``merge_invariants()`` and ``InvariantSet.merge()`` to accept a fourth parameter ``action_invariants`` alongside plan, project, and global tiers. The module docstring, ``InvariantScope`` docstring, and ``InvariantService.get_effective_invariants()`` now all reflect the correct precedence chain: ``plan > action > project > global``. Added ``action_name`` parameter to ``get_effective_invariants()`` so action-scoped invariants are collected and passed through the merge pipeline instead of silently dropped. All docstrings across both files were corrected from ``plan > project > global`` to ``plan > action > project > global``. Comprehensive Behave scenarios added covering four-tier merge precedence, action-before-project ordering, action override of project with same text, and effective invariant computation with all four scopes.
step texts to avoid case-sensitive collisions between different step modules that
prevented all Behave tests from loading. Renamed steps in
`edge_case_plan_steps.py`, `plan_executor_coverage_boost_steps.py`,
+3 -1
View File
@@ -8,13 +8,14 @@
* Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
* Luis Mendes <luis.p.mendes@gmail.com>
* Rui Hu <rui.hu@cleverthis.com>
* HAL 9000 <hal9000@cleverthis.com> has contributed fix for #10813 — wiring DecisionService into PlanExecutor for strategy decision persistence during strategize.
# Details
Below are some of the specific details of various contributions.
* Jeffrey Phillips Freeman has acted as Lead Developer, daily contributor, and Project Owner.
* Jeffrey Phillips Freeman has contributed an implementation for the invariant propagation fix (PR #10881 / issue #9131): added `_propagate_invariant_decisions()` to `SubplanService` to propagate all `invariant_enforced` decisions from parent plans to child plan decision trees during subplan spawn, satisfying the specification requirement for invariant propagation across hierarchical plan execution.
* Jeffrey Phillips Freeman has contributed the invariant merge precedence fix (#9126): restored the missing ACTION scope in ``merge_invariants()`` and ``InvariantSet.merge()``, corrected all module docstrings from ``plan > project > global`` to the spec-compliant ``plan > action > project > global``, and added comprehensive BDD test coverage for four-tier merge precedence.
* Brent E. Edwards has contributed quality assurance, test coverage, and CI pipeline improvements.
* HAL 9000 has contributed automated implementation, bug fixes, and feature development as part of the CleverAgents automation pool.
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
@@ -44,5 +45,6 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
* HAL 9000 has contributed the DecisionService wiring for PlanExecutor strategize persistence fix (#10813): added decision_service to the PlanExecutor constructor and wired it from the CLI dependency-injection container in `_get_plan_executor()`, plus implemented `_persist_strategy_decisions()` to persist strategy decisions as domain `Decision` objects.
* HAL 9000 has contributed the A2A module rename standardization BDD tests (PR #10583 / issue #8615): comprehensive Behave test suite validating that all 22 A2A symbols are properly exported from `cleveragents.a2a`, no legacy ACP references remain in the module source, and documentation uses correct A2A naming conventions — fixing inline imports, unused behave symbols, cross-scenario context dependencies, and missing type annotations.
* HAL 9000 has contributed the `ActorSelectionOverlay._render``_refresh_display` rename fix (PR #11176 / issue #11039, Epic #8174): renamed `_render()` method to `_refresh_display()` to avoid shadowing Textual's `Widget._render()`, fixing a crash in textual >=1.0 where `get_content_height()` would receive `None` and raise `AttributeError: 'NoneType' object has no attribute 'get_height'`.
+5 -5
View File
@@ -74,7 +74,7 @@ class MergeSmallSuite:
def time_merge_small(self) -> None:
"""Benchmark merge with ~13 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeMediumSuite:
@@ -88,7 +88,7 @@ class MergeMediumSuite:
def time_merge_medium(self) -> None:
"""Benchmark merge with ~50 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeLargeSuite:
@@ -102,7 +102,7 @@ class MergeLargeSuite:
def time_merge_large(self) -> None:
"""Benchmark merge with ~250 invariants."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class MergeDeduplicationSuite:
@@ -137,7 +137,7 @@ class MergeDeduplicationSuite:
def time_merge_dedup(self) -> None:
"""Benchmark merge with 60 invariants, all duplicates."""
merge_invariants(self.plan, self.project, self.global_invs)
merge_invariants(self.plan, [], self.project, self.global_invs)
class InvariantSetMergeSuite:
@@ -151,7 +151,7 @@ class InvariantSetMergeSuite:
def time_invariant_set_merge(self) -> None:
"""Benchmark InvariantSet.merge()."""
InvariantSet.merge(self.plan, self.project, self.global_invs)
InvariantSet.merge(self.plan, [], self.project, self.global_invs)
class ServiceEffectiveSuite:
+63
View File
@@ -0,0 +1,63 @@
Feature: Auth middleware emits security events
As a platform operator
I want authentication outcomes emitted as domain events
So the audit pipeline captures auth success and failure details
Scenario: Successful authentication emits AUTH_SUCCESS
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I authenticate with token "tok_secret_123" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be true
And the auth event bus should have exactly 1 event
And the latest auth event type should be AUTH_SUCCESS
And the latest auth event should contain detail "user_identity" with value "alice@example.com"
And the latest auth event should contain detail "ip_address" with value "10.0.0.5"
And the latest auth event should contain detail "token_prefix" with value "tok_se..."
Scenario: Short token success masks token_prefix
Given an auth middleware with expected token "short"
And an auth recording event bus
When I authenticate with token "short" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be true
And the latest auth event type should be AUTH_SUCCESS
And the latest auth event should contain detail "token_prefix" with value "***..."
Scenario: Failed authentication emits AUTH_FAILURE
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I authenticate with token "tok_wrong_999" identity "alice@example.com" and ip "10.0.0.5"
Then the auth result should be false
And the auth event bus should have exactly 1 event
And the latest auth event type should be AUTH_FAILURE
And the latest auth event should contain detail "attempted_identity" with value "alice@example.com"
And the latest auth event should contain detail "failure_reason" with value "invalid_token"
And the latest auth event should contain detail "token_prefix" with value "tok_wr..."
Scenario: Missing configured token emits AUTH_FAILURE
Given an auth middleware with no configured token
And an auth recording event bus
When I authenticate with token "tok_any_111" identity "bob@example.com" and ip "10.0.0.9"
Then the auth result should be false
And the latest auth event type should be AUTH_FAILURE
And the latest auth event should contain detail "failure_reason" with value "token_not_configured"
Scenario: Empty authentication token is rejected before emission
Given an auth middleware with expected token "tok_secret_123"
And an auth recording event bus
When I try to authenticate with an empty token
Then an auth ValueError should be raised
And the auth event bus should have exactly 0 event
Scenario: Auth middleware events persist through AuditEventSubscriber
Given auth middleware is wired to AuditEventSubscriber with expected token "tok_secret_123"
When I authenticate with token "tok_secret_123" identity "carol@example.com" and ip "10.1.2.3"
And I authenticate with token "tok_bad_123" identity "dave@example.com" and ip "10.1.2.4"
Then the auth result history should be "true,false"
And the audit log should contain 1 auth_success entry from middleware
And the latest auth_success audit entry should contain detail "ip_address" with value "10.1.2.3"
And the latest auth_success audit entry should contain detail "token_prefix" with value "tok_se..."
And the audit log should contain 1 auth_failure entry from middleware
And the latest auth_failure audit entry should contain detail "ip_address" with value "10.1.2.4"
And the latest auth_failure audit entry should contain detail "attempted_identity" with value "dave@example.com"
And the latest auth_failure audit entry should contain detail "failure_reason" with value "invalid_token"
And the latest auth_failure audit entry should contain detail "token_prefix" with value "tok_ba..."
+100 -3
View File
@@ -418,7 +418,7 @@ Feature: Consolidated Domain Models
Scenario: InvariantScope has four values
Then InvariantScope should have values "global, project, action, plan"
# === Merge Precedence (plan > project > global) ===
# === Merge Precedence (plan > action > project > global) ===
Scenario: Merge with no duplicates preserves all invariants
@@ -480,6 +480,75 @@ Feature: Consolidated Domain Models
And the merged invariant at index 0 should have text "LOG ALL CHANGES"
Scenario: Merge with all four scopes preserves action tier — Issue #9126
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
And I have action invariants
| text | source |
| Action rule | action1 |
And I have project invariants
| text | source |
| Project rule | proj1 |
And I have global invariants
| text | source |
| Global rule | system |
When I merge the invariants
Then the merged set should have 4 invariants
And plan invariants appear before action invariants in merge
And action invariants appear before project invariants in merge
And the merged invariant at index 0 should have text "Plan rule"
And the merged invariant at index 1 should have text "Action rule"
And the merged invariant at index 2 should have text "Project rule"
And the merged invariant at index 3 should have text "Global rule"
Scenario: Action invariant overrides project with same text
Given I have plan invariants
| text | source |
And I have action invariants
| text | source |
| Log all changes | action1 |
And I have project invariants
| text | source |
| Log all changes | proj1 |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "action"
Scenario: Action invariant is included in de-duplication with plan override
Given I have plan invariants
| text | source |
| Log all changes | plan1 |
And I have action invariants
| text | source |
| Log all changes | action1 |
And I have project invariants
| text | source |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "plan"
Scenario: Inactive action invariants are excluded from merge
Given I have plan invariants with an inactive entry
And I have action invariants
| text | source |
| Active | action1 |
And I have project invariants
| text | source |
And I have global invariants
| text | source |
When I merge the invariants
Then the merged set should have 1 invariants
And the merged invariant at index 0 should have scope "action"
Scenario: Inactive invariants are excluded from merge
Given I have plan invariants with an inactive entry
And I have project invariants
@@ -492,7 +561,7 @@ Feature: Consolidated Domain Models
# === InvariantSet merge class method ===
Scenario: InvariantSet.merge produces correct result
Scenario: InvariantSet.merge produces correct result
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
@@ -505,7 +574,25 @@ Feature: Consolidated Domain Models
When I merge using InvariantSet
Then the invariant set should have 3 invariants
# === Service: Add/List/Remove ===
Scenario: InvariantSet.merge with four-tier precedence — Issue #9126
Given I have plan invariants
| text | source |
| Plan rule | plan1 |
And I have action invariants
| text | source |
| Action rule | action1 |
And I have project invariants
| text | source |
| Project rule | proj1 |
And I have global invariants
| text | source |
| Global rule | system |
When I merge using InvariantSet
Then the invariant set should have 4 invariants
# === Service: Add/List/Remove ===
Scenario: Service add invariant
@@ -574,6 +661,16 @@ Feature: Consolidated Domain Models
And the effective set should contain global invariants last
Scenario: Effective invariants include action scope — Issue #9126
Given an invariant service with invariants at all four scopes
When I get effective invariants for plan "plan1", action "action1", and project "proj1"
Then the effective set should contain plan invariants first
And the effective set should contain action invariants second
And the effective set should contain project invariants after action
And the effective set should contain global invariants last
And the effective set should have 4 invariants
Scenario: Effective invariants de-duplicate across scopes
Given an invariant service with duplicate text across scopes
When I get effective invariants for plan "plan1" and project "proj1"
+2
View File
@@ -9,6 +9,7 @@ from .fake_provider import FakeProviderInfo, FakeProviderRegistry
from .lsp_transport_mock import MockLspTransport, parse_lsp_responses
from .mock_ai_provider import MockAIProvider
from .mock_mcp_transport import MockMCPTransport
from .recording_event_bus import RecordingEventBus
from .transient_fail_audit_service import TransientFailAuditService
# NOTE: tdd_test_helpers is NOT re-exported here because it imports
@@ -22,6 +23,7 @@ __all__ = [
"MockAIProvider",
"MockLspTransport",
"MockMCPTransport",
"RecordingEventBus",
"TransientFailAuditService",
"parse_lsp_responses",
]
+25
View File
@@ -0,0 +1,25 @@
"""Minimal in-memory event bus used for Behave assertions."""
from __future__ import annotations
from collections.abc import Callable
from cleveragents.infrastructure.events.models import DomainEvent
from cleveragents.infrastructure.events.types import EventType
class RecordingEventBus:
"""Minimal in-memory event bus used for Behave assertions."""
def __init__(self) -> None:
self.events: list[DomainEvent] = []
def emit(self, event: DomainEvent) -> None:
self.events.append(event)
def subscribe(
self,
event_type: EventType,
handler: Callable[[DomainEvent], None],
) -> None:
_ = (event_type, handler)
@@ -0,0 +1,165 @@
"""Step definitions for auth_middleware_events.feature."""
from __future__ import annotations
from behave import given, then, when
from behave.runner import Context
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from cleveragents.application.services.audit_event_subscriber import (
AuditEventSubscriber,
)
from cleveragents.application.services.audit_service import AuditService
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
from cleveragents.config.settings import Settings
from cleveragents.infrastructure.database.models import Base
from cleveragents.infrastructure.events.reactive import ReactiveEventBus
from cleveragents.infrastructure.events.types import EventType
from features.mocks.recording_event_bus import RecordingEventBus
def _fresh_audit_service() -> AuditService:
Settings._instance = None
settings = Settings(database_url="sqlite:///:memory:")
engine = create_engine("sqlite:///:memory:", echo=False)
Base.metadata.create_all(engine)
session = sessionmaker(bind=engine)()
return AuditService(settings=settings, session=session)
@given('an auth middleware with expected token "{token}"')
def step_auth_middleware_with_token(context: Context, token: str) -> None:
context.expected_token = token
@given("an auth middleware with no configured token")
def step_auth_middleware_no_token(context: Context) -> None:
context.expected_token = None
@given("an auth recording event bus")
def step_auth_recording_event_bus(context: Context) -> None:
context.event_bus = RecordingEventBus()
context.auth_middleware = TokenAuthMiddleware(
expected_token=context.expected_token,
event_bus=context.event_bus,
)
context.auth_result_history = []
@given('auth middleware is wired to AuditEventSubscriber with expected token "{token}"')
def step_auth_middleware_audit_pipeline(context: Context, token: str) -> None:
context.audit_service = _fresh_audit_service()
context.event_bus = ReactiveEventBus()
context.audit_subscriber = AuditEventSubscriber(
audit_service=context.audit_service,
event_bus=context.event_bus,
)
context.auth_middleware = TokenAuthMiddleware(
expected_token=token,
event_bus=context.event_bus,
)
context.auth_result_history = []
@when('I authenticate with token "{token}" identity "{identity}" and ip "{ip_address}"')
def step_authenticate(
context: Context, token: str, identity: str, ip_address: str
) -> None:
context.auth_result = context.auth_middleware.authenticate(
token,
identity=identity,
ip_address=ip_address,
)
context.auth_result_history.append(context.auth_result)
@when("I try to authenticate with an empty token")
def step_try_authenticate_empty_token(context: Context) -> None:
context.auth_error = None
try:
context.auth_middleware.authenticate("", identity="empty@example.com")
except ValueError as exc:
context.auth_error = exc
@then("the auth result should be true")
def step_auth_result_true(context: Context) -> None:
assert context.auth_result is True
@then("the auth result should be false")
def step_auth_result_false(context: Context) -> None:
assert context.auth_result is False
@then("the auth event bus should have exactly {count:d} event")
def step_auth_event_count(context: Context, count: int) -> None:
assert len(context.event_bus.events) == count, (
f"Expected {count} events, got {len(context.event_bus.events)}"
)
@then("the latest auth event type should be {event_type_name}")
def step_latest_auth_event_type(context: Context, event_type_name: str) -> None:
expected = EventType[event_type_name]
actual = context.event_bus.events[-1].event_type
assert actual == expected, f"Expected event type {expected}, got {actual}"
@then(
'the latest auth event should contain detail "{key}" with value "{expected_value}"'
)
def step_latest_auth_event_detail(
context: Context, key: str, expected_value: str
) -> None:
details = context.event_bus.events[-1].details
assert details.get(key) == expected_value, (
f"Expected {key}={expected_value!r}, got {details.get(key)!r}"
)
@then("an auth ValueError should be raised")
def step_auth_value_error(context: Context) -> None:
assert context.auth_error is not None, (
"Expected ValueError but no exception was raised"
)
assert isinstance(context.auth_error, ValueError), (
f"Expected ValueError, got {type(context.auth_error).__name__}"
)
@then('the auth result history should be "{history_csv}"')
def step_auth_result_history(context: Context, history_csv: str) -> None:
expected = [part.strip().lower() == "true" for part in history_csv.split(",")]
assert context.auth_result_history == expected, (
f"Expected history {expected}, got {context.auth_result_history}"
)
@then("the audit log should contain 1 auth_success entry from middleware")
def step_audit_has_auth_success(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="auth_success")
assert len(entries) == 1, f"Expected 1 auth_success entry, got {len(entries)}"
@then("the audit log should contain 1 auth_failure entry from middleware")
def step_audit_has_auth_failure(context: Context) -> None:
entries = context.audit_service.list_entries(event_type="auth_failure")
assert len(entries) == 1, f"Expected 1 auth_failure entry, got {len(entries)}"
@then(
'the latest {audit_event_type} audit entry should contain detail "{key}" with value "{expected_value}"'
)
def step_latest_audit_entry_detail(
context: Context, audit_event_type: str, key: str, expected_value: str
) -> None:
entries = context.audit_service.list_entries(event_type=audit_event_type)
assert entries, f"Expected at least one {audit_event_type} audit entry"
details = entries[0].details
assert details.get(key) == expected_value, (
f"Expected {audit_event_type}.{key}={expected_value!r}, "
f"got {details.get(key)!r}"
)
+98
View File
@@ -135,6 +135,11 @@ def step_plan_invariants(context):
context.plan_invariants = _parse_invariant_table(context, InvariantScope.PLAN)
@given("I have action invariants")
def step_action_invariants(context):
context.action_invariants = _parse_invariant_table(context, InvariantScope.ACTION)
@given("I have project invariants")
def step_project_invariants(context):
context.project_invariants = _parse_invariant_table(context, InvariantScope.PROJECT)
@@ -160,6 +165,7 @@ def step_plan_invariants_inactive(context):
def step_merge(context):
context.merged = merge_invariants(
getattr(context, "plan_invariants", []),
getattr(context, "action_invariants", []),
getattr(context, "project_invariants", []),
getattr(context, "global_invariants", []),
)
@@ -180,6 +186,43 @@ def step_merged_scope(context, idx, scope):
assert context.merged[idx].scope.value == scope
@then("action invariants appear before project invariants in merge")
def step_action_before_project(context):
"""Verify ACTION tier comes before PROJECT tier in the merged result."""
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.merged if i.scope == InvariantScope.PROJECT]
assert len(action_invs) > 0, "No action invariants found in merged result"
assert len(project_invs) > 0, "No project invariants found in merged result"
first_action_idx = context.merged.index(action_invs[0])
first_project_idx = context.merged.index(project_invs[0])
assert first_action_idx < first_project_idx, (
f"Action invariant at index {first_action_idx} "
f"should appear before project invariant at index {first_project_idx}"
)
@then("plan invariants appear before action invariants in merge")
def step_plan_before_action(context):
"""Verify PLAN tier comes before ACTION tier in the merged result."""
plan_invs = [i for i in context.merged if i.scope == InvariantScope.PLAN]
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
assert len(plan_invs) > 0, "No plan invariants found in merged result"
assert len(action_invs) > 0, "No action invariants found in merged result"
first_plan_idx = context.merged.index(plan_invs[0])
first_action_idx = context.merged.index(action_invs[0])
assert first_plan_idx < first_action_idx, (
f"Plan invariant at index {first_plan_idx} "
f"should appear before action invariant at index {first_action_idx}"
)
@then("action invariants are preserved in merge result")
def step_action_invariants_preserved(context):
"""Verify that action-scoped invariants appear in the merged output."""
action_invs = [i for i in context.merged if i.scope == InvariantScope.ACTION]
assert len(action_invs) > 0, "Expected action-scoped invariants in merge result"
# ================================================================
# InvariantSet
# ================================================================
@@ -189,6 +232,7 @@ def step_merged_scope(context, idx, scope):
def step_merge_invariant_set(context):
inv_set = InvariantSet.merge(
getattr(context, "plan_invariants", []),
getattr(context, "action_invariants", []),
getattr(context, "project_invariants", []),
getattr(context, "global_invariants", []),
)
@@ -389,6 +433,15 @@ def step_service_all_scopes(context):
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
@given("an invariant service with invariants at all four scopes")
def step_service_all_four_scopes(context):
context.service = InvariantService()
context.service.add_invariant("Global rule", InvariantScope.GLOBAL, "system")
context.service.add_invariant("Project rule", InvariantScope.PROJECT, "proj1")
context.service.add_invariant("Action rule", InvariantScope.ACTION, "action1")
context.service.add_invariant("Plan rule", InvariantScope.PLAN, "plan1")
@when('I get effective invariants for plan "{plan_id}" and project "{project}"')
def step_effective(context, plan_id, project):
context.effective = context.service.get_effective_invariants(
@@ -396,6 +449,15 @@ def step_effective(context, plan_id, project):
)
@when(
'I get effective invariants for plan "{plan_id}", action "{action}", and project "{project}"'
)
def step_effective_with_action(context, plan_id, action, project):
context.effective = context.service.get_effective_invariants(
plan_id=plan_id, action_name=action, project_name=project
)
@then("the effective set should contain plan invariants first")
def step_effective_plan_first(context):
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
@@ -404,6 +466,35 @@ def step_effective_plan_first(context):
assert first_plan_idx == 0
@then("the effective set should contain action invariants second")
def step_effective_action_second(context):
"""Verify ACTION tier appears after PLAN and before PROJECT."""
plan_invs = [i for i in context.effective if i.scope == InvariantScope.PLAN]
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
assert len(plan_invs) > 0, "No plan invariants found"
assert len(action_invs) > 0, "No action invariants found — bug #9126 not fixed!"
assert len(project_invs) > 0, "No project invariants found"
first_action_idx = context.effective.index(action_invs[0])
if plan_invs:
last_plan_idx = context.effective.index(plan_invs[-1])
assert first_action_idx > last_plan_idx
first_project_idx = context.effective.index(project_invs[0])
assert first_project_idx > first_action_idx
@then("the effective set should contain project invariants after action")
def step_effective_project_after_action(context):
"""Verify PROJECT tier appears after ACTION tier."""
action_invs = [i for i in context.effective if i.scope == InvariantScope.ACTION]
project_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
assert len(action_invs) > 0, "No action invariants found"
assert len(project_invs) > 0, "No project invariants found"
first_proj_idx = context.effective.index(project_invs[0])
last_action_idx = context.effective.index(action_invs[-1])
assert first_proj_idx > last_action_idx
@then("the effective set should contain project invariants second")
def step_effective_project_second(context):
proj_invs = [i for i in context.effective if i.scope == InvariantScope.PROJECT]
@@ -435,6 +526,13 @@ def step_no_duplicates(context):
assert len(texts) == len(set(texts)), f"Duplicates found: {texts}"
@then("the effective set should have {count:d} invariants")
def step_effective_count(context, count):
assert len(context.effective) == count, (
f"Expected {count}, got {len(context.effective)}"
)
# ================================================================
# Enforcement
# ================================================================
+10 -1
View File
@@ -57,7 +57,6 @@ E2E Plan Lifecycle Through Audit Pipeline
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} E2E_OK
User Identity Field Propagates Through Audit Pipeline
[Documentation] DomainEvent.user_identity propagates via AuditEventSubscriber to AuditService DB entry
[Tags] observability audit identity
@@ -87,3 +86,13 @@ User Identity Field Precedence Over Details
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} IDENTITY_PRECEDENCE_OK
Auth Middleware Emits Audit Events
[Documentation] End-to-end: TokenAuthMiddleware emits AUTH_SUCCESS/AUTH_FAILURE through audit pipeline
[Tags] observability audit auth
${result}= Run Process ${PYTHON} ${HELPER} auth_middleware_pipeline
... cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} AUTH_PIPELINE_OK
+100
View File
@@ -23,6 +23,9 @@ from cleveragents.application.services.audit_event_subscriber import ( # noqa:
from cleveragents.application.services.audit_service import ( # noqa: E402
AuditService,
)
from cleveragents.application.services.auth_middleware import ( # noqa: E402
TokenAuthMiddleware,
)
from cleveragents.application.services.plan_lifecycle_service import ( # noqa: E402
PlanLifecycleService,
)
@@ -323,6 +326,102 @@ def user_identity_field_precedence() -> None:
print("IDENTITY_PRECEDENCE_OK")
def auth_middleware_pipeline() -> None:
"""E2E: TokenAuthMiddleware -> EventBus -> AuditEventSubscriber -> DB."""
svc = _make_audit_service()
bus = ReactiveEventBus()
AuditEventSubscriber(audit_service=svc, event_bus=bus)
middleware = TokenAuthMiddleware(expected_token="tok_secret_123", event_bus=bus)
success = middleware.authenticate(
"tok_secret_123",
identity="carol@example.com",
ip_address="10.1.2.3",
)
failure = middleware.authenticate(
"tok_bad_123",
identity="dave@example.com",
ip_address="10.1.2.4",
)
if not success or failure:
print(
"FAIL: unexpected auth boolean results "
f"success={success} failure={failure}",
file=sys.stderr,
)
sys.exit(1)
success_entries = svc.list_entries(event_type="auth_success")
failure_entries = svc.list_entries(event_type="auth_failure")
if len(success_entries) != 1:
print(
f"FAIL: expected 1 auth_success entry, got {len(success_entries)}",
file=sys.stderr,
)
sys.exit(1)
if len(failure_entries) != 1:
print(
f"FAIL: expected 1 auth_failure entry, got {len(failure_entries)}",
file=sys.stderr,
)
sys.exit(1)
s_entry = success_entries[0]
f_entry = failure_entries[0]
s_details = s_entry.details
f_details = f_entry.details
if s_entry.user_identity != "carol@example.com":
print(
f"FAIL: unexpected auth_success identity {s_entry.user_identity!r}",
file=sys.stderr,
)
sys.exit(1)
if f_entry.user_identity != "unauthenticated":
print(
f"FAIL: unexpected auth_failure identity {f_entry.user_identity!r}",
file=sys.stderr,
)
sys.exit(1)
if s_details.get("ip_address") != "10.1.2.3":
print(
f"FAIL: unexpected auth_success ip {s_details.get('ip_address')!r}",
file=sys.stderr,
)
sys.exit(1)
if s_details.get("token_prefix") != "tok_se...":
print(
"FAIL: auth_success token_prefix mismatch",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("ip_address") != "10.1.2.4":
print(
f"FAIL: unexpected auth_failure ip {f_details.get('ip_address')!r}",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("attempted_identity") != "dave@example.com":
print(
"FAIL: auth_failure attempted_identity mismatch",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("failure_reason") != "invalid_token":
print(
f"FAIL: unexpected auth_failure reason {f_details.get('failure_reason')!r}",
file=sys.stderr,
)
sys.exit(1)
if f_details.get("token_prefix") != "tok_ba...":
print(
"FAIL: auth_failure token_prefix mismatch",
file=sys.stderr,
)
sys.exit(1)
print("AUTH_PIPELINE_OK")
# ---------------------------------------------------------------------------
# Dispatch
# ---------------------------------------------------------------------------
@@ -336,6 +435,7 @@ _COMMANDS = {
"user_identity_field": user_identity_field,
"user_identity_details_fallback": user_identity_details_fallback,
"user_identity_field_precedence": user_identity_field_precedence,
"auth_middleware_pipeline": auth_middleware_pipeline,
}
if __name__ == "__main__":
+2
View File
@@ -867,6 +867,7 @@ def invariants_enforced_during_strategize() -> None:
merged = merge_invariants(
plan_invariants=[plan_inv],
action_invariants=[],
project_invariants=[project_inv],
global_invariants=[global_inv],
)
@@ -890,6 +891,7 @@ def invariants_enforced_during_strategize() -> None:
invariant_set = InvariantSet.merge(
plan_invariants=[plan_inv],
action_invariants=[],
project_invariants=[project_inv],
global_invariants=[global_inv],
)
+27
View File
@@ -23,6 +23,7 @@ from cleveragents.application.services.audit_event_subscriber import (
AuditEventSubscriber,
)
from cleveragents.application.services.audit_service import AuditService
from cleveragents.application.services.auth_middleware import TokenAuthMiddleware
from cleveragents.application.services.automation_profile_service import (
AutomationProfileService,
)
@@ -410,6 +411,25 @@ def _resolve_auto_reindex() -> bool:
return True
def _resolve_server_token() -> str | None:
"""Resolve ``server.token`` from config, returning ``None`` when unset."""
try:
svc = ConfigService()
resolved = svc.resolve("server.token")
raw = resolved.value
if raw is None:
return None
token = str(raw).strip()
return token if token else None
except Exception as exc:
_logger.warning(
"server_token_resolution_failed",
error_type=type(exc).__name__,
error_message=redact_value(str(exc)),
)
return None
def _build_analyzer_registry() -> AnalyzerRegistry:
"""Build an AnalyzerRegistry with built-in analyzers registered."""
from cleveragents.domain.models.acms.python_analyzer import PythonAnalyzer
@@ -667,6 +687,13 @@ class Container(containers.DeclarativeContainer):
# Event Bus - Singleton (one shared instance for the process)
event_bus = providers.Singleton(ReactiveEventBus)
# Per-request auth middleware with fresh token resolution from config.
auth_middleware = providers.Factory(
TokenAuthMiddleware,
expected_token=providers.Callable(_resolve_server_token),
event_bus=event_bus,
)
# Services - Factory (new instance per request with injected dependencies)
project_service = providers.Factory(
ProjectService,
@@ -37,19 +37,13 @@ _logger = structlog.get_logger(__name__)
SECURITY_EVENT_MAP: dict[EventType, str] = {
EventType.PLAN_APPLIED: "plan_applied",
EventType.PLAN_CANCELLED: "plan_cancelled",
# NOTE: RESOURCE_MODIFIED has no producing service yet — the tool
# execution framework does not emit this event. The subscriber handler
# is intentionally registered so that audit entries are automatically
# created once tool-write event emission is implemented.
# Emitted by ResourceFileWatcher when indexed resources change.
EventType.RESOURCE_MODIFIED: "resource_modified",
EventType.CORRECTION_APPLIED: "correction_applied",
EventType.CONFIG_CHANGED: "config_changed",
EventType.ENTITY_DELETED: "entity_deleted",
EventType.SESSION_CREATED: "session_created",
# NOTE: AUTH_SUCCESS and AUTH_FAILURE have no producing service yet —
# server-mode authentication is not implemented. The subscriber handlers
# are registered so that audit entries are automatically created once
# server auth emits these events.
# Emitted by TokenAuthMiddleware during bearer-token checks.
EventType.AUTH_SUCCESS: "auth_success",
EventType.AUTH_FAILURE: "auth_failure",
}
@@ -0,0 +1,189 @@
"""Authentication middleware that emits audit-friendly auth events.
This service encapsulates bearer-token authentication checks and emits
``AUTH_SUCCESS`` / ``AUTH_FAILURE`` domain events through an injected
``EventBus`` so the audit pipeline can persist login outcomes.
Spec alignment:
- docs/specification.md § Event System
- docs/specification.md § Audit Logging (auth_success / auth_failure)
"""
from __future__ import annotations
import hmac
from typing import TYPE_CHECKING
import structlog
from cleveragents.infrastructure.events.models import DomainEvent
from cleveragents.infrastructure.events.types import EventType
from cleveragents.shared.redaction import redact_value
if TYPE_CHECKING:
from cleveragents.infrastructure.events.protocol import EventBus
_logger = structlog.get_logger(__name__)
_TOKEN_PREFIX_LEN = 6
_SHORT_TOKEN_MASK = "***..."
def _token_prefix(token: str) -> str:
"""Return a safe token prefix for audit details.
Never returns the full token value. Tokens shorter than or equal to the
prefix length are collapsed to a constant mask to avoid full disclosure.
"""
if len(token) <= _TOKEN_PREFIX_LEN:
return _SHORT_TOKEN_MASK
return f"{token[:_TOKEN_PREFIX_LEN]}..."
def _normalize_optional_text(field_name: str, value: str | None) -> str | None:
"""Normalize optional text fields with guard-first validation."""
if value is None:
return None
if not isinstance(value, str):
raise TypeError(f"{field_name} must be a string or None")
normalized = value.strip()
if not normalized:
raise ValueError(f"{field_name} must not be empty when provided")
return normalized
class TokenAuthMiddleware:
"""Authenticate bearer tokens and emit AUTH_* events.
This class is intentionally transport-agnostic. A future server-mode
middleware can delegate authentication decisions to this component and
inherit consistent event emission semantics.
"""
def __init__(
self,
expected_token: str | None,
event_bus: EventBus | None = None,
) -> None:
"""Initialize middleware with an expected token and optional event bus.
Args:
expected_token: Configured bearer token for successful auth.
event_bus: Event bus used for AUTH_SUCCESS/AUTH_FAILURE emission.
"""
self._expected_token = _normalize_optional_text(
"expected_token", expected_token
)
self._event_bus = event_bus
def authenticate(
self,
token: str,
*,
identity: str | None = None,
ip_address: str | None = None,
) -> bool:
"""Authenticate *token* and emit a corresponding auth event.
Args:
token: Bearer token supplied by the caller.
identity: Optional user identity / principal for audit context.
ip_address: Optional client IP address for audit context.
Returns:
``True`` if authentication succeeds, otherwise ``False``.
"""
if not isinstance(token, str):
raise TypeError("token must be a string")
normalized_token = token.strip()
if not normalized_token:
raise ValueError("token must not be empty")
normalized_identity = _normalize_optional_text("identity", identity)
normalized_ip = _normalize_optional_text("ip_address", ip_address)
if self._expected_token is None:
self._emit_auth_failure(
attempted_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
reason="token_not_configured",
)
return False
if hmac.compare_digest(normalized_token, self._expected_token):
self._emit_auth_success(
user_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
)
return True
self._emit_auth_failure(
attempted_identity=normalized_identity,
ip_address=normalized_ip,
token_prefix=_token_prefix(normalized_token),
reason="invalid_token",
)
return False
def _emit_auth_success(
self,
*,
user_identity: str | None,
ip_address: str | None,
token_prefix: str,
) -> None:
"""Emit AUTH_SUCCESS if an event bus is configured."""
if self._event_bus is None:
return
details: dict[str, str] = {
"user_identity": user_identity or "unknown",
"token_prefix": token_prefix,
}
if ip_address is not None:
details["ip_address"] = ip_address
self._emit_event(EventType.AUTH_SUCCESS, details)
def _emit_auth_failure(
self,
*,
attempted_identity: str | None,
ip_address: str | None,
token_prefix: str,
reason: str,
) -> None:
"""Emit AUTH_FAILURE if an event bus is configured."""
if self._event_bus is None:
return
details: dict[str, str] = {
"attempted_identity": attempted_identity or "unknown",
"user_identity": "unauthenticated",
"failure_reason": reason,
"token_prefix": token_prefix,
}
if ip_address is not None:
details["ip_address"] = ip_address
self._emit_event(EventType.AUTH_FAILURE, details)
def _emit_event(self, event_type: EventType, details: dict[str, str]) -> None:
"""Emit one auth event and swallow publish-side failures."""
bus = self._event_bus
if bus is None:
return
try:
bus.emit(
DomainEvent(
event_type=event_type,
details=details,
)
)
except Exception as exc: # pragma: no cover - defensive logging path
_logger.warning(
"auth_event_emit_failed",
event_type=event_type.value,
error_type=type(exc).__name__,
error_message=redact_value(str(exc)),
)
__all__ = ["TokenAuthMiddleware"]
@@ -3,6 +3,7 @@
from __future__ import annotations
import fnmatch
import json
from pathlib import PurePath
from typing import Any, Protocol
@@ -70,6 +71,72 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
return ProjectContextPolicy().resolve_view("execute")
return policy.resolve_view("execute")
def _resolve_hot_max_tokens(self, project_names: list[str]) -> int:
"""Resolve the effective ``hot_max_tokens`` budget for *project_names*.
Looks up each project's :attr:`ContextConfig.hot_max_tokens` from its
stored context configuration. Projects that do not have an explicit
setting are excluded from the aggregation so they do not affect
the computed value.
Returns the **maximum** of all explicitly-set project-level values,
falling back to :attr:`_hot_max_tokens` (the caller-passed global
default) when no project overrides are present.
"""
from typing import cast
candidates: list[int] = []
for namespaced_name in project_names:
row = None
try:
session = self._project_repository._session() # type: ignore[attr-defined]
from cleveragents.infrastructure.database.models import (
NamespacedProjectModel,
)
row = (
session.query(NamespacedProjectModel)
.filter_by(namespaced_name=namespaced_name)
.first()
)
session.close()
except AttributeError:
self._logger.warning(
"hot_max_tokens_session_factory_missing",
project_name=namespaced_name,
)
continue
except Exception:
self._logger.warning(
"hot_max_tokens_lookup_failed",
project_name=namespaced_name,
exc_info=True,
)
if row is not None and row.context_policy_json is not None:
try:
config_dict = json.loads(cast(str, row.context_policy_json))
tokens = config_dict.get("hot_max_tokens")
if tokens is not None and isinstance(tokens, int) and tokens > 0:
candidates.append(tokens)
except (ValueError, TypeError):
self._logger.warning(
"hot_max_tokens_parse_failed",
project_name=namespaced_name,
)
if candidates:
effective = max(candidates)
self._logger.info(
"hot_max_tokens_resolved_from_projects",
project_names=project_names,
project_values=candidates,
effective=effective,
)
return effective
# No project overrides --- use the caller-passed global default.
return self._hot_max_tokens
@staticmethod
def _path_matches(path: str, include: list[str], exclude: list[str]) -> bool:
"""Return whether *path* passes include/exclude path globs.
@@ -226,14 +293,21 @@ class ACMSExecutePhaseContextAssembler(ExecutePhaseContextAssembler):
)
return None
budget = CoreContextBudget(max_tokens=self._hot_max_tokens, reserved_tokens=0)
# Resolve effective hot_max_tokens: project-level overrides take precedence
# over the global default. When multiple projects have explicit values,
# use the maximum so all projects can contribute within their biggest budget.
effective_hot_max_tokens = self._resolve_hot_max_tokens(project_names)
budget = CoreContextBudget(
max_tokens=effective_hot_max_tokens, reserved_tokens=0
)
request = ContextRequest(
query=(
f"Execute-phase context for plan {plan.identity.plan_id} "
f"({', '.join(project_names)})"
),
purpose="llm_execute_phase_prompt",
max_tokens=self._hot_max_tokens,
max_tokens=effective_hot_max_tokens,
)
payload = self._pipeline.assemble(
plan_id=plan.identity.plan_id,
@@ -11,8 +11,8 @@ a dict keyed by invariant ID.
## Merge Precedence
Effective invariants are computed using plan > project > global order.
See ``merge_invariants`` for de-duplication semantics.
Effective invariants are computed using plan > action > project > global
order. See ``merge_invariants`` for de-duplication semantics.
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
"""
@@ -124,6 +124,7 @@ class InvariantService:
if effective:
return self.get_effective_invariants(
plan_id=source_name if scope == InvariantScope.PLAN else None,
action_name=source_name if scope == InvariantScope.ACTION else None,
project_name=source_name if scope == InvariantScope.PROJECT else None,
)
@@ -167,16 +168,22 @@ class InvariantService:
def get_effective_invariants(
self,
plan_id: str | None = None,
action_name: str | None = None,
project_name: str | None = None,
) -> list[Invariant]:
"""Return the merged precedence chain for a plan/project context.
"""Return the merged precedence chain for a plan/action/project context.
Collects active invariants from each scope tier and merges them
using plan > project > global precedence.
using plan > action > project > global precedence.
Args:
plan_id: Optional plan identifier to collect plan-scoped
invariants.
action_name: Optional action name to collect action-scoped
invariants (promoted to plan-level during reconciliation).
When ``None``, the action tier is omitted for backward
compatibility. Pass ``"*"`` to include all action-scoped
invariants regardless of source name.
project_name: Optional project name to collect project-scoped
invariants.
@@ -191,6 +198,13 @@ class InvariantService:
if inv.scope == InvariantScope.PLAN
and (plan_id is None or inv.source_name == plan_id)
]
action_invs = [
inv
for inv in active
if inv.scope == InvariantScope.ACTION
and action_name is not None # Only include when explicitly requested
and (inv.source_name == action_name or action_name == "*")
]
project_invs = [
inv
for inv in active
@@ -199,7 +213,7 @@ class InvariantService:
]
global_invs = [inv for inv in active if inv.scope == InvariantScope.GLOBAL]
return merge_invariants(plan_invs, project_invs, global_invs)
return merge_invariants(plan_invs, action_invs, project_invs, global_invs)
def enforce_invariants(
self,
+227 -5
View File
@@ -307,6 +307,223 @@ def _plan_spec_dict(plan: Any) -> dict[str, object]:
return {"plan": str(plan)}
def _status_output_dict(
plan: Any,
duration_ms: int | None = None,
) -> dict[str, object]:
"""Build the spec-required status output envelope.
Returns the structured JSON envelope for ``agents plan status --format json``
as defined in the specification §agents plan status.
The envelope structure is::
{
"command": "plan status",
"status": "ok",
"exit_code": 0,
"data": {
"plan_id": "...",
"phase": "...",
"state": "...",
"action": "...",
"project": "...",
"automation": "...",
"attempt": 1,
"progress": [...],
"timing": {"started": "HH:MM:SS", ...},
"execution": {...},
"cost": {...}
},
"timing": {"started": "...", "duration_ms": ...},
"messages": ["Status refreshed"]
}
Args:
plan: The ``Plan`` domain model.
duration_ms: Elapsed milliseconds for command execution. ``None`` when
not available.
Returns:
A JSON-serialisable dict matching the spec-required envelope.
"""
from cleveragents.domain.models.core.plan import ExecutionEnvPriority
from cleveragents.domain.models.core.plan import Plan as LifecyclePlan
if not isinstance(plan, LifecyclePlan):
# Legacy plan fallback — return minimal spec-aligned envelope
return {
"command": "plan status",
"status": "ok",
"exit_code": 0,
"data": {"plan": str(plan)},
"timing": {},
"messages": ["Status refreshed"],
}
plan_id = plan.identity.plan_id
# ── Phase → progress steps mapping (spec §Example 17) ────────────────────
phase_map: dict[str, list[dict[str, str]]] = {
"strategize": [
{"step": "Strategize", "status": "running"},
{"step": "Execute", "status": "queued"},
{"step": "Apply", "status": "queued"},
{"step": "Applied", "status": "queued"},
],
"execute": [
{"step": "Strategize", "status": "done"},
{"step": "Execute", "status": "running"},
{"step": "Apply", "status": "queued"},
{"step": "Applied", "status": "queued"},
],
"apply": [
{"step": "Strategize", "status": "done"},
{"step": "Execute", "status": "done"},
{"step": "Apply", "status": "running"},
{"step": "Applied", "status": "queued"},
],
"applied": [
{"step": "Strategize", "status": "done"},
{"step": "Execute", "status": "done"},
{"step": "Apply", "status": "done"},
{"step": "Applied", "status": "done"},
],
}
# ── Progress steps (spec §Example 17) ────────────────────────────────────
progress: list[dict[str, str]] = phase_map.get(
plan.phase.value,
[{"step": "Unknown", "status": "queued"}],
)
# Deep-copy to avoid mutating the template dict.
progress = [{**step} for step in progress]
# ── Apply processing-state adjustments to progress ───────────────────────
if plan.processing_state == ProcessingState.COMPLETE:
for entry in progress:
if entry["status"] == "running":
entry["status"] = "done"
elif plan.processing_state == ProcessingState.ERRORED:
for entry in progress:
if entry["status"] == "running":
entry["status"] = "error"
# ── Timing (internal to data) ────────────────────────────────────────────
internal_timing: dict[str, object] = {}
strategy_start = plan.timestamps.strategize_started_at
if strategy_start is not None:
internal_timing["started"] = strategy_start.strftime("%H:%M:%S")
# Elapsed: time since strategize started (or creation) until now or
# the last phase transition.
ref_time = strategy_start or plan.timestamps.created_at
elapsed_seconds: float = 0
end_ref = (
plan.timestamps.updated_at
if plan.processing_state in (
ProcessingState.COMPLETE,
ProcessingState.CANCELLED,
)
else datetime.now(UTC)
)
if end_ref.tzinfo is None:
end_ref = end_ref.replace(tzinfo=UTC)
if ref_time.tzinfo is None:
ref_time = ref_time.replace(tzinfo=UTC)
delta = (end_ref - ref_time).total_seconds()
elapsed_seconds = max(0, delta)
hours, rem = divmod(int(elapsed_seconds), 3600)
minutes, seconds = divmod(rem, 60)
internal_timing["elapsed"] = f"{hours:02d}:{minutes:02d}:{seconds:02d}"
# ETA: rough estimate based on total time divided by phase count.
if elapsed_seconds > 0 and plan.processing_state not in (
ProcessingState.COMPLETE,
ProcessingState.CANCELLED,
):
phases_elapsed = {"strategize": 1, "execute": 2, "apply": 3}.get(
plan.phase.value, 1
)
total_phases = 4
eta_seconds = (elapsed_seconds / max(phases_elapsed, 1)) * (
total_phases - phases_elapsed
)
eh, erem = divmod(int(eta_seconds), 3600)
em, es = divmod(erem, 60)
internal_timing["eta"] = f"{eh:02d}:{em:02d}:{es:02d}"
# ── Execution detail (internal to data) ──────────────────────────────────
exec_info: dict[str, object] = {}
if plan.execution_environment:
exec_info["sandbox"] = plan.execution_environment
else:
exec_info["sandbox"] = "git_worktree"
if plan.last_completed_step >= 0:
exec_info["tool_calls"] = plan.last_completed_step + 1
if plan.validation_summary and plan.validation_summary.get("total_files_changed"):
exec_info["files_modified"] = plan.validation_summary["total_files_changed"]
if hasattr(plan, "child_plans") and plan.child_plans is not None:
completed = sum(1 for cp in plan.child_plans if cp.is_terminal)
total = len(plan.child_plans)
exec_info["child_plans"] = f"{completed}/{total} complete"
if hasattr(plan, "checkpoint_count"):
exec_info["checkpoints"] = plan.checkpoint_count
# ── Cost (internal to data) ──────────────────────────────────────────────
cost_info: dict[str, object] = {}
if plan.estimation_result is not None:
est = plan.estimation_result.as_display_dict()
if est.get("estimated_cost_usd") is not None:
cost_info["tokens_used"] = est.get("estimated_tokens", 0)
cost_info["cost_so_far"] = est.get("estimated_cost_usd", 0)
cost_info["estimated"] = est.get("total_estimated_cost_usd", 0)
# ── Top-level timing envelope ────────────────────────────────────────────
top_timing: dict[str, object] = {}
strategy_ts = plan.timestamps.strategize_started_at or plan.timestamps.created_at
if strategy_ts is not None:
top_timing["started"] = strategy_ts.isoformat()
if duration_ms is not None:
top_timing["duration_ms"] = duration_ms
# ── Build data payload (spec §13674) ─────────────────────────────────────
data: dict[str, object] = {
"plan_id": plan_id,
"phase": plan.phase.value,
"state": plan.processing_state.value,
"action": plan.action_name,
# Project: use the first project link; fall back to "(none)"
"project": (
plan.project_links[0].project_name if plan.project_links else "(none)"
),
}
auto_profile = plan.automation_profile
data["automation"] = (
auto_profile.profile_name if auto_profile else "(unset)"
)
data["attempt"] = plan.identity.attempt
data["progress"] = progress
if internal_timing:
data["timing"] = internal_timing
if exec_info:
data["execution"] = exec_info
if cost_info:
data["cost"] = cost_info
return {
"command": "plan status",
"status": "ok",
"exit_code": 0,
"data": data,
"timing": top_timing,
"messages": ["Status refreshed"],
}
def _execute_output_dict(
plan: Any,
started_at: datetime | None = None,
@@ -2354,10 +2571,15 @@ def plan_status(
)
return
# Non-rich formats
# Non-rich formats (spec-required JSON envelope)
if fmt != OutputFormat.RICH.value:
data = [_plan_spec_dict(p) for p in plans]
console.print(format_output(data, fmt))
envelope = {
"command": "plan status",
"status": "ok",
"exit_code": 0,
"data": [_plan_spec_dict(p) for p in plans],
}
console.print(format_output(envelope, fmt))
return
# Show summary table
@@ -2400,8 +2622,8 @@ def plan_status(
plan = service.get_plan(plan_id)
if fmt != OutputFormat.RICH.value:
data = _plan_spec_dict(plan)
console.print(format_output(data, fmt))
envelope = _status_output_dict(plan)
console.print(format_output(envelope, fmt))
return
_print_lifecycle_plan(plan, title="Plan Status")
@@ -2,9 +2,10 @@
Invariants are natural-language constraints on plan execution, scoped at
global, project, action, or plan level. When an action is used, its
invariants are promoted to plan-level. The runtime precedence chain is:
invariants are promoted to plan-level and participate in the merge.
The runtime precedence chain is:
plan > project > global
plan > action > project > global
They are reconciled by the Invariant Reconciliation Actor at the start
of Strategize and recorded as ``invariant_enforced`` decisions.
@@ -21,8 +22,8 @@ of Strategize and recorded as ``invariant_enforced`` decisions.
## Merge Precedence
When computing the effective set of invariants for a plan, the merge
order is **plan > project > global**. Duplicate texts (case-insensitive)
are de-duplicated, keeping the highest-precedence copy.
order is **plan > action > project > global**. Duplicate texts
(case-insensitive) are de-duplicated, keeping the highest-precedence copy.
Based on ``docs/specification.md`` and implementation plan Stage M3.5.
"""
@@ -39,8 +40,10 @@ from ulid import ULID
class InvariantScope(StrEnum):
"""Scope at which an invariant applies.
Precedence (highest to lowest): PLAN > PROJECT > GLOBAL.
ACTION invariants are promoted to PLAN scope at ``plan use`` time.
Precedence (highest to lowest): PLAN > ACTION > PROJECT > GLOBAL.
ACTION invariants are promoted to plan-level and participate in the
merge during reconciliation, sitting between PLAN and PROJECT in the
precedence chain.
"""
GLOBAL = "global"
@@ -137,10 +140,11 @@ class InvariantSet(BaseModel):
def merge(
cls,
plan_invariants: list[Invariant],
project_invariants: list[Invariant],
global_invariants: list[Invariant],
action_invariants: list[Invariant] | None = None,
project_invariants: list[Invariant] | None = None,
global_invariants: list[Invariant] | None = None,
) -> InvariantSet:
"""Merge invariants respecting plan > project > global precedence.
"""Merge invariants respecting plan > action > project > global precedence.
De-duplicates by text (case-insensitive), keeping the copy from
the highest-precedence tier. Within each tier, source ordering
@@ -148,6 +152,9 @@ class InvariantSet(BaseModel):
Args:
plan_invariants: Plan-level invariants (highest precedence).
action_invariants: Action-scoped invariants (second-highest
precedence; promoted to plan-level when an
action is used).
project_invariants: Project-level invariants.
global_invariants: Global-level invariants (lowest precedence).
@@ -156,7 +163,12 @@ class InvariantSet(BaseModel):
"""
return cls(
invariants=tuple(
merge_invariants(plan_invariants, project_invariants, global_invariants)
merge_invariants(
plan_invariants,
action_invariants or [],
project_invariants or [],
global_invariants or [],
)
)
)
@@ -165,10 +177,11 @@ class InvariantSet(BaseModel):
def merge_invariants(
plan_invariants: list[Invariant],
project_invariants: list[Invariant],
global_invariants: list[Invariant],
action_invariants: list[Invariant] | None = None,
project_invariants: list[Invariant] | None = None,
global_invariants: list[Invariant] | None = None,
) -> list[Invariant]:
"""Merge invariants implementing plan > project > global precedence.
"""Merge invariants implementing plan > action > project > global precedence.
De-duplicates by text (case-insensitive). The first occurrence
(from the highest-precedence tier) wins. Within each tier, the
@@ -176,8 +189,13 @@ def merge_invariants(
Args:
plan_invariants: Plan-level invariants (highest precedence).
project_invariants: Project-level invariants.
action_invariants: Action-scoped invariants (second-highest
precedence; promoted to plan level when an action is used).
Defaults to empty list when ``None``.
project_invariants: Project-level invariants. Defaults to empty
list when ``None``.
global_invariants: Global-level invariants (lowest precedence).
Defaults to empty list when ``None``.
Returns:
A de-duplicated list of invariants in precedence order.
@@ -185,7 +203,12 @@ def merge_invariants(
seen: set[str] = set()
result: list[Invariant] = []
for inv_list in (plan_invariants, project_invariants, global_invariants):
for inv_list in (
plan_invariants,
action_invariants or [],
project_invariants or [],
global_invariants or [],
):
for inv in inv_list:
if not inv.active:
continue
+1
View File
@@ -42,6 +42,7 @@ _SENSITIVE_SUBSTRINGS: set[str] = {
# though they contain a sensitive substring (e.g. "token_count").
_FALSE_POSITIVE_KEYS: set[str] = {
"token_count",
"token_prefix",
"token_limit",
"token_usage",
"input_tokens",