feat(acms): implement context analysis engine with tier distribution and budget utilization metrics
CI / push-validation (pull_request) Successful in 23s
CI / helm (pull_request) Successful in 31s
CI / build (pull_request) Successful in 57s
CI / lint (pull_request) Successful in 1m17s
CI / quality (pull_request) Successful in 1m29s
CI / security (pull_request) Successful in 1m33s
CI / typecheck (pull_request) Successful in 1m45s
CI / integration_tests (pull_request) Successful in 3m38s
CI / e2e_tests (pull_request) Successful in 4m0s
CI / unit_tests (pull_request) Successful in 5m5s
CI / docker (pull_request) Successful in 1m49s
CI / benchmark-publish (pull_request) Has been skipped
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 3s
CI / push-validation (push) Successful in 25s
CI / helm (push) Successful in 33s
CI / quality (push) Successful in 1m15s
CI / build (push) Successful in 58s
CI / lint (push) Successful in 1m39s
CI / typecheck (push) Successful in 1m45s
CI / security (push) Successful in 1m45s
CI / integration_tests (push) Successful in 3m56s
CI / e2e_tests (push) Successful in 4m34s
CI / unit_tests (push) Successful in 5m21s
CI / docker (push) Successful in 1m49s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Has started running
CI / coverage (push) Successful in 11m24s
CI / status-check (push) Successful in 4s
CI / benchmark-regression (pull_request) Successful in 1h4m44s

Introduces a new ContextAnalysisEngine service to unify ACMS context analysis across tiered storage.
It computes and exposes key observability metrics and provides flexible formatting for tooling and dashboards.
Metrics include:
- entry_count(): total entries across all tiers
- tier_distribution(): per-tier counts and sizes (hot, warm, cold)
- budget_utilization(): current total size vs. configured max, as a percentage
- top_files(n): top-N entries by access_count in descending order
- analyze(top_n): combined analysis result aggregating the above metrics
- format_json() and format_text() static formatters for machine- and human-friendly output

A new CLI command context analyze has been added at src/cleveragents/cli/commands/context.py to surface the feature from the command line.
Unit tests live under features/acms_context_analysis_engine.feature (29 scenarios) with step definitions in features/steps/acms_context_analysis_engine_steps.py.
The commit wires the new command into the CLI, enables test-driven validation of the metrics, and lays the groundwork for integration with dashboards.

ISSUES CLOSED: #9984
This commit was merged in pull request #10775.
This commit is contained in:
2026-04-19 13:56:43 +00:00
committed by Forgejo
parent 4f23ece138
commit 3b66bf08f5
4 changed files with 1166 additions and 0 deletions
@@ -0,0 +1,195 @@
Feature: ACMS Context Analysis Engine
As a CleverAgents user
I want to analyze the ACMS context index
So that I can understand how my context budget is being used
# entry_count
Scenario: entry_count returns zero for empty index
Given an empty ContextAnalysisEngine
When I call entry_count
Then the entry count should be 0
Scenario: entry_count returns total across all tiers
Given a ContextAnalysisEngine with fragments in all tiers
When I call entry_count
Then the entry count should be 3
# ── tier_distribution ──────────────────────────────────────
Scenario: tier_distribution returns zero counts for empty index
Given an empty ContextAnalysisEngine
When I call tier_distribution
Then the hot tier count should be 0
And the warm tier count should be 0
And the cold tier count should be 0
Scenario: tier_distribution counts fragments per tier
Given a ContextAnalysisEngine with one hot fragment of content "hello"
When I call tier_distribution
Then the hot tier count should be 1
And the hot tier size_bytes should be 5
Scenario: tier_distribution counts warm fragments
Given a ContextAnalysisEngine with one warm fragment of content "world"
When I call tier_distribution
Then the warm tier count should be 1
And the warm tier size_bytes should be 5
Scenario: tier_distribution counts cold fragments
Given a ContextAnalysisEngine with one cold fragment of content "cold"
When I call tier_distribution
Then the cold tier count should be 1
And the cold tier size_bytes should be 4
Scenario: tier_distribution aggregates sizes across multiple fragments
Given a ContextAnalysisEngine with two hot fragments of content "ab" and "cde"
When I call tier_distribution
Then the hot tier count should be 2
And the hot tier size_bytes should be 5
# ── budget_utilization ─────────────────────────────────────
Scenario: budget_utilization returns zero for empty index
Given an empty ContextAnalysisEngine with max_total_size 1000
When I call budget_utilization
Then the current_bytes should be 0
And the max_bytes should be 1000
And the utilization_pct should be 0.0
Scenario: budget_utilization computes percentage correctly
Given a ContextAnalysisEngine with a hot fragment of content "hello" and max_total_size 10
When I call budget_utilization
Then the current_bytes should be 5
And the max_bytes should be 10
And the utilization_pct should be 50.0
Scenario: budget_utilization caps at 100 percent when over budget
Given a ContextAnalysisEngine with a hot fragment of content "hello world" and max_total_size 5
When I call budget_utilization
Then the utilization_pct should be 100.0
Scenario: budget_utilization returns zero when max_bytes is zero
Given an empty ContextAnalysisEngine with max_total_size 0
When I call budget_utilization
Then the utilization_pct should be 0.0
# ── top_files ──────────────────────────────────────────────
Scenario: top_files returns empty list for empty index
Given an empty ContextAnalysisEngine
When I call top_files with n 10
Then the top files list should be empty
Scenario: top_files returns entries sorted by access_count descending
Given a ContextAnalysisEngine with fragments having access counts 5 and 2 and 8
When I call top_files with n 10
Then the top files should be ordered by access_count descending
Scenario: top_files respects the n limit
Given a ContextAnalysisEngine with fragments having access counts 5 and 2 and 8
When I call top_files with n 2
Then the top files list should have 2 entries
Scenario: top_files raises ValueError for non-positive n
Given an empty ContextAnalysisEngine
When I call top_files with n 0
Then a ValueError should be raised for top_files n
Scenario: top_files includes fragment_id resource_id access_count and tier
Given a ContextAnalysisEngine with one hot fragment with resource_id "uko:file/main.py" and access_count 3
When I call top_files with n 10
Then the first top file should have resource_id "uko:file/main.py"
And the first top file should have access_count 3
And the first top file should have tier "hot"
# ── analyze ────────────────────────────────────────────────
Scenario: analyze returns combined AnalysisResult
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
Then the analysis entry_count should be 3
And the analysis tier_distribution should have hot count 1
And the analysis top_files should not be empty
# ── format_json ────────────────────────────────────────────
Scenario: format_json returns valid JSON with all keys
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
And I format the result as JSON
Then the acms JSON output should contain key "entry_count"
And the acms JSON output should contain key "tier_distribution"
And the acms JSON output should contain key "budget_utilization"
And the acms JSON output should contain key "top_files"
Scenario: format_json tier_distribution has hot warm cold keys
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
And I format the result as JSON
Then the acms JSON tier_distribution should have keys "hot" "warm" "cold"
Scenario: format_json budget_utilization has required keys
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
And I format the result as JSON
Then the acms JSON budget_utilization should have keys "current_bytes" "max_bytes" "utilization_pct"
# ── format_text ────────────────────────────────────────────
Scenario: format_text returns human-readable output
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
And I format the result as text
Then the text output should contain "ACMS Context Analysis"
And the text output should contain "Tier Distribution"
And the text output should contain "Budget Utilization"
And the text output should contain "Top"
Scenario: format_text shows no entries when index is empty
Given an empty ContextAnalysisEngine
When I call analyze with top_n 10
And I format the result as text
Then the text output should contain "(no entries)"
# ── to_dict ────────────────────────────────────────────────
Scenario: AnalysisResult to_dict contains all keys
Given a ContextAnalysisEngine with fragments in all tiers
When I call analyze with top_n 10
Then the result to_dict should contain key "entry_count"
And the result to_dict should contain key "tier_distribution"
And the result to_dict should contain key "budget_utilization"
And the result to_dict should contain key "top_files"
Scenario: TierStats to_dict returns count and size_bytes
Given a TierStats with count 3 and size_bytes 100
When I call to_dict on TierStats
Then the TierStats dict should have count 3 and size_bytes 100
Scenario: BudgetUtilization to_dict rounds utilization_pct
Given a BudgetUtilization with current 50 max 100 pct 50.123456
When I call to_dict on BudgetUtilization
Then the BudgetUtilization dict utilization_pct should be 50.12
Scenario: TopFileEntry to_dict returns all fields
Given a TopFileEntry with fragment_id "f1" resource_id "r1" access_count 5 tier "hot"
When I call to_dict on TopFileEntry
Then the TopFileEntry dict should have all fields
# ── default max_total_size ─────────────────────────────────
Scenario: engine uses hot-tier budget as default max_total_size
Given an empty ContextAnalysisEngine without explicit max_total_size
When I call budget_utilization
Then the max_bytes should be the hot-tier budget
# ── CLI analyze command ────────────────────────────────────
Scenario: context analyze CLI command produces text output
When I invoke the context analyze CLI command with empty tier service
Then the CLI output should contain "ACMS Context Analysis"
Scenario: context analyze CLI command produces JSON output with --format json
When I invoke the context analyze CLI command with empty tier service and format json
Then the acms CLI JSON output should contain key "entry_count"