HAL9000
3b66bf08f5
CI / push-validation (pull_request) Successful in 23s
CI / helm (pull_request) Successful in 31s
CI / build (pull_request) Successful in 57s
CI / lint (pull_request) Successful in 1m17s
CI / quality (pull_request) Successful in 1m29s
CI / security (pull_request) Successful in 1m33s
CI / typecheck (pull_request) Successful in 1m45s
CI / integration_tests (pull_request) Successful in 3m38s
CI / e2e_tests (pull_request) Successful in 4m0s
CI / unit_tests (pull_request) Successful in 5m5s
CI / docker (pull_request) Successful in 1m49s
CI / benchmark-publish (pull_request) Has been skipped
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 3s
CI / push-validation (push) Successful in 25s
CI / helm (push) Successful in 33s
CI / quality (push) Successful in 1m15s
CI / build (push) Successful in 58s
CI / lint (push) Successful in 1m39s
CI / typecheck (push) Successful in 1m45s
CI / security (push) Successful in 1m45s
CI / integration_tests (push) Successful in 3m56s
CI / e2e_tests (push) Successful in 4m34s
CI / unit_tests (push) Successful in 5m21s
CI / docker (push) Successful in 1m49s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Has started running
CI / coverage (push) Successful in 11m24s
CI / status-check (push) Successful in 4s
CI / benchmark-regression (pull_request) Successful in 1h4m44s
Introduces a new ContextAnalysisEngine service to unify ACMS context analysis across tiered storage. It computes and exposes key observability metrics and provides flexible formatting for tooling and dashboards. Metrics include: - entry_count(): total entries across all tiers - tier_distribution(): per-tier counts and sizes (hot, warm, cold) - budget_utilization(): current total size vs. configured max, as a percentage - top_files(n): top-N entries by access_count in descending order - analyze(top_n): combined analysis result aggregating the above metrics - format_json() and format_text() static formatters for machine- and human-friendly output A new CLI command context analyze has been added at src/cleveragents/cli/commands/context.py to surface the feature from the command line. Unit tests live under features/acms_context_analysis_engine.feature (29 scenarios) with step definitions in features/steps/acms_context_analysis_engine_steps.py. The commit wires the new command into the CLI, enables test-driven validation of the metrics, and lays the groundwork for integration with dashboards. ISSUES CLOSED: #9984
196 lines
9.2 KiB
Gherkin
196 lines
9.2 KiB
Gherkin
Feature: ACMS Context Analysis Engine
|
|
As a CleverAgents user
|
|
I want to analyze the ACMS context index
|
|
So that I can understand how my context budget is being used
|
|
|
|
# ── entry_count ────────────────────────────────────────────
|
|
|
|
Scenario: entry_count returns zero for empty index
|
|
Given an empty ContextAnalysisEngine
|
|
When I call entry_count
|
|
Then the entry count should be 0
|
|
|
|
Scenario: entry_count returns total across all tiers
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call entry_count
|
|
Then the entry count should be 3
|
|
|
|
# ── tier_distribution ──────────────────────────────────────
|
|
|
|
Scenario: tier_distribution returns zero counts for empty index
|
|
Given an empty ContextAnalysisEngine
|
|
When I call tier_distribution
|
|
Then the hot tier count should be 0
|
|
And the warm tier count should be 0
|
|
And the cold tier count should be 0
|
|
|
|
Scenario: tier_distribution counts fragments per tier
|
|
Given a ContextAnalysisEngine with one hot fragment of content "hello"
|
|
When I call tier_distribution
|
|
Then the hot tier count should be 1
|
|
And the hot tier size_bytes should be 5
|
|
|
|
Scenario: tier_distribution counts warm fragments
|
|
Given a ContextAnalysisEngine with one warm fragment of content "world"
|
|
When I call tier_distribution
|
|
Then the warm tier count should be 1
|
|
And the warm tier size_bytes should be 5
|
|
|
|
Scenario: tier_distribution counts cold fragments
|
|
Given a ContextAnalysisEngine with one cold fragment of content "cold"
|
|
When I call tier_distribution
|
|
Then the cold tier count should be 1
|
|
And the cold tier size_bytes should be 4
|
|
|
|
Scenario: tier_distribution aggregates sizes across multiple fragments
|
|
Given a ContextAnalysisEngine with two hot fragments of content "ab" and "cde"
|
|
When I call tier_distribution
|
|
Then the hot tier count should be 2
|
|
And the hot tier size_bytes should be 5
|
|
|
|
# ── budget_utilization ─────────────────────────────────────
|
|
|
|
Scenario: budget_utilization returns zero for empty index
|
|
Given an empty ContextAnalysisEngine with max_total_size 1000
|
|
When I call budget_utilization
|
|
Then the current_bytes should be 0
|
|
And the max_bytes should be 1000
|
|
And the utilization_pct should be 0.0
|
|
|
|
Scenario: budget_utilization computes percentage correctly
|
|
Given a ContextAnalysisEngine with a hot fragment of content "hello" and max_total_size 10
|
|
When I call budget_utilization
|
|
Then the current_bytes should be 5
|
|
And the max_bytes should be 10
|
|
And the utilization_pct should be 50.0
|
|
|
|
Scenario: budget_utilization caps at 100 percent when over budget
|
|
Given a ContextAnalysisEngine with a hot fragment of content "hello world" and max_total_size 5
|
|
When I call budget_utilization
|
|
Then the utilization_pct should be 100.0
|
|
|
|
Scenario: budget_utilization returns zero when max_bytes is zero
|
|
Given an empty ContextAnalysisEngine with max_total_size 0
|
|
When I call budget_utilization
|
|
Then the utilization_pct should be 0.0
|
|
|
|
# ── top_files ──────────────────────────────────────────────
|
|
|
|
Scenario: top_files returns empty list for empty index
|
|
Given an empty ContextAnalysisEngine
|
|
When I call top_files with n 10
|
|
Then the top files list should be empty
|
|
|
|
Scenario: top_files returns entries sorted by access_count descending
|
|
Given a ContextAnalysisEngine with fragments having access counts 5 and 2 and 8
|
|
When I call top_files with n 10
|
|
Then the top files should be ordered by access_count descending
|
|
|
|
Scenario: top_files respects the n limit
|
|
Given a ContextAnalysisEngine with fragments having access counts 5 and 2 and 8
|
|
When I call top_files with n 2
|
|
Then the top files list should have 2 entries
|
|
|
|
Scenario: top_files raises ValueError for non-positive n
|
|
Given an empty ContextAnalysisEngine
|
|
When I call top_files with n 0
|
|
Then a ValueError should be raised for top_files n
|
|
|
|
Scenario: top_files includes fragment_id resource_id access_count and tier
|
|
Given a ContextAnalysisEngine with one hot fragment with resource_id "uko:file/main.py" and access_count 3
|
|
When I call top_files with n 10
|
|
Then the first top file should have resource_id "uko:file/main.py"
|
|
And the first top file should have access_count 3
|
|
And the first top file should have tier "hot"
|
|
|
|
# ── analyze ────────────────────────────────────────────────
|
|
|
|
Scenario: analyze returns combined AnalysisResult
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
Then the analysis entry_count should be 3
|
|
And the analysis tier_distribution should have hot count 1
|
|
And the analysis top_files should not be empty
|
|
|
|
# ── format_json ────────────────────────────────────────────
|
|
|
|
Scenario: format_json returns valid JSON with all keys
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
And I format the result as JSON
|
|
Then the acms JSON output should contain key "entry_count"
|
|
And the acms JSON output should contain key "tier_distribution"
|
|
And the acms JSON output should contain key "budget_utilization"
|
|
And the acms JSON output should contain key "top_files"
|
|
|
|
Scenario: format_json tier_distribution has hot warm cold keys
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
And I format the result as JSON
|
|
Then the acms JSON tier_distribution should have keys "hot" "warm" "cold"
|
|
|
|
Scenario: format_json budget_utilization has required keys
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
And I format the result as JSON
|
|
Then the acms JSON budget_utilization should have keys "current_bytes" "max_bytes" "utilization_pct"
|
|
|
|
# ── format_text ────────────────────────────────────────────
|
|
|
|
Scenario: format_text returns human-readable output
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
And I format the result as text
|
|
Then the text output should contain "ACMS Context Analysis"
|
|
And the text output should contain "Tier Distribution"
|
|
And the text output should contain "Budget Utilization"
|
|
And the text output should contain "Top"
|
|
|
|
Scenario: format_text shows no entries when index is empty
|
|
Given an empty ContextAnalysisEngine
|
|
When I call analyze with top_n 10
|
|
And I format the result as text
|
|
Then the text output should contain "(no entries)"
|
|
|
|
# ── to_dict ────────────────────────────────────────────────
|
|
|
|
Scenario: AnalysisResult to_dict contains all keys
|
|
Given a ContextAnalysisEngine with fragments in all tiers
|
|
When I call analyze with top_n 10
|
|
Then the result to_dict should contain key "entry_count"
|
|
And the result to_dict should contain key "tier_distribution"
|
|
And the result to_dict should contain key "budget_utilization"
|
|
And the result to_dict should contain key "top_files"
|
|
|
|
Scenario: TierStats to_dict returns count and size_bytes
|
|
Given a TierStats with count 3 and size_bytes 100
|
|
When I call to_dict on TierStats
|
|
Then the TierStats dict should have count 3 and size_bytes 100
|
|
|
|
Scenario: BudgetUtilization to_dict rounds utilization_pct
|
|
Given a BudgetUtilization with current 50 max 100 pct 50.123456
|
|
When I call to_dict on BudgetUtilization
|
|
Then the BudgetUtilization dict utilization_pct should be 50.12
|
|
|
|
Scenario: TopFileEntry to_dict returns all fields
|
|
Given a TopFileEntry with fragment_id "f1" resource_id "r1" access_count 5 tier "hot"
|
|
When I call to_dict on TopFileEntry
|
|
Then the TopFileEntry dict should have all fields
|
|
|
|
# ── default max_total_size ─────────────────────────────────
|
|
|
|
Scenario: engine uses hot-tier budget as default max_total_size
|
|
Given an empty ContextAnalysisEngine without explicit max_total_size
|
|
When I call budget_utilization
|
|
Then the max_bytes should be the hot-tier budget
|
|
|
|
# ── CLI analyze command ────────────────────────────────────
|
|
|
|
Scenario: context analyze CLI command produces text output
|
|
When I invoke the context analyze CLI command with empty tier service
|
|
Then the CLI output should contain "ACMS Context Analysis"
|
|
|
|
Scenario: context analyze CLI command produces JSON output with --format json
|
|
When I invoke the context analyze CLI command with empty tier service and format json
|
|
Then the acms CLI JSON output should contain key "entry_count"
|