Luis Mendes
ff42d59d6d
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 32s
CI / typecheck (pull_request) Successful in 33s
CI / unit_tests (pull_request) Successful in 2m22s
CI / integration_tests (pull_request) Successful in 2m52s
CI / docker (pull_request) Successful in 39s
CI / coverage (pull_request) Successful in 4m13s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 31s
CI / typecheck (push) Successful in 32s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m31s
CI / integration_tests (push) Successful in 3m0s
CI / docker (push) Successful in 39s
CI / coverage (push) Successful in 5m5s
CI / benchmark-publish (push) Successful in 14m43s
CI / benchmark-regression (pull_request) Successful in 25m44s
Wire all four project context CLI commands (inspect, simulate, set, show) to live ACMS pipeline services via ContextTierService and CRP models. - context inspect: queries ContextTierService for tier metrics and per-project fragments with filtering by strategy/focus/breadth/depth - context simulate: dry-run context assembly using CRP models with configurable token budget and assembly strategies - context set: 12 new ACMS pipeline options (hot_max_tokens, warm_max_decisions, cold_max_decisions, summary_max_tokens, temporal_scope, auto_refresh, focus_area, breadth, depth, assembly_strategy, retrieval_strategy, summary_strategy) - context show: displays ACMS pipeline configuration alongside policy - `context inspect` displayed global tier fragment counts (hot/warm/cold) across all projects instead of counts for the target project only. Add `ContextTierService.get_scoped_metrics(project_names)` which uses `ScopedBackendView` to filter fragment counts to the specified projects while keeping hit/miss counters as global service-level cache metrics. Update `context_inspect()` to call `get_scoped_metrics([project])` instead of `get_metrics()`. - `context simulate --focus` accepted focus URIs and passed them to the `ContextRequest` model but never used them to filter the fragment list, making the `--focus` flag a no-op. Add focus URI filtering in `_simulate_context_assembly()` after `get_scoped_view()` — filters `project_fragments` by matching each fragment's `resource_id` against the supplied focus URIs. Also fixes redaction false positives for hot_max_tokens and summary_max_tokens keys, and Rich Console line-wrapping in test output that caused JSON parse failures. Includes 28 new Behave BDD scenarios for wiring coverage, updated Robot Framework integration tests, and reference documentation. ISSUES CLOSED: #499
156 lines
6.7 KiB
Gherkin
156 lines
6.7 KiB
Gherkin
Feature: M5 ACMS pipeline and large-project context smoke tests
|
|
As a developer working with the CleverAgents M5 milestone
|
|
I want to verify the ACMS context pipeline end-to-end
|
|
So that context assembly, budget enforcement, and multi-project context work correctly
|
|
|
|
Background:
|
|
Given a m5 smoke test runner
|
|
And a m5 smoke mocked context service
|
|
|
|
# --- Fixture loading ---
|
|
|
|
Scenario: M5 smoke load ACMS context policy fixture
|
|
When I m5 smoke load the ACMS context policy fixture
|
|
Then the m5 smoke policy fixture should have a default view
|
|
And the m5 smoke policy fixture should have a strategize view
|
|
And the m5 smoke policy fixture should have an execute view
|
|
|
|
Scenario: M5 smoke load large project context fixture
|
|
When I m5 smoke load the large project context fixture
|
|
Then the m5 smoke large project fixture should have file entries
|
|
And the m5 smoke large project fixture should have context tiers
|
|
|
|
Scenario: M5 smoke load context analysis results fixture
|
|
When I m5 smoke load the context analysis results fixture
|
|
Then the m5 smoke analysis fixture should have a summary
|
|
And the m5 smoke analysis fixture should have dependencies
|
|
And the m5 smoke analysis fixture should have relevance scores
|
|
|
|
# --- Context policy resolution ---
|
|
|
|
Scenario: M5 smoke resolve default view from empty policy
|
|
Given a m5 smoke empty context policy
|
|
When I m5 smoke resolve the view for phase "default"
|
|
Then the m5 smoke resolved view should include all paths
|
|
|
|
Scenario: M5 smoke resolve strategize view inherits from default
|
|
Given a m5 smoke policy with only default view
|
|
When I m5 smoke resolve the view for phase "strategize"
|
|
Then the m5 smoke resolved view should match the default view
|
|
|
|
Scenario: M5 smoke resolve strategize view with override
|
|
Given a m5 smoke policy with strategize override
|
|
When I m5 smoke resolve the view for phase "strategize"
|
|
Then the m5 smoke resolved view should use the strategize override
|
|
|
|
Scenario: M5 smoke resolve execute view inherits from strategize
|
|
Given a m5 smoke policy with strategize override
|
|
When I m5 smoke resolve the view for phase "execute"
|
|
Then the m5 smoke resolved view should use the strategize override
|
|
|
|
Scenario: M5 smoke resolve apply view falls through to default
|
|
Given a m5 smoke policy with only default view
|
|
When I m5 smoke resolve the view for phase "apply"
|
|
Then the m5 smoke resolved view should match the default view
|
|
|
|
Scenario: M5 smoke invalid phase raises ValueError
|
|
Given a m5 smoke empty context policy
|
|
When I m5 smoke resolve the view for invalid phase "invalid"
|
|
Then a m5 smoke ValueError should be raised
|
|
|
|
# --- Budget enforcement ---
|
|
|
|
Scenario: M5 smoke max file size rejects oversized files
|
|
Given a m5 smoke context view with max_file_size 1024
|
|
Then a m5 smoke file of size 2048 should exceed the budget
|
|
And a m5 smoke file of size 512 should be within the budget
|
|
|
|
Scenario: M5 smoke max total size limits aggregate context
|
|
Given a m5 smoke context view with max_total_size 8192
|
|
Then a m5 smoke aggregate of size 10000 should exceed the budget
|
|
And a m5 smoke aggregate of size 4096 should be within the budget
|
|
|
|
Scenario: M5 smoke zero max file size is invalid
|
|
When I m5 smoke create a context view with max_file_size 0
|
|
Then a m5 smoke validation error should be raised
|
|
|
|
Scenario: M5 smoke negative max total size is invalid
|
|
When I m5 smoke create a context view with max_total_size -1
|
|
Then a m5 smoke validation error should be raised
|
|
|
|
Scenario: M5 smoke None size limits allow unlimited
|
|
Given a m5 smoke context view with no size limits
|
|
Then a m5 smoke file of size 999999999 should be within the budget
|
|
And a m5 smoke aggregate of size 999999999 should be within the budget
|
|
|
|
# --- Context assembly via CLI ---
|
|
|
|
Scenario: M5 smoke context list with empty project
|
|
Given a m5 smoke project with no context
|
|
When I m5 smoke invoke context list
|
|
Then the m5 smoke context list should succeed
|
|
And the m5 smoke context list output should be empty
|
|
|
|
Scenario: M5 smoke context add file to project
|
|
Given a m5 smoke project with mocked context service
|
|
When I m5 smoke invoke context add with path "src/main.py"
|
|
Then the m5 smoke context add should succeed
|
|
|
|
Scenario: M5 smoke context show displays file content
|
|
Given a m5 smoke project with context entries
|
|
When I m5 smoke invoke context show for path "src/main.py"
|
|
Then the m5 smoke context show should succeed
|
|
And the m5 smoke context show output should contain content
|
|
|
|
Scenario: M5 smoke context clear removes all entries
|
|
Given a m5 smoke project with context entries
|
|
When I m5 smoke invoke context clear
|
|
Then the m5 smoke context clear should succeed
|
|
|
|
# --- Project context policy CLI ---
|
|
|
|
Scenario: M5 smoke project context show displays policy
|
|
Given a m5 smoke project with a saved context policy
|
|
When I m5 smoke invoke project context show
|
|
Then the m5 smoke project context show should succeed
|
|
And the m5 smoke project context output should contain phase views
|
|
|
|
Scenario: M5 smoke project context inspect is wired to ACMS
|
|
When I m5 smoke invoke project context inspect
|
|
Then the m5 smoke project context inspect should succeed
|
|
|
|
Scenario: M5 smoke project context simulate is wired to ACMS
|
|
When I m5 smoke invoke project context simulate
|
|
Then the m5 smoke project context simulate should succeed
|
|
|
|
# --- Context analysis agent ---
|
|
|
|
Scenario: M5 smoke context analysis produces summary
|
|
Given a m5 smoke mocked context analysis agent
|
|
When I m5 smoke invoke context analysis
|
|
Then the m5 smoke analysis result should have a non-empty summary
|
|
|
|
Scenario: M5 smoke context analysis produces dependencies
|
|
Given a m5 smoke mocked context analysis agent
|
|
When I m5 smoke invoke context analysis
|
|
Then the m5 smoke analysis result should have dependency entries
|
|
|
|
Scenario: M5 smoke context analysis produces relevance scores
|
|
Given a m5 smoke mocked context analysis agent
|
|
When I m5 smoke invoke context analysis
|
|
Then the m5 smoke analysis result should have relevance scores
|
|
And all m5 smoke relevance scores should be between 0 and 1
|
|
|
|
# --- Multi-project context ---
|
|
|
|
Scenario: M5 smoke multiple projects have independent context
|
|
Given m5 smoke project "proj-a" with 3 context files
|
|
And m5 smoke project "proj-b" with 5 context files
|
|
Then m5 smoke project "proj-a" should have 3 context entries
|
|
And m5 smoke project "proj-b" should have 5 context entries
|
|
|
|
Scenario: M5 smoke context exclusion patterns filter correctly
|
|
Given a m5 smoke context view excluding "**/__pycache__/**"
|
|
Then the m5 smoke path "src/__pycache__/module.pyc" should be excluded
|
|
And the m5 smoke path "src/module.py" should not be excluded
|