brent.edwards
6531440431
CI / build (push) Successful in 24s
CI / lint (push) Successful in 3m16s
CI / quality (push) Successful in 3m45s
CI / typecheck (push) Successful in 3m52s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m1s
CI / unit_tests (push) Successful in 7m8s
CI / integration_tests (push) Successful in 7m1s
CI / docker (push) Successful in 1m6s
CI / e2e_tests (push) Successful in 9m49s
CI / coverage (push) Successful in 12m22s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 19m34s
## Summary Implement the estimation actor as a functional actor type. The estimation actor provides cost, time, and resource estimates for plan operations, running after Strategize completes (before Execute). Estimation is informational only and optional — plans work without an estimation actor configured. ### Changes **New files:** - `src/cleveragents/domain/models/core/estimation.py` — `EstimationResult` frozen Pydantic model with fields for cost (USD), tokens, steps, child plans, time, risk level/factors, summary - `features/estimation_actor.feature` — 12 Behave test scenarios (model validation, serialization, stub actor, plan integration, optional behavior) - `features/steps/estimation_actor_steps.py` — Step definitions - `robot/estimation_actor.robot` — 6 Robot integration tests - `robot/helper_estimation_actor.py` — Robot test helper **Modified files:** - `domain/models/acms/tiers.py` — Added `ESTIMATOR` to `ActorRole` enum - `domain/models/core/plan.py` — Added `estimation_result: EstimationResult | None` field, estimation data in `as_cli_dict()` - `application/services/plan_executor.py` — Added `EstimationStubActor` class - `application/services/plan_lifecycle_service.py` — Added `_run_estimation()` method, invoked in `execute_plan()` - `cli/commands/plan.py` — Display estimation results in plan status - `vulture_whitelist.py` — Added 12 new public symbols ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,818 scenarios) | | `nox -s integration_tests` | PASS (1,512 tests) | | `nox -s coverage_report` | 98% (>= 97%) | Closes #890 Reviewed-on: #962 Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
192 lines
6.8 KiB
Gherkin
192 lines
6.8 KiB
Gherkin
Feature: ACMS Context Tiers
|
|
As a developer
|
|
I want hot/warm/cold context tiers with actor views and project scoping
|
|
So that the context management system can efficiently manage fragment lifecycle
|
|
|
|
# ---- ContextTier enum ----
|
|
|
|
Scenario: ContextTier has three values
|
|
Then ContextTier should have values "hot", "warm", "cold"
|
|
|
|
# ---- ActorRole enum ----
|
|
|
|
Scenario: ActorRole has four values
|
|
Then ActorRole should have values "strategist", "executor", "reviewer", "estimator"
|
|
|
|
# ---- TieredFragment ----
|
|
|
|
Scenario: Create a valid TieredFragment
|
|
Given a TieredFragment with id "frag-001" and content "hello world"
|
|
Then the fragment tier should be "hot"
|
|
And the fragment token_count should be 0
|
|
And the fragment access_count should be 0
|
|
|
|
Scenario: TieredFragment rejects empty fragment_id
|
|
Then creating a TieredFragment with empty fragment_id should raise ValidationError
|
|
|
|
Scenario: TieredFragment rejects negative token_count
|
|
Then creating a TieredFragment with negative token_count should raise ValidationError
|
|
|
|
# ---- TierBudget ----
|
|
|
|
Scenario: TierBudget has sensible defaults
|
|
Given a default TierBudget
|
|
Then max_tokens_hot should be 8000
|
|
And max_decisions_warm should be 500
|
|
And max_decisions_cold should be 5000
|
|
|
|
# ---- ActorContextView ----
|
|
|
|
Scenario: Strategist sees all tiers by default
|
|
Given an ActorContextView for "strategist"
|
|
Then effective tiers should be "hot", "warm", "cold"
|
|
|
|
Scenario: Executor sees hot and warm by default
|
|
Given an ActorContextView for "executor"
|
|
Then effective tiers should be "hot", "warm"
|
|
|
|
Scenario: Reviewer sees only hot by default
|
|
Given an ActorContextView for "reviewer"
|
|
Then effective tiers should be "hot"
|
|
|
|
# ---- TierMetrics ----
|
|
|
|
Scenario: TierMetrics defaults to zero
|
|
Given a default TierMetrics
|
|
Then total_fragments should be 0
|
|
And hot_hit_rate should be 0.0
|
|
|
|
# ---- ScopedBackendView ----
|
|
|
|
Scenario: ScopedBackendView filters by project
|
|
Given a ScopedBackendView for project "proj-a"
|
|
And a TieredFragment with id "frag-a" in project "proj-a"
|
|
And a TieredFragment with id "frag-b" in project "proj-b"
|
|
Then fragment "frag-a" should be visible
|
|
And fragment "frag-b" should not be visible
|
|
|
|
Scenario: ScopedBackendView excludes empty project_name
|
|
Given a ScopedBackendView for project "proj-a"
|
|
And a TieredFragment with id "frag-x" and no project
|
|
Then fragment "frag-x" should not be visible
|
|
|
|
# ---- ContextTierService: store/get ----
|
|
|
|
Scenario: Store and retrieve a fragment
|
|
Given a ContextTierService
|
|
When I store a fragment with id "f1" content "data" in tier "hot"
|
|
Then tier getting "f1" should return content "data"
|
|
|
|
Scenario: Get returns None for unknown fragment
|
|
Given a ContextTierService
|
|
Then tier getting "nonexistent" should return None
|
|
|
|
# ---- ContextTierService: promotion ----
|
|
|
|
Scenario: Promote from cold to warm
|
|
Given a ContextTierService
|
|
When I store a fragment with id "f2" content "cold-data" in tier "cold"
|
|
And I promote "f2"
|
|
Then "f2" should be in tier "warm"
|
|
|
|
Scenario: Promote from warm to hot
|
|
Given a ContextTierService
|
|
When I store a fragment with id "f3" content "warm-data" in tier "warm"
|
|
And I promote "f3"
|
|
Then "f3" should be in tier "hot"
|
|
|
|
Scenario: Promote unknown fragment returns None
|
|
Given a ContextTierService
|
|
Then promoting "unknown" should return None
|
|
|
|
# ---- ContextTierService: demotion ----
|
|
|
|
Scenario: Demote from hot to warm
|
|
Given a ContextTierService
|
|
When I store a fragment with id "f4" content "hot-data" in tier "hot"
|
|
And I demote "f4"
|
|
Then "f4" should be in tier "warm"
|
|
|
|
Scenario: Demote from warm to cold with summarization
|
|
Given a ContextTierService
|
|
When I store a long fragment "f5" in tier "warm"
|
|
And I demote "f5"
|
|
Then "f5" should be in tier "cold"
|
|
And "f5" content should be summarized
|
|
|
|
Scenario: Demote unknown fragment returns None
|
|
Given a ContextTierService
|
|
Then demoting "unknown" should return None
|
|
|
|
# ---- ContextTierService: LRU eviction ----
|
|
|
|
Scenario: Evict LRU fragments from hot tier
|
|
Given a ContextTierService
|
|
When I store a fragment with id "e1" content "first" in tier "hot"
|
|
And I store a fragment with id "e2" content "second" in tier "hot"
|
|
And I store a fragment with id "e3" content "third" in tier "hot"
|
|
And I evict 2 LRU fragments from "hot"
|
|
Then "hot" tier should have 1 fragment
|
|
|
|
Scenario: Evict with invalid count raises ValueError
|
|
Given a ContextTierService
|
|
Then evicting 0 from "hot" should raise ValueError
|
|
|
|
# ---- ContextTierService: actor views ----
|
|
|
|
Scenario: Strategist sees fragments from all tiers
|
|
Given a ContextTierService
|
|
When I store fragment "a1" with content "hot" tier "hot" project "p1"
|
|
And I store fragment "a2" with content "warm" tier "warm" project "p1"
|
|
And I store fragment "a3" with content "cold" tier "cold" project "p1"
|
|
Then strategist for project "p1" should see 3 fragments
|
|
|
|
Scenario: Executor sees only hot and warm
|
|
Given a ContextTierService
|
|
When I store fragment "b1" with content "hot" tier "hot" project "p1"
|
|
And I store fragment "b2" with content "warm" tier "warm" project "p1"
|
|
And I store fragment "b3" with content "cold" tier "cold" project "p1"
|
|
Then executor for project "p1" should see 2 fragments
|
|
|
|
Scenario: Reviewer sees only hot
|
|
Given a ContextTierService
|
|
When I store fragment "c1" with content "hot" tier "hot" project "p1"
|
|
And I store fragment "c2" with content "warm" tier "warm" project "p1"
|
|
Then reviewer for project "p1" should see 1 fragment
|
|
|
|
# ---- ContextTierService: project scoping ----
|
|
|
|
Scenario: Scoped view filters by project
|
|
Given a ContextTierService
|
|
When I store fragment "s1" with content "yes" tier "hot" project "alpha"
|
|
And I store fragment "s2" with content "no" tier "hot" project "beta"
|
|
Then scoped view for "alpha" should have 1 fragment
|
|
|
|
# ---- ContextTierService: metrics ----
|
|
|
|
Scenario: Metrics track hits and misses
|
|
Given a ContextTierService
|
|
When I store a fragment with id "m1" content "data" in tier "hot"
|
|
And I get fragment "m1"
|
|
And I get fragment "nonexistent"
|
|
Then hot_hit_count should be 1
|
|
And hot_miss_count should be 1
|
|
|
|
# ---- DI Container ----
|
|
|
|
Scenario: DI container provides context_tier_service
|
|
Given the DI container is reset
|
|
Then the container should provide a context_tier_service
|
|
And the context_tier_service should be a ContextTierService
|
|
|
|
Scenario: DI container context_tier_service is singleton
|
|
Given the DI container is reset
|
|
Then resolving context_tier_service twice should return the same instance
|
|
|
|
# ---- Settings ----
|
|
|
|
Scenario: Settings include context tier budget fields
|
|
Then settings should have context_max_tokens_hot
|
|
And settings should have context_max_decisions_warm
|
|
And settings should have context_max_decisions_cold
|