34c2acc354
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m18s
CI / typecheck (pull_request) Successful in 4m3s
CI / security (pull_request) Successful in 4m13s
CI / build (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 3m42s
CI / integration_tests (pull_request) Successful in 8m50s
CI / unit_tests (pull_request) Successful in 9m14s
CI / e2e_tests (pull_request) Successful in 9m28s
CI / docker (pull_request) Successful in 1m9s
CI / coverage (pull_request) Successful in 10m24s
CI / status-check (pull_request) Successful in 3s
CI / benchmark-regression (pull_request) Successful in 48m36s
CI / build (push) Successful in 22s
CI / lint (push) Successful in 3m43s
CI / typecheck (push) Successful in 4m19s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m28s
CI / quality (push) Successful in 4m21s
CI / integration_tests (push) Successful in 9m12s
CI / e2e_tests (push) Successful in 9m29s
CI / unit_tests (push) Successful in 9m36s
CI / docker (push) Successful in 1m8s
CI / coverage (push) Successful in 10m33s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 25m40s
Implement the missing runtime logic for the ACMS context tier service. Previously only data models and manual promote()/demote()/evict_lru() methods existed. This commit adds: - Auto-promotion on access: get() now promotes fragments one tier up when access_count reaches the configurable promotion_threshold (default: 5 accesses). The access counter resets after each successful promotion so fragments must accumulate fresh accesses before the next tier transition. - Staleness enforcement: new enforce_staleness() method demotes hot fragments older than hot_ttl (default: 24h) to warm, and warm fragments older than warm_ttl to cold. A snapshot of existing warm-tier IDs prevents double-demotion in a single pass. - Budget enforcement on store and promote: store() and promote() now enforce TierBudget.max_tokens_hot by evicting LRU hot-tier fragments until the token budget is met. The eviction loop uses incremental token tracking to avoid recomputing the sum. - Event emission: Added TIER_PROMOTED, TIER_DEMOTED, TIER_EVICTED event types to EventType enum. All tier transitions emit DomainEvent instances through the optional EventBus. - Configuration: Added context_tier_promotion_threshold, context_tier_hot_ttl_hours, context_tier_warm_ttl_hours settings. The warm TTL setting also accepts the spec-defined env var CLEVERAGENTS_CTX_WARM_HOURS as an alias. - DI wiring: container.py now injects event_bus into context_tier_service. Review fixes applied (code review on PR #1150): - C1: Reset access_count to 0 after each auto-promotion to prevent chain promotion that bypassed the warm tier. - C2: Call _enforce_hot_budget() inside promote() warm-to-hot path so auto-promoted fragments respect the token budget. - H1: Corrected _enforce_hot_budget() docstring: actual complexity is O(n + n*k) not O(n), since min() scans remaining entries on each eviction. - M1: Added CLEVERAGENTS_CTX_WARM_HOURS as an additional env var alias for context_tier_warm_ttl_hours per specification line 30555. Review fixes applied (second code review on PR #1150): - B-CRIT-1: Fixed data loss in promote() warm-to-hot: emit TIER_PROMOTED before _enforce_hot_budget(), and if the promoted fragment is evicted by budget, restore it to the warm tier instead of silently losing it. - B-HIGH-1: Fixed self-eviction on store(): fragments whose token_count exceeds the entire hot-tier budget are now redirected to the warm tier with a warning log. - B-MED-1: Wrapped _emit_tier_event() in try/except so a failing event bus does not break tier operations (best-effort emission). - B-MED-2: Fixed event ordering so TIER_PROMOTED fires before any budget-triggered TIER_EVICTED events. - D-LOW-1: Fixed type hint in Robot helper (dict[str, Callable]). - D-LOW-2: Added __all__ export to context_tiers.py. - S-LOW-1: Added thread-safety docstring note to ContextTierService. Review fixes applied (third code review on PR #1150): - B-MED-1: Added TIER_DEMOTED event emission for oversized fragment redirect in store(), closing the observability gap where the only tier transition without event emission was the hot-to-warm redirect for fragments exceeding the entire hot-tier budget. - S-LOW-1: Added CLEVERAGENTS_CTX_HOT_HOURS as an additional env var alias for context_tier_hot_ttl_hours, for consistency with the warm-tier alias CLEVERAGENTS_CTX_WARM_HOURS. - S-LOW-2: Added docstring note to enforce_staleness() reconciling the hot-tier TTL with the specification statement that hot-tier retention is "Until resource removed" (TTL controls tier placement, not data retention). Review fixes applied (fourth code review on PR #1150): - B-HIGH-1: Reset access_count to 0 on demotion so that demoted fragments must accumulate fresh accesses before re-promotion. Without this reset, a previously popular fragment whose access_count already exceeded the promotion threshold would be re-promoted on the very next get() call, making staleness enforcement ineffective. Review fixes applied (freemo APPROVED review on PR #1150): - #1: Removed all # type: ignore annotations from test files. Fixed _EventCollector, _FailingBus, and _NullBus subscribe() signatures to use Callable[[DomainEvent], None] matching the EventBus protocol. Replaced dict-spread TieredFragment construction with explicit keyword arguments and post-construction assignment. - #2: Extracted runtime policy logic (enforce_staleness, _maybe_auto_promote, _re_fetch_after_promotion, _enforce_hot_budget, _emit_tier_event) into TierRuntimeMixin in tier_runtime.py to reduce context_tiers.py toward the 500-line guideline. - #3: Added fragment_id non-empty validation guard to promote() and demote() per CONTRIBUTING.md argument validation policy. - #8: Renamed _resolve to _re_fetch_after_promotion for clarity. Removed @tdd_expected_fail from TDD tests (Behave + Robot) as the bug is now fixed. All 3 TDD scenarios pass normally. Tests: 27 Behave scenarios (24 feature + 3 TDD), 4 Robot integration tests, 4 ASV benchmark suites. ISSUES CLOSED: #821
368 lines
12 KiB
Python
368 lines
12 KiB
Python
"""Step definitions for context tier runtime promotion/demotion/eviction.
|
|
|
|
Covers the runtime logic added by issue #821: auto-promotion on access,
|
|
time-based staleness enforcement, budget-based eviction on store, and
|
|
tier transition event emission via the ``EventBus``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Callable
|
|
from datetime import UTC, datetime, timedelta
|
|
from typing import Any
|
|
|
|
from behave import given, then, when
|
|
|
|
from cleveragents.application.services.context_tiers import ContextTierService
|
|
from cleveragents.domain.models.acms.tiers import (
|
|
ContextTier,
|
|
TierBudget,
|
|
TieredFragment,
|
|
)
|
|
from cleveragents.infrastructure.events.models import DomainEvent
|
|
from cleveragents.infrastructure.events.types import EventType
|
|
|
|
__all__: list[str] = []
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _make_fragment(
|
|
fragment_id: str,
|
|
tier: ContextTier,
|
|
token_count: int = 50,
|
|
content: str = "test content",
|
|
last_accessed: datetime | None = None,
|
|
) -> TieredFragment:
|
|
"""Create a ``TieredFragment`` with the given properties."""
|
|
frag = TieredFragment(
|
|
fragment_id=fragment_id,
|
|
content=content,
|
|
tier=tier,
|
|
token_count=token_count,
|
|
project_name="test-project",
|
|
)
|
|
if last_accessed is not None:
|
|
frag.last_accessed = last_accessed
|
|
return frag
|
|
|
|
|
|
class _EventCollector:
|
|
"""Simple event bus replacement that collects emitted events."""
|
|
|
|
def __init__(self) -> None:
|
|
self.events: list[DomainEvent] = []
|
|
|
|
def emit(self, event: DomainEvent) -> None:
|
|
self.events.append(event)
|
|
|
|
def subscribe(
|
|
self,
|
|
event_type: EventType,
|
|
handler: Callable[[DomainEvent], None],
|
|
) -> None:
|
|
pass # Not needed for test assertions
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Background / Given steps
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given("a context tier service with default settings")
|
|
def step_service_defaults(context: Any) -> None:
|
|
context.tier_service = ContextTierService(settings=None)
|
|
|
|
|
|
@given('a fragment "{fid}" stored in the cold tier with {tokens:d} tokens')
|
|
def step_store_cold(context: Any, fid: str, tokens: int) -> None:
|
|
frag = _make_fragment(fid, ContextTier.COLD, token_count=tokens)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given('a fragment "{fid}" stored in the warm tier with {tokens:d} tokens')
|
|
def step_store_warm(context: Any, fid: str, tokens: int) -> None:
|
|
frag = _make_fragment(fid, ContextTier.WARM, token_count=tokens)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given('a fragment "{fid}" stored in the hot tier with {tokens:d} tokens')
|
|
def step_store_hot(context: Any, fid: str, tokens: int) -> None:
|
|
frag = _make_fragment(fid, ContextTier.HOT, token_count=tokens)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given(
|
|
'a fragment "{fid}" stored in the hot tier with stale timestamp of {hours:d} hours'
|
|
)
|
|
def step_store_hot_stale(context: Any, fid: str, hours: int) -> None:
|
|
stale = datetime.now(tz=UTC) - timedelta(hours=hours)
|
|
frag = _make_fragment(fid, ContextTier.HOT, last_accessed=stale)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given('a fragment "{fid}" stored in the hot tier with fresh timestamp')
|
|
def step_store_hot_fresh(context: Any, fid: str) -> None:
|
|
frag = _make_fragment(fid, ContextTier.HOT)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given(
|
|
'a fragment "{fid}" stored in the warm tier with stale timestamp of {hours:d} hours'
|
|
)
|
|
def step_store_warm_stale(context: Any, fid: str, hours: int) -> None:
|
|
stale = datetime.now(tz=UTC) - timedelta(hours=hours)
|
|
frag = _make_fragment(fid, ContextTier.WARM, last_accessed=stale)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given("a service with hot tier budget of {tokens:d} tokens")
|
|
def step_service_custom_budget(context: Any, tokens: int) -> None:
|
|
svc = ContextTierService(settings=None)
|
|
svc._budget = TierBudget(max_tokens_hot=tokens)
|
|
context.tier_service = svc
|
|
|
|
|
|
@given(
|
|
'a fragment "{fid}" stored in the hot tier with {tokens:d} tokens and old timestamp'
|
|
)
|
|
def step_store_hot_old(context: Any, fid: str, tokens: int) -> None:
|
|
old = datetime.now(tz=UTC) - timedelta(minutes=10)
|
|
frag = _make_fragment(fid, ContextTier.HOT, token_count=tokens, last_accessed=old)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given(
|
|
'a fragment "{fid}" stored in the hot tier'
|
|
" with {tokens:d} tokens and recent timestamp"
|
|
)
|
|
def step_store_hot_recent(context: Any, fid: str, tokens: int) -> None:
|
|
frag = _make_fragment(fid, ContextTier.HOT, token_count=tokens)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@given("a context tier service with an event bus")
|
|
def step_service_with_bus(context: Any) -> None:
|
|
collector = _EventCollector()
|
|
svc = ContextTierService(settings=None, event_bus=collector)
|
|
context.tier_service = svc
|
|
context.event_collector = collector
|
|
|
|
|
|
@given("a context tier service with an event bus and hot budget of {tokens:d} tokens")
|
|
def step_service_with_bus_budget(context: Any, tokens: int) -> None:
|
|
collector = _EventCollector()
|
|
svc = ContextTierService(settings=None, event_bus=collector)
|
|
svc._budget = TierBudget(max_tokens_hot=tokens)
|
|
context.tier_service = svc
|
|
context.event_collector = collector
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# When steps
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when('I access "{fid}" via get {n:d} times')
|
|
def step_access_n(context: Any, fid: str, n: int) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
for _ in range(n):
|
|
svc.get(fid)
|
|
|
|
|
|
@when("I invoke enforce_staleness on the service")
|
|
def step_enforce_staleness(context: Any) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
context.staleness_result = svc.enforce_staleness()
|
|
|
|
|
|
@when('I store fragment "{fid}" in the hot tier with {tokens:d} tokens')
|
|
def step_store_hot_overflow(context: Any, fid: str, tokens: int) -> None:
|
|
frag = _make_fragment(fid, ContextTier.HOT, token_count=tokens)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@when('I promote fragment "{fid}"')
|
|
def step_promote(context: Any, fid: str) -> None:
|
|
context.tier_service.promote(fid)
|
|
|
|
|
|
@when('I demote fragment "{fid}"')
|
|
def step_demote(context: Any, fid: str) -> None:
|
|
context.tier_service.demote(fid)
|
|
|
|
|
|
@when("I evict {count:d} LRU fragment from the hot tier")
|
|
def step_evict_lru_hot(context: Any, count: int) -> None:
|
|
context.tier_service.evict_lru(ContextTier.HOT, count)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Then steps
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@then('fragment "{fid}" should be in the warm tier')
|
|
def step_in_warm(context: Any, fid: str) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
frag = svc._find_fragment(fid)
|
|
assert frag is not None, f"Fragment '{fid}' not found in any tier"
|
|
assert frag.tier == ContextTier.WARM, f"Expected '{fid}' in WARM, got {frag.tier}"
|
|
|
|
|
|
@then('fragment "{fid}" should be in the hot tier')
|
|
def step_in_hot(context: Any, fid: str) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
frag = svc._find_fragment(fid)
|
|
assert frag is not None, f"Fragment '{fid}' not found in any tier"
|
|
assert frag.tier == ContextTier.HOT, f"Expected '{fid}' in HOT, got {frag.tier}"
|
|
|
|
|
|
@then('fragment "{fid}" should be in the cold tier')
|
|
def step_in_cold(context: Any, fid: str) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
frag = svc._find_fragment(fid)
|
|
assert frag is not None, f"Fragment '{fid}' not found in any tier"
|
|
assert frag.tier == ContextTier.COLD, f"Expected '{fid}' in COLD, got {frag.tier}"
|
|
|
|
|
|
@then('fragment "{fid}" should not be in any tier')
|
|
def step_not_in_any(context: Any, fid: str) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
frag = svc._find_fragment(fid)
|
|
assert frag is None, f"Expected '{fid}' to be evicted, but found in {frag.tier}"
|
|
|
|
|
|
@then("the staleness result should contain {n:d} fragment IDs")
|
|
def step_staleness_count(context: Any, n: int) -> None:
|
|
result: list[str] = context.staleness_result
|
|
assert len(result) == n, (
|
|
f"Expected {n} demoted fragments, got {len(result)}: {result}"
|
|
)
|
|
|
|
|
|
@then('a TIER_PROMOTED event should have been emitted for "{fid}"')
|
|
def step_promoted_event(context: Any, fid: str) -> None:
|
|
collector: _EventCollector = context.event_collector
|
|
promoted = [
|
|
e
|
|
for e in collector.events
|
|
if e.event_type == EventType.TIER_PROMOTED
|
|
and e.details.get("fragment_id") == fid
|
|
]
|
|
assert len(promoted) >= 1, (
|
|
f"No TIER_PROMOTED event for '{fid}'. "
|
|
f"Events: {[e.event_type for e in collector.events]}"
|
|
)
|
|
|
|
|
|
@then('a TIER_DEMOTED event should have been emitted for "{fid}"')
|
|
def step_demoted_event(context: Any, fid: str) -> None:
|
|
collector: _EventCollector = context.event_collector
|
|
demoted = [
|
|
e
|
|
for e in collector.events
|
|
if e.event_type == EventType.TIER_DEMOTED
|
|
and e.details.get("fragment_id") == fid
|
|
]
|
|
assert len(demoted) >= 1, (
|
|
f"No TIER_DEMOTED event for '{fid}'. "
|
|
f"Events: {[e.event_type for e in collector.events]}"
|
|
)
|
|
|
|
|
|
@then('a TIER_EVICTED event should have been emitted for "{fid}"')
|
|
def step_evicted_event(context: Any, fid: str) -> None:
|
|
collector: _EventCollector = context.event_collector
|
|
evicted = [
|
|
e
|
|
for e in collector.events
|
|
if e.event_type == EventType.TIER_EVICTED
|
|
and e.details.get("fragment_id") == fid
|
|
]
|
|
assert len(evicted) >= 1, (
|
|
f"No TIER_EVICTED event for '{fid}'. "
|
|
f"Events: {[e.event_type for e in collector.events]}"
|
|
)
|
|
|
|
|
|
@then(
|
|
'the last TIER_PROMOTED event should have from_tier "{from_t}" and to_tier "{to_t}"'
|
|
)
|
|
def step_promoted_event_details(context: Any, from_t: str, to_t: str) -> None:
|
|
collector: _EventCollector = context.event_collector
|
|
promoted = [e for e in collector.events if e.event_type == EventType.TIER_PROMOTED]
|
|
assert len(promoted) >= 1, "No TIER_PROMOTED events"
|
|
last = promoted[-1]
|
|
assert last.details["from_tier"] == from_t, (
|
|
f"Expected from_tier='{from_t}', got '{last.details['from_tier']}'"
|
|
)
|
|
assert last.details["to_tier"] == to_t, (
|
|
f"Expected to_tier='{to_t}', got '{last.details['to_tier']}'"
|
|
)
|
|
|
|
|
|
@then(
|
|
'the last TIER_DEMOTED event should have from_tier "{from_t}" and to_tier "{to_t}"'
|
|
)
|
|
def step_demoted_event_details(context: Any, from_t: str, to_t: str) -> None:
|
|
collector: _EventCollector = context.event_collector
|
|
demoted = [e for e in collector.events if e.event_type == EventType.TIER_DEMOTED]
|
|
assert len(demoted) >= 1, "No TIER_DEMOTED events"
|
|
last = demoted[-1]
|
|
assert last.details["from_tier"] == from_t, (
|
|
f"Expected from_tier='{from_t}', got '{last.details['from_tier']}'"
|
|
)
|
|
assert last.details["to_tier"] == to_t, (
|
|
f"Expected to_tier='{to_t}', got '{last.details['to_tier']}'"
|
|
)
|
|
|
|
|
|
class _FailingBus:
|
|
"""Event bus stub that raises on every emit() call."""
|
|
|
|
def emit(self, event: DomainEvent) -> None:
|
|
msg = "Simulated event bus failure"
|
|
raise RuntimeError(msg)
|
|
|
|
def subscribe(
|
|
self,
|
|
event_type: EventType,
|
|
handler: Callable[[DomainEvent], None],
|
|
) -> None:
|
|
pass
|
|
|
|
|
|
@given("a context tier service with a failing event bus")
|
|
def step_service_with_failing_bus(context: Any) -> None:
|
|
bus = _FailingBus()
|
|
svc = ContextTierService(settings=None, event_bus=bus)
|
|
context.tier_service = svc
|
|
|
|
|
|
@given(
|
|
'a fragment "{fid}" stored in the warm tier'
|
|
" with {tokens:d} tokens and old timestamp"
|
|
)
|
|
def step_store_warm_old(context: Any, fid: str, tokens: int) -> None:
|
|
old = datetime.now(tz=UTC) - timedelta(minutes=10)
|
|
frag = _make_fragment(fid, ContextTier.WARM, token_count=tokens, last_accessed=old)
|
|
context.tier_service.store(frag)
|
|
|
|
|
|
@when("I invoke enforce_staleness on the service with a hot TTL of {hours:d} hours")
|
|
def step_enforce_staleness_custom_ttl(context: Any, hours: int) -> None:
|
|
svc: ContextTierService = context.tier_service
|
|
svc._hot_ttl = timedelta(hours=hours)
|
|
context.staleness_result = svc.enforce_staleness()
|
|
|
|
|
|
@given("a context tier service with promotion threshold {threshold:d}")
|
|
def step_service_custom_threshold(context: Any, threshold: int) -> None:
|
|
svc = ContextTierService(settings=None)
|
|
svc._promotion_threshold = threshold
|
|
context.tier_service = svc
|