Files
cleveragents-core/robot/helper_context_tier_runtime.py
Luis Mendes 34c2acc354
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m18s
CI / typecheck (pull_request) Successful in 4m3s
CI / security (pull_request) Successful in 4m13s
CI / build (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 3m42s
CI / integration_tests (pull_request) Successful in 8m50s
CI / unit_tests (pull_request) Successful in 9m14s
CI / e2e_tests (pull_request) Successful in 9m28s
CI / docker (pull_request) Successful in 1m9s
CI / coverage (pull_request) Successful in 10m24s
CI / status-check (pull_request) Successful in 3s
CI / benchmark-regression (pull_request) Successful in 48m36s
CI / build (push) Successful in 22s
CI / lint (push) Successful in 3m43s
CI / typecheck (push) Successful in 4m19s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m28s
CI / quality (push) Successful in 4m21s
CI / integration_tests (push) Successful in 9m12s
CI / e2e_tests (push) Successful in 9m29s
CI / unit_tests (push) Successful in 9m36s
CI / docker (push) Successful in 1m8s
CI / coverage (push) Successful in 10m33s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 25m40s
fix(acms): implement context tier runtime promotion/demotion/eviction
Implement the missing runtime logic for the ACMS context tier service.
Previously only data models and manual promote()/demote()/evict_lru()
methods existed. This commit adds:

- Auto-promotion on access: get() now promotes fragments one tier up
  when access_count reaches the configurable promotion_threshold
  (default: 5 accesses).  The access counter resets after each
  successful promotion so fragments must accumulate fresh accesses
  before the next tier transition.
- Staleness enforcement: new enforce_staleness() method demotes hot
  fragments older than hot_ttl (default: 24h) to warm, and warm
  fragments older than warm_ttl to cold. A snapshot of existing
  warm-tier IDs prevents double-demotion in a single pass.
- Budget enforcement on store and promote: store() and promote()
  now enforce TierBudget.max_tokens_hot by evicting LRU hot-tier
  fragments until the token budget is met.  The eviction loop uses
  incremental token tracking to avoid recomputing the sum.
- Event emission: Added TIER_PROMOTED, TIER_DEMOTED, TIER_EVICTED
  event types to EventType enum. All tier transitions emit
  DomainEvent instances through the optional EventBus.
- Configuration: Added context_tier_promotion_threshold,
  context_tier_hot_ttl_hours, context_tier_warm_ttl_hours settings.
  The warm TTL setting also accepts the spec-defined env var
  CLEVERAGENTS_CTX_WARM_HOURS as an alias.
- DI wiring: container.py now injects event_bus into
  context_tier_service.

Review fixes applied (code review on PR #1150):
- C1: Reset access_count to 0 after each auto-promotion to prevent
  chain promotion that bypassed the warm tier.
- C2: Call _enforce_hot_budget() inside promote() warm-to-hot path
  so auto-promoted fragments respect the token budget.
- H1: Corrected _enforce_hot_budget() docstring: actual complexity
  is O(n + n*k) not O(n), since min() scans remaining entries on
  each eviction.
- M1: Added CLEVERAGENTS_CTX_WARM_HOURS as an additional env var
  alias for context_tier_warm_ttl_hours per specification line 30555.

Review fixes applied (second code review on PR #1150):
- B-CRIT-1: Fixed data loss in promote() warm-to-hot: emit
  TIER_PROMOTED before _enforce_hot_budget(), and if the promoted
  fragment is evicted by budget, restore it to the warm tier instead
  of silently losing it.
- B-HIGH-1: Fixed self-eviction on store(): fragments whose
  token_count exceeds the entire hot-tier budget are now redirected
  to the warm tier with a warning log.
- B-MED-1: Wrapped _emit_tier_event() in try/except so a failing
  event bus does not break tier operations (best-effort emission).
- B-MED-2: Fixed event ordering so TIER_PROMOTED fires before any
  budget-triggered TIER_EVICTED events.
- D-LOW-1: Fixed type hint in Robot helper (dict[str, Callable]).
- D-LOW-2: Added __all__ export to context_tiers.py.
- S-LOW-1: Added thread-safety docstring note to ContextTierService.

Review fixes applied (third code review on PR #1150):
- B-MED-1: Added TIER_DEMOTED event emission for oversized fragment
  redirect in store(), closing the observability gap where the only
  tier transition without event emission was the hot-to-warm redirect
  for fragments exceeding the entire hot-tier budget.
- S-LOW-1: Added CLEVERAGENTS_CTX_HOT_HOURS as an additional env var
  alias for context_tier_hot_ttl_hours, for consistency with the
  warm-tier alias CLEVERAGENTS_CTX_WARM_HOURS.
- S-LOW-2: Added docstring note to enforce_staleness() reconciling
  the hot-tier TTL with the specification statement that hot-tier
  retention is "Until resource removed" (TTL controls tier placement,
  not data retention).

Review fixes applied (fourth code review on PR #1150):
- B-HIGH-1: Reset access_count to 0 on demotion so that demoted
  fragments must accumulate fresh accesses before re-promotion.
  Without this reset, a previously popular fragment whose
  access_count already exceeded the promotion threshold would be
  re-promoted on the very next get() call, making staleness
  enforcement ineffective.

Review fixes applied (freemo APPROVED review on PR #1150):
- #1: Removed all # type: ignore annotations from test files.
  Fixed _EventCollector, _FailingBus, and _NullBus subscribe()
  signatures to use Callable[[DomainEvent], None] matching the
  EventBus protocol.  Replaced dict-spread TieredFragment construction
  with explicit keyword arguments and post-construction assignment.
- #2: Extracted runtime policy logic (enforce_staleness,
  _maybe_auto_promote, _re_fetch_after_promotion, _enforce_hot_budget,
  _emit_tier_event) into TierRuntimeMixin in tier_runtime.py to
  reduce context_tiers.py toward the 500-line guideline.
- #3: Added fragment_id non-empty validation guard to promote() and
  demote() per CONTRIBUTING.md argument validation policy.
- #8: Renamed _resolve to _re_fetch_after_promotion for clarity.

Removed @tdd_expected_fail from TDD tests (Behave + Robot) as the
bug is now fixed. All 3 TDD scenarios pass normally.

Tests: 27 Behave scenarios (24 feature + 3 TDD), 4 Robot integration
tests, 4 ASV benchmark suites.

ISSUES CLOSED: #821
2026-03-25 17:49:06 +00:00

159 lines
4.7 KiB
Python

"""Robot Framework helper for context tier runtime integration tests.
Each command exercises the runtime logic added by issue #821:
auto-promotion, staleness enforcement, and budget enforcement.
Usage::
python helper_context_tier_runtime.py <command>
Commands:
auto-promote -- Verify auto-promotion on repeated access.
staleness -- Verify enforce_staleness() demotes stale fragments.
budget-evict -- Verify budget enforcement on store.
event-emission -- Verify tier transition events are emitted.
"""
from __future__ import annotations
import sys
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from pathlib import Path
# Ensure source tree is importable
_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from cleveragents.application.services.context_tiers import ( # noqa: E402
ContextTierService,
)
from cleveragents.domain.models.acms.tiers import ( # noqa: E402
ContextTier,
TierBudget,
TieredFragment,
)
from cleveragents.infrastructure.events.models import DomainEvent # noqa: E402
from cleveragents.infrastructure.events.types import EventType # noqa: E402
class _EventCollector:
"""Simple event bus replacement that collects emitted events."""
def __init__(self) -> None:
self.events: list[DomainEvent] = []
def emit(self, event: DomainEvent) -> None:
self.events.append(event)
def subscribe(
self,
event_type: EventType,
handler: Callable[[DomainEvent], None],
) -> None:
pass
def _make_frag(
fid: str,
tier: ContextTier,
tokens: int = 50,
last_accessed: datetime | None = None,
) -> TieredFragment:
frag = TieredFragment(
fragment_id=fid,
content="test",
tier=tier,
token_count=tokens,
project_name="test",
)
if last_accessed is not None:
frag.last_accessed = last_accessed
return frag
def cmd_auto_promote() -> None:
svc = ContextTierService(settings=None)
frag = _make_frag("frag-01", ContextTier.COLD)
svc.store(frag)
for _ in range(5):
svc.get("frag-01")
result = svc._find_fragment("frag-01")
assert result is not None, "Fragment disappeared"
assert result.tier == ContextTier.WARM, (
f"Expected WARM (not HOT — counter reset prevents chain promotion)"
f" after 5 accesses from cold, got {result.tier}"
)
print("context-tier-runtime-auto-promote-ok")
def cmd_staleness() -> None:
svc = ContextTierService(settings=None)
stale = datetime.now(tz=UTC) - timedelta(hours=25)
frag = _make_frag("stale-01", ContextTier.HOT, last_accessed=stale)
svc.store(frag)
demoted = svc.enforce_staleness()
assert "stale-01" in demoted, f"Expected stale-01 in demoted list: {demoted}"
result = svc._find_fragment("stale-01")
assert result is not None, "Fragment disappeared"
assert result.tier == ContextTier.WARM, (
f"Expected WARM after staleness demotion, got {result.tier}"
)
print("context-tier-runtime-staleness-ok")
def cmd_budget_evict() -> None:
svc = ContextTierService(settings=None)
svc._budget = TierBudget(max_tokens_hot=100)
old = datetime.now(tz=UTC) - timedelta(minutes=10)
svc.store(_make_frag("old-01", ContextTier.HOT, tokens=50, last_accessed=old))
svc.store(_make_frag("new-01", ContextTier.HOT, tokens=50))
svc.store(_make_frag("overflow-01", ContextTier.HOT, tokens=50))
total = sum(f.token_count for f in svc._hot.values())
assert total <= 100, f"Hot tier over budget: {total} > 100"
old_frag = svc._find_fragment("old-01")
assert old_frag is None, "old-01 should have been evicted"
print("context-tier-runtime-budget-evict-ok")
def cmd_event_emission() -> None:
collector = _EventCollector()
svc = ContextTierService(settings=None, event_bus=collector)
svc.store(_make_frag("evt-01", ContextTier.COLD))
svc.promote("evt-01")
promoted = [e for e in collector.events if e.event_type == EventType.TIER_PROMOTED]
assert len(promoted) >= 1, f"No TIER_PROMOTED events: {collector.events}"
assert promoted[0].details["fragment_id"] == "evt-01"
print("context-tier-runtime-event-emission-ok")
_COMMANDS: dict[str, Callable[[], None]] = {
"auto-promote": cmd_auto_promote,
"staleness": cmd_staleness,
"budget-evict": cmd_budget_evict,
"event-emission": cmd_event_emission,
}
def main() -> None:
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
valid = ", ".join(sorted(_COMMANDS.keys()))
print(f"Usage: {sys.argv[0]} <{valid}>", file=sys.stderr)
sys.exit(1)
cmd = _COMMANDS[sys.argv[1]]
cmd()
if __name__ == "__main__":
main()