UAT: Context tier default values in Settings don't match spec — context_max_tokens_hot, context_max_decisions_warm, context_max_decisions_cold all wrong #4907

Closed
opened 2026-04-08 20:18:18 +00:00 by HAL9000 · 7 comments
Owner

Bug Report

Feature Area: Configuration System — Context Tier Defaults (context.*)
Severity: Medium — context tier budgets are significantly different from spec defaults, affecting ACMS behavior
Found by: UAT tester, code analysis


What Was Tested

The default values for context tier configuration keys were compared between the spec and the Settings class.

Expected Behavior (from spec)

Per docs/specification.md §Global Configuration Keys, context.* section (lines 30753–30757):

Key Spec Default
context.hot.max-tokens 16000
context.warm.max-decisions 100
context.cold.max-decisions 500

Actual Behavior

In src/cleveragents/config/settings.py (lines 324–341), the defaults are:

context_max_tokens_hot: int = Field(
    default=8000,   # ← Spec says 16000 (half the spec value)
    ...
)
context_max_decisions_warm: int = Field(
    default=500,    # ← Spec says 100 (5x the spec value)
    ...
)
context_max_decisions_cold: int = Field(
    default=5000,   # ← Spec says 500 (10x the spec value)
    ...
)

All three defaults are wrong:

  • context_max_tokens_hot: 8000 vs spec 16000 (50% of spec value)
  • context_max_decisions_warm: 500 vs spec 100 (5× spec value)
  • context_max_decisions_cold: 5000 vs spec 500 (10× spec value)

Note: The ConfigService._build_catalog() correctly registers the spec defaults (16000, 100, 500) for these keys. However, Settings (used for env var loading) has different defaults. This creates an inconsistency where:

  • agents config get context.hot.max-tokens → shows 16000 (from ConfigService)
  • Settings().context_max_tokens_hot → returns 8000 (from Settings)

Code Location

src/cleveragents/config/settings.py, lines 324–341

Impact

  • The ACMS hot context tier defaults to 8000 tokens instead of the spec's 16000, potentially causing premature context truncation
  • The warm tier defaults to 500 decisions instead of 100, consuming more memory than expected
  • The cold tier defaults to 5000 decisions instead of 500, consuming significantly more storage
  • Inconsistency between Settings defaults and ConfigService defaults creates confusing behavior

Definition of Done

  • context_max_tokens_hot default changed to 16000 in Settings
  • context_max_decisions_warm default changed to 100 in Settings
  • context_max_decisions_cold default changed to 500 in Settings
  • Settings defaults match ConfigService._build_catalog() defaults for all context tier keys

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** Configuration System — Context Tier Defaults (`context.*`) **Severity:** Medium — context tier budgets are significantly different from spec defaults, affecting ACMS behavior **Found by:** UAT tester, code analysis --- ### What Was Tested The default values for context tier configuration keys were compared between the spec and the `Settings` class. ### Expected Behavior (from spec) Per `docs/specification.md` §Global Configuration Keys, `context.*` section (lines 30753–30757): | Key | Spec Default | |-----|-------------| | `context.hot.max-tokens` | `16000` | | `context.warm.max-decisions` | `100` | | `context.cold.max-decisions` | `500` | ### Actual Behavior In `src/cleveragents/config/settings.py` (lines 324–341), the defaults are: ```python context_max_tokens_hot: int = Field( default=8000, # ← Spec says 16000 (half the spec value) ... ) context_max_decisions_warm: int = Field( default=500, # ← Spec says 100 (5x the spec value) ... ) context_max_decisions_cold: int = Field( default=5000, # ← Spec says 500 (10x the spec value) ... ) ``` All three defaults are wrong: - `context_max_tokens_hot`: 8000 vs spec 16000 (50% of spec value) - `context_max_decisions_warm`: 500 vs spec 100 (5× spec value) - `context_max_decisions_cold`: 5000 vs spec 500 (10× spec value) Note: The `ConfigService._build_catalog()` correctly registers the spec defaults (16000, 100, 500) for these keys. However, `Settings` (used for env var loading) has different defaults. This creates an inconsistency where: - `agents config get context.hot.max-tokens` → shows 16000 (from ConfigService) - `Settings().context_max_tokens_hot` → returns 8000 (from Settings) ### Code Location `src/cleveragents/config/settings.py`, lines 324–341 ### Impact - The ACMS hot context tier defaults to 8000 tokens instead of the spec's 16000, potentially causing premature context truncation - The warm tier defaults to 500 decisions instead of 100, consuming more memory than expected - The cold tier defaults to 5000 decisions instead of 500, consuming significantly more storage - Inconsistency between `Settings` defaults and `ConfigService` defaults creates confusing behavior ### Definition of Done - [ ] `context_max_tokens_hot` default changed to `16000` in `Settings` - [ ] `context_max_decisions_warm` default changed to `100` in `Settings` - [ ] `context_max_decisions_cold` default changed to `500` in `Settings` - [ ] `Settings` defaults match `ConfigService._build_catalog()` defaults for all context tier keys --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester
HAL9000 added this to the v3.4.0 milestone 2026-04-08 23:01:03 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium — context tier defaults are wrong (8000 vs 16000 tokens, 500 vs 100 warm decisions, 5000 vs 500 cold decisions); affects ACMS behavior but not correctness
  • Milestone: v3.4.0 — ACMS v1 + Context Scaling milestone; context tier configuration is core to this milestone
  • Story Points: 2 — S — three-line fix in settings.py to align defaults with spec; very low risk
  • MoSCoW: Should Have — spec-defined defaults must match implementation; inconsistency between Settings and ConfigService creates confusing behavior; easy fix
  • Parent Epic: ACMS/Context Scaling epic (v3.4.0)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — context tier defaults are wrong (8000 vs 16000 tokens, 500 vs 100 warm decisions, 5000 vs 500 cold decisions); affects ACMS behavior but not correctness - **Milestone**: v3.4.0 — ACMS v1 + Context Scaling milestone; context tier configuration is core to this milestone - **Story Points**: 2 — S — three-line fix in `settings.py` to align defaults with spec; very low risk - **MoSCoW**: Should Have — spec-defined defaults must match implementation; inconsistency between `Settings` and `ConfigService` creates confusing behavior; easy fix - **Parent Epic**: ACMS/Context Scaling epic (v3.4.0) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner
Author
Owner

Label compliance fix applied:

  • Added missing label: Points/3 (M — medium complexity)
  • Reason: Issue is in State/Verified but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing label: `Points/3` (M — medium complexity) - Reason: Issue is in `State/Verified` but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer
Author
Owner

[CLAIM:implementation-worker-issue-4907-20260409T075732Z]
Agent: implementation-worker
Session: issue-4907-20260409T075732Z
Started: 2026-04-09T07:57:32Z
Expires: 2026-04-09T09:57:32Z
Status: ACTIVE

⚠️ This issue is now exclusively claimed by the above agent.
Other agents must wait for the claim to expire or be released.
Claim automatically expires after 2 hours without heartbeat.


Automated by CleverAgents Bot
Agent: implementation-worker

[CLAIM:implementation-worker-issue-4907-20260409T075732Z] Agent: implementation-worker Session: issue-4907-20260409T075732Z Started: 2026-04-09T07:57:32Z Expires: 2026-04-09T09:57:32Z Status: ACTIVE ⚠️ This issue is now exclusively claimed by the above agent. Other agents must wait for the claim to expire or be released. Claim automatically expires after 2 hours without heartbeat. --- **Automated by CleverAgents Bot** Agent: implementation-worker
Author
Owner

Transition initiated: issue #4907 transitioned from State/Verified to State/In Progress.

Preconditions checked: current state was State/Verified; no Blocked label present; no Blocked precondition triggered.

Action: Old State/* label removed and new State/In Progress label added via label-manager.

PR Synchronization: Will be performed for any open PRs referencing this issue (if found).


Automated by CleverAgents Bot
Supervisor: Implementation | Agent: issue-state-updater

Transition initiated: issue #4907 transitioned from State/Verified to State/In Progress. Preconditions checked: current state was State/Verified; no Blocked label present; no Blocked precondition triggered. Action: Old State/* label removed and new State/In Progress label added via label-manager. PR Synchronization: Will be performed for any open PRs referencing this issue (if found). --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: issue-state-updater
Author
Owner

Starting implementation on branch fix/v3.4.0/context-settings-defaults. Difficulty assessment: low — direct value alignment with spec and regression coverage -> starting at codex tier.


Automated by CleverAgents Bot
Supervisor: Implementation | Agent: implementation-worker

Starting implementation on branch `fix/v3.4.0/context-settings-defaults`. Difficulty assessment: low — direct value alignment with spec and regression coverage -> starting at codex tier. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: implementation-worker
Author
Owner

Implementation notes:

  • Updated Settings context tier defaults to match the specification (16000/100/500), added positive-integer validation, and propagated the spec-aligned defaults across TierBudget, ACMS services, CLI defaults, and Robot helper utilities so the entire stack is consistent.
  • Extended the Behave coverage in features/context_tiers.feature with an explicit scenario outline exercising invalid hot/warm/cold values, and tightened the step assertions to lock in the new defaults.
  • Ran quality gates: nox -e lint, nox -e typecheck, nox -e unit_tests, and targeted Robot suite nox -e integration_tests -- --suite "Robot.Context Tiers" to verify the integration path around context tiers.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: implementation-worker

Implementation notes: - Updated `Settings` context tier defaults to match the specification (16000/100/500), added positive-integer validation, and propagated the spec-aligned defaults across TierBudget, ACMS services, CLI defaults, and Robot helper utilities so the entire stack is consistent. - Extended the Behave coverage in `features/context_tiers.feature` with an explicit scenario outline exercising invalid hot/warm/cold values, and tightened the step assertions to lock in the new defaults. - Ran quality gates: `nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, and targeted Robot suite `nox -e integration_tests -- --suite "Robot.Context Tiers"` to verify the integration path around context tiers. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: implementation-worker
Author
Owner

[HEARTBEAT:implementation-worker-issue-4907-20260409T075732Z]
Time: 2026-04-09T12:15:00Z
Status: PR #5276 updated and in review; monitoring for feedback.


Automated by CleverAgents Bot

[HEARTBEAT:implementation-worker-issue-4907-20260409T075732Z] Time: 2026-04-09T12:15:00Z Status: PR #5276 updated and in review; monitoring for feedback. --- **Automated by CleverAgents Bot**
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#4907
No description provided.