cleveragents/cleveragents-core

Fork 3

Proposal: fix ca-test-infra-improver — reduce health comment spam on session state issue #3385

New issue

Closed

opened 2026-04-05 16:20:45 +00:00 by freemo · 1 comment

freemo commented

2026-04-05 16:20:45 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: workflow_fix
Affected Agent: ca-test-infra-improver
Evidence:

During the Session 3 build session (issue #3377, started 2026-04-05), the ca-test-infra-improver pool supervisor posted 44 out of 52 total comments on the session state issue in approximately 20 minutes (15:05–15:20 UTC). Specific evidence:

Iteration 1–9 (15:05–15:09): Posted 9 health comments, one every ~17 seconds, all showing "0/8 areas analyzed"
Iteration 10–11 (15:09–15:10): Progress changed to "2/8 areas analyzed" — only 2 comments were informative
Iteration 12–44 (15:10–15:20): Posted 33 consecutive identical health comments, all showing "2/8 areas analyzed, 5/8 active workers, 0 issues filed" with zero state change

Root cause: The agent definition (line 237) specifies if cycle % 60 == 0 for health posting, meaning health comments should only appear every 60th monitoring cycle (~10 minutes). However, the Gemini 2.5 Pro model running the agent ignores this modulo guard and posts on every single monitoring iteration.

The pseudocode structure is ambiguous — the cycle variable is incremented at the outer loop level (line 159), but the health posting check appears to be inside the inner while active: monitoring loop (lines 214–248). The model interprets the monitoring iterations as the cycle count and posts on each one.

Impact:

44 of 52 comments (85%) on the session state issue are from this one agent
Other agents' status updates are drowned out and nearly impossible to find
The session state issue becomes unusable for monitoring overall system health
Forgejo notification spam for anyone watching the issue

Proposed Change

Modify the ca-test-infra-improver.md agent definition to make the health posting constraint unambiguous and impossible to misinterpret:

Move the health posting logic outside the inner monitoring loop — Make it structurally clear that health comments belong to the outer supervision cycle, not the inner polling loop.
Add an explicit "DO NOT" constraint at the top of the Pool Supervision Loop section:

CRITICAL: Do NOT post health comments on every monitoring iteration. Health comments MUST be posted at most once every 10 minutes. Use a separate last_health_post_time timestamp to enforce this. If less than 10 minutes have elapsed since the last health comment, do NOT post.
Add a state-change-only posting rule: Health comments should only be posted when there is a meaningful state change (worker completed, new worker dispatched, all areas analyzed) OR when the 10-minute timer expires — whichever comes first.
Replace the cycle % 60 guard with an explicit timestamp check — The modulo approach is fragile because it depends on the model correctly tracking which loop level the cycle counter belongs to. A timestamp-based approach is unambiguous:
```
last_health_time = 0
...
if current_time - last_health_time >= 600:  # 10 minutes
    post health comment
    last_health_time = current_time
```

Expected Impact

Reduces health comment volume from ~44 per 20 minutes to ~2 per 20 minutes (one every 10 minutes)
Session state issue becomes usable for monitoring all agents
Other agents' status updates become visible
Reduced Forgejo notification spam

Risk Assessment

Very low risk: This change only affects the frequency of status reporting, not the agent's analysis or issue-filing behavior.
Potential concern: If the 10-minute interval is too long, important state changes might be delayed. However, the state-change-only posting rule mitigates this — significant events (worker completion, all areas done) will still be posted immediately.

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: workflow_fix **Affected Agent**: ca-test-infra-improver **Evidence**: During the Session 3 build session (issue #3377, started 2026-04-05), the `ca-test-infra-improver` pool supervisor posted **44 out of 52 total comments** on the session state issue in approximately 20 minutes (15:05–15:20 UTC). Specific evidence: 1. **Iteration 1–9** (15:05–15:09): Posted 9 health comments, one every ~17 seconds, all showing "0/8 areas analyzed" 2. **Iteration 10–11** (15:09–15:10): Progress changed to "2/8 areas analyzed" — only 2 comments were informative 3. **Iteration 12–44** (15:10–15:20): Posted **33 consecutive identical health comments**, all showing "2/8 areas analyzed, 5/8 active workers, 0 issues filed" with zero state change **Root cause**: The agent definition (line 237) specifies `if cycle % 60 == 0` for health posting, meaning health comments should only appear every 60th monitoring cycle (~10 minutes). However, the Gemini 2.5 Pro model running the agent **ignores this modulo guard** and posts on every single monitoring iteration. The pseudocode structure is ambiguous — the `cycle` variable is incremented at the outer loop level (line 159), but the health posting check appears to be inside the inner `while active:` monitoring loop (lines 214–248). The model interprets the monitoring iterations as the cycle count and posts on each one. **Impact**: - 44 of 52 comments (85%) on the session state issue are from this one agent - Other agents' status updates are drowned out and nearly impossible to find - The session state issue becomes unusable for monitoring overall system health - Forgejo notification spam for anyone watching the issue ### Proposed Change Modify the `ca-test-infra-improver.md` agent definition to make the health posting constraint unambiguous and impossible to misinterpret: 1. **Move the health posting logic outside the inner monitoring loop** — Make it structurally clear that health comments belong to the outer supervision cycle, not the inner polling loop. 2. **Add an explicit "DO NOT" constraint** at the top of the Pool Supervision Loop section: > **CRITICAL: Do NOT post health comments on every monitoring iteration. Health comments MUST be posted at most once every 10 minutes. Use a separate `last_health_post_time` timestamp to enforce this. If less than 10 minutes have elapsed since the last health comment, do NOT post.** 3. **Add a state-change-only posting rule**: Health comments should only be posted when there is a meaningful state change (worker completed, new worker dispatched, all areas analyzed) OR when the 10-minute timer expires — whichever comes first. 4. **Replace the `cycle % 60` guard with an explicit timestamp check** — The modulo approach is fragile because it depends on the model correctly tracking which loop level the cycle counter belongs to. A timestamp-based approach is unambiguous: ``` last_health_time = 0 ... if current_time - last_health_time >= 600: # 10 minutes post health comment last_health_time = current_time ``` ### Expected Impact - Reduces health comment volume from ~44 per 20 minutes to ~2 per 20 minutes (one every 10 minutes) - Session state issue becomes usable for monitoring all agents - Other agents' status updates become visible - Reduced Forgejo notification spam ### Risk Assessment - **Very low risk**: This change only affects the frequency of status reporting, not the agent's analysis or issue-filing behavior. - **Potential concern**: If the 10-minute interval is too long, important state changes might be delayed. However, the state-change-only posting rule mitigates this — significant events (worker completion, all areas done) will still be posted immediately. --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo referenced this issue

2026-04-05 16:21:58 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo added this to the v3.7.0 milestone

2026-04-05 17:19:37 +00:00

freemo referenced this issue

2026-04-05 17:19:48 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue from a commit

2026-04-05 18:30:05 +00:00

chore(agents): fix ca-test-infra-improver health comment spam

~~freemo referenced this issue 2026-04-05 18:30:23 +00:00~~

chore(agents): fix ca-test-infra-improver health comment spam #3478

freemo referenced this issue

2026-04-05 18:32:43 +00:00

Proposal: improve issue-implementor — add crash-safe health posting and graceful exit signaling #3483

freemo referenced this issue

2026-04-05 18:33:22 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue from a commit

2026-04-05 19:32:37 +00:00

chore(agents): fix ca-test-infra-improver health comment spam

~~freemo referenced this issue 2026-04-05 19:32:52 +00:00~~

chore(agents): fix ca-test-infra-improver — reduce health comment spam on session state issue #3551

freemo referenced this issue

2026-04-05 19:36:07 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo commented

2026-04-05 20:05:13 +00:00

Author

Owner

State label reconciliation:

Previous state: State/In Progress + State/Verified (conflicting — two State/* labels)
Corrected to: State/In Review
Reason: Issue had two conflicting State/* labels. PR #3478 implements this proposal, so State/In Review is the correct current state.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

State label reconciliation: - Previous state: `State/In Progress` + `State/Verified` (conflicting — two State/* labels) - Corrected to: `State/In Review` - Reason: Issue had two conflicting State/* labels. PR #3478 implements this proposal, so `State/In Review` is the correct current state. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

freemo referenced this issue

2026-04-05 20:06:19 +00:00

chore(agents): fix ca-test-infra-improver — reduce health comment spam on session state issue #3551

freemo referenced this issue

2026-04-05 20:07:47 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:13:46 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo removed this from the v3.7.0 milestone

2026-04-06 23:52:32 +00:00