Proposal: improve ca-uat-tester — stop creating separate issues for progress reports #2423

New issue

Open

opened 2026-04-03 18:00:07 +00:00 by freemo · 0 comments

freemo commented

2026-04-03 18:00:07 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: Prompt improvement / workflow fix
Affected Agent: ca-uat-tester
Evidence:

The UAT pool supervisor is creating a new Forgejo issue every ~10 minutes for progress reports, despite the agent definition explicitly stating:

"NEVER create new Forgejo issues for progress reports. In Pool Supervisor Mode, create ONE tracking issue at startup and post ALL progress updates as comments on that single issue."

Quantified impact: As of 2026-04-03 17:52 UTC, there are 48+ open [UAT-SUPERVISOR] Progress Report issues created today alone, spanning cycles 1051 through 1521. Each is a separate issue with titles like:

[UAT-SUPERVISOR] Progress Report — Cycle 1521 — v3.7.0 (#2422)
[UAT-SUPERVISOR] Progress Report — Cycle 1511 — v3.7.0 (#2421)
[UAT-SUPERVISOR] Progress Report — Cycle 1501 — v3.7.0 (#2398)
... (48+ more)

These are created approximately every 10 minutes (every 10 cycles), each as a brand new issue rather than a comment on a single tracking issue.

Root cause analysis: The agent definition already contains the correct instruction in the Pool Supervision Loop section (lines 105-107 and 214-234), but the instruction is not being followed. This suggests the instruction is either:

Not prominent enough in the prompt — it's buried in the pseudocode block
Competing with the progress report template which doesn't explicitly reference the tracking issue
The agent may be losing state (tracking issue number) between cycles and defaulting to creating new issues

Proposed Change

Strengthen the ca-uat-tester.md agent definition with the following modifications:

Add a prominent WARNING block at the very top of the Pool Supervision Loop section (before the pseudocode) that explicitly states the single-tracking-issue requirement in bold, with a "DO NOT" instruction.
Add a pre-check step in the progress report section: Before posting any progress report, the agent must first query Forgejo for its existing tracking issue (by searching for [CA-AUTO] UAT Pool Supervisor in the title). If found, post as a comment. If not found, create one. This makes the behavior resilient to state loss.
Rename the progress report title format to clearly distinguish progress comments from the tracking issue title. The tracking issue should use [CA-AUTO] UAT Pool Supervisor — <milestone> — Session Tracker and progress reports should be comments, never issues.
Add an explicit "NEVER" rule in the Important Rules section specifically about progress report issues.

Expected Impact

Eliminates 48+ spam issues per day from the issue tracker
Reduces noise for human observers monitoring the project
Reduces Forgejo API load (creating issues is heavier than posting comments)
Makes UAT progress easier to follow (all in one issue thread)

Risk Assessment

Low risk: The instruction already exists in the agent definition — this change only makes it more prominent and adds a resilience mechanism
Potential concern: If the agent is losing state due to restarts, the "search for existing tracking issue" fallback ensures correct behavior even after crashes
No behavioral change: The agent should already be doing this; we're just making the instruction harder to ignore

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: Prompt improvement / workflow fix **Affected Agent**: `ca-uat-tester` **Evidence**: The UAT pool supervisor is creating a **new Forgejo issue every ~10 minutes** for progress reports, despite the agent definition explicitly stating: > "NEVER create new Forgejo issues for progress reports. In Pool Supervisor Mode, create ONE tracking issue at startup and post ALL progress updates as comments on that single issue." **Quantified impact**: As of 2026-04-03 17:52 UTC, there are **48+ open `[UAT-SUPERVISOR] Progress Report` issues** created today alone, spanning cycles 1051 through 1521. Each is a separate issue with titles like: - `[UAT-SUPERVISOR] Progress Report — Cycle 1521 — v3.7.0` (#2422) - `[UAT-SUPERVISOR] Progress Report — Cycle 1511 — v3.7.0` (#2421) - `[UAT-SUPERVISOR] Progress Report — Cycle 1501 — v3.7.0` (#2398) - ... (48+ more) These are created approximately every 10 minutes (every 10 cycles), each as a brand new issue rather than a comment on a single tracking issue. **Root cause analysis**: The agent definition already contains the correct instruction in the Pool Supervision Loop section (lines 105-107 and 214-234), but the instruction is not being followed. This suggests the instruction is either: 1. Not prominent enough in the prompt — it's buried in the pseudocode block 2. Competing with the progress report template which doesn't explicitly reference the tracking issue 3. The agent may be losing state (tracking issue number) between cycles and defaulting to creating new issues ### Proposed Change Strengthen the `ca-uat-tester.md` agent definition with the following modifications: 1. **Add a prominent WARNING block** at the very top of the Pool Supervision Loop section (before the pseudocode) that explicitly states the single-tracking-issue requirement in bold, with a "DO NOT" instruction. 2. **Add a pre-check step** in the progress report section: Before posting any progress report, the agent must first query Forgejo for its existing tracking issue (by searching for `[CA-AUTO] UAT Pool Supervisor` in the title). If found, post as a comment. If not found, create one. This makes the behavior resilient to state loss. 3. **Rename the progress report title format** to clearly distinguish progress comments from the tracking issue title. The tracking issue should use `[CA-AUTO] UAT Pool Supervisor — <milestone> — Session Tracker` and progress reports should be comments, never issues. 4. **Add an explicit "NEVER" rule** in the Important Rules section specifically about progress report issues. ### Expected Impact - **Eliminates 48+ spam issues per day** from the issue tracker - Reduces noise for human observers monitoring the project - Reduces Forgejo API load (creating issues is heavier than posting comments) - Makes UAT progress easier to follow (all in one issue thread) ### Risk Assessment - **Low risk**: The instruction already exists in the agent definition — this change only makes it more prominent and adds a resilience mechanism - **Potential concern**: If the agent is losing state due to restarts, the "search for existing tracking issue" fallback ensures correct behavior even after crashes - **No behavioral change**: The agent should already be doing this; we're just making the instruction harder to ignore --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo added this to the v3.7.0 milestone

2026-04-03 18:00:45 +00:00

freemo referenced this issue

2026-04-03 18:23:13 +00:00

Task: Autonomous build progress report — v3.7.0 Production Push — 2026-04-03 #2436

freemo referenced this issue

2026-04-03 18:25:44 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.x — Session Tracker #2435

freemo referenced this issue

2026-04-03 18:37:27 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.x — Session Tracker #2435

freemo referenced this issue