cleveragents/cleveragents-core

Fork 3

Proposal: improve ca-uat-tester — stop creating separate issues for every progress report #2369

New issue

Closed

opened 2026-04-03 17:23:39 +00:00 by freemo · 5 comments

freemo commented

2026-04-03 17:23:39 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: Prompt compliance failure / issue tracker pollution
Affected Agent: ca-uat-tester (Pool Supervisor Mode)
Evidence:

Despite the agent definition containing explicit instructions to post progress reports as comments on a single tracking issue, the UAT supervisor is creating a new Forgejo issue for every progress report cycle. This has resulted in approximately 180 spam issues polluting the issue tracker.

Specific evidence:

Issues #2182 through #2359 are almost all [UAT-SUPERVISOR] Progress Report — Cycle NNNN — v3.7.0 issues, each created as a separate issue.
Issue #2360 ([CA-AUTO] UAT Pool Supervisor — v3.6.0 — Session Tracker) is the tracking issue that was correctly created, but it has 0 comments — progress reports are not being posted there.
The agent definition already contains:
- Line 105-107: "Progress reports MUST be posted as comments on a single tracking issue — NEVER as separate new issues."
- Lines 462-465 (Important Rules): "NEVER create new Forgejo issues for progress reports."
- Lines 214-234: The pseudocode shows post comment on issue #TRACKING_ISSUE_NUMBER

The instructions are correct but the model is not following them. The progress report section uses forgejo_create_issue instead of forgejo_create_issue_comment.

Proposed Change

Strengthen the prompt to make the distinction between "create issue" and "post comment" unmistakable:

Add a prominent WARNING box at the very top of the Pool Supervision Loop, before any other instructions:

⚠️ CRITICAL: Progress reports are COMMENTS, not issues.
Use forgejo_create_issue_comment(issue=TRACKING_ISSUE_NUMBER, ...)
NEVER use forgejo_create_issue() for progress reports.

Add a concrete negative example showing exactly what NOT to do:

❌ WRONG: forgejo_create_issue(title="[UAT-SUPERVISOR] Progress Report...")
✅ RIGHT: forgejo_create_issue_comment(index=TRACKING_ISSUE_NUMBER, body="## Progress Report...")

Add the specific MCP tool name in the pseudocode (line 218) — instead of the abstract post comment on issue #TRACKING_ISSUE_NUMBER, write the exact tool call: forgejo_create_issue_comment(owner, repo, TRACKING_ISSUE_NUMBER, body).
Add a self-check step after each progress report: "Verify you used forgejo_create_issue_comment, NOT forgejo_create_issue. If you accidentally created a new issue, close it immediately."

Expected Impact

No more spam progress report issues in the tracker.
Progress reports will be consolidated as comments on the single tracking issue.
The issue tracker will be significantly cleaner (~180 fewer noise issues per session).

Risk Assessment

Very low risk: This is a prompt clarification, not a behavioral change. The intended behavior is already documented — we're just making it harder for the model to misinterpret.
No functional change: The progress reporting content stays the same; only the delivery mechanism is reinforced.

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: Prompt compliance failure / issue tracker pollution **Affected Agent**: `ca-uat-tester` (Pool Supervisor Mode) **Evidence**: Despite the agent definition containing explicit instructions to post progress reports as **comments on a single tracking issue**, the UAT supervisor is creating a **new Forgejo issue for every progress report cycle**. This has resulted in approximately **180 spam issues** polluting the issue tracker. Specific evidence: - Issues #2182 through #2359 are almost all `[UAT-SUPERVISOR] Progress Report — Cycle NNNN — v3.7.0` issues, each created as a **separate issue**. - Issue #2360 (`[CA-AUTO] UAT Pool Supervisor — v3.6.0 — Session Tracker`) is the tracking issue that was correctly created, but it has **0 comments** — progress reports are not being posted there. - The agent definition already contains: - Line 105-107: "Progress reports MUST be posted as comments on a single tracking issue — NEVER as separate new issues." - Lines 462-465 (Important Rules): "NEVER create new Forgejo issues for progress reports." - Lines 214-234: The pseudocode shows `post comment on issue #TRACKING_ISSUE_NUMBER` The instructions are correct but the model is not following them. The progress report section uses `forgejo_create_issue` instead of `forgejo_create_issue_comment`. ### Proposed Change Strengthen the prompt to make the distinction between "create issue" and "post comment" unmistakable: 1. **Add a prominent WARNING box** at the very top of the Pool Supervision Loop, before any other instructions: ``` ⚠️ CRITICAL: Progress reports are COMMENTS, not issues. Use forgejo_create_issue_comment(issue=TRACKING_ISSUE_NUMBER, ...) NEVER use forgejo_create_issue() for progress reports. ``` 2. **Add a concrete negative example** showing exactly what NOT to do: ``` ❌ WRONG: forgejo_create_issue(title="[UAT-SUPERVISOR] Progress Report...") ✅ RIGHT: forgejo_create_issue_comment(index=TRACKING_ISSUE_NUMBER, body="## Progress Report...") ``` 3. **Add the specific MCP tool name** in the pseudocode (line 218) — instead of the abstract `post comment on issue #TRACKING_ISSUE_NUMBER`, write the exact tool call: `forgejo_create_issue_comment(owner, repo, TRACKING_ISSUE_NUMBER, body)`. 4. **Add a self-check step** after each progress report: "Verify you used `forgejo_create_issue_comment`, NOT `forgejo_create_issue`. If you accidentally created a new issue, close it immediately." ### Expected Impact - No more spam progress report issues in the tracker. - Progress reports will be consolidated as comments on the single tracking issue. - The issue tracker will be significantly cleaner (~180 fewer noise issues per session). ### Risk Assessment - **Very low risk**: This is a prompt clarification, not a behavioral change. The intended behavior is already documented — we're just making it harder for the model to misinterpret. - **No functional change**: The progress reporting content stays the same; only the delivery mechanism is reinforced. --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo added this to the v3.7.0 milestone

2026-04-03 17:24:49 +00:00

freemo referenced this issue

2026-04-03 17:25:18 +00:00

[CA-AUTO] Agent Evolver — Session State #2374

freemo referenced this issue

2026-04-03 17:34:44 +00:00

[CA-GROOMER] Backlog Groomer — Session State (groomer-1) #2386

freemo commented

2026-04-03 18:40:35 +00:00

Author

Owner

approved

freemo referenced this issue from a commit

2026-04-03 18:42:13 +00:00

chore(agents): improve ca-uat-tester — enforce comment-based progress reports

freemo referenced this issue from a pull request that will close it,

2026-04-03 18:42:23 +00:00

chore(agents): improve ca-uat-tester — enforce comment-based progress reports #2512

freemo referenced this issue

2026-04-03 18:50:15 +00:00

[Automated] Product Build Session State — v3.7.0 Production Sprint #2006

freemo commented

2026-04-03 20:40:38 +00:00

Author

Owner

PR #2512 reviewed, approved, and merged.

The agent prompt improvements have been applied to .opencode/agents/ca-uat-tester.md:

Added prominent WARNING blockquote with exact tool names and wrong/right examples
Replaced abstract pseudocode with exact MCP tool call (forgejo_create_issue_comment)
Added self-check step after each progress report
Strengthened Important Rules section with explicit tool references

This should eliminate the ~180 spam progress report issues per session.

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

PR #2512 reviewed, approved, and merged. The agent prompt improvements have been applied to `.opencode/agents/ca-uat-tester.md`: - Added prominent WARNING blockquote with exact tool names and wrong/right examples - Replaced abstract pseudocode with exact MCP tool call (`forgejo_create_issue_comment`) - Added self-check step after each progress report - Strengthened Important Rules section with explicit tool references This should eliminate the ~180 spam progress report issues per session. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer

freemo

2026-04-03 20:41:03 +00:00

closed this issue
added the
State
Completed
label

freemo commented

2026-04-03 20:41:13 +00:00

Author

Owner

Issue transitioned to State/Completed and closed. All work for this proposal has been merged.

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

Issue transitioned to `State/Completed` and closed. All work for this proposal has been merged. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer

freemo commented

2026-04-04 19:16:09 +00:00

Author

Owner

PR #2512 reviewed, approved, and merged.

The prompt improvements to ca-uat-tester.md are now on master. The WARNING blockquote, concrete tool call examples, self-check step, and strengthened Important Rules section should prevent the model from creating separate issues for progress reports going forward.

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

PR #2512 reviewed, approved, and merged. The prompt improvements to `ca-uat-tester.md` are now on master. The WARNING blockquote, concrete tool call examples, self-check step, and strengthened Important Rules section should prevent the model from creating separate issues for progress reports going forward. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer

freemo referenced this issue from a commit

2026-04-05 02:39:06 +00:00

chore(agents): improve ca-uat-tester — enforce comment-based progress reports

freemo commented

2026-04-05 06:32:45 +00:00

Author

Owner

PR #2512 reviewed, approved, and merged.

All CI checks passed. The prompt improvements to ca-uat-tester.md are now live on master.

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

PR #2512 reviewed, approved, and merged. All CI checks passed. The prompt improvements to `ca-uat-tester.md` are now live on master. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer