cleveragents/cleveragents-core

Fork 3

Proposal: improve ca-test-infra-improver — prevent massive duplicate issue creation #1802

New issue

Closed

opened 2026-04-02 23:52:50 +00:00 by freemo · 3 comments

freemo commented

2026-04-02 23:52:50 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: Prompt improvement — duplicate issue explosion
Affected Agent: ca-test-infra-improver (Worker Mode)
Evidence: During the v3.7.0 session and preceding sessions, the test infrastructure improver has created 48+ TEST-INFRA issues, with massive duplication across focus areas. Specific duplicate clusters:

Topic	Duplicate Issues	Count
Dependency caching	#1589, #1639, #1652, #1654, #1662, #1667	6
Matrix builds / Python versions	#1646, #1657, #1665, #1672, #1676, #1712, #1731	7
Parallelize CI jobs	#1650, #1679, #1703, #1730, #1741	5
Redundant setup elimination	#1604, #1658, #1701	3
Unable to clone repository	#1673, #1686, #1691, #1694, #1699, #1713, #1732	7

Root Cause: The agent's "Duplicate Avoidance" section is too vague. It says:

"Before filing any issue: 1. Search Forgejo for existing issues with 'TEST-INFRA:' prefix. 2. Check for similar titles/descriptions. 3. If potential duplicate found, skip."

This guidance is insufficient because:

The agent doesn't perform rigorous keyword matching — it files issues with slightly different wording about the same topic
Workers dispatched in parallel don't coordinate — they each independently analyze and file, creating duplicates across sessions
The agent doesn't list what existing issues it found during dedup, making it impossible to verify dedup was attempted

Proposed Change

Modify the Duplicate Avoidance section and Issue Filing section in ca-test-infra-improver.md to:

Mandatory keyword search before filing — Before creating any issue, the agent MUST search Forgejo for ALL open issues containing the key nouns from the proposed title (e.g., if proposing "Implement dependency caching", search for "dependency caching", "cache", "caching"). If ANY issue with overlapping keywords exists, the agent MUST NOT file a new issue.
Dedup proof in issue body — Every filed issue MUST include a "Duplicate Check" section listing the search queries used and the existing issues found (or "none found"). This makes dedup auditable.
Broader search scope — Search must include BOTH open AND closed issues, since closed duplicates indicate the topic has already been addressed.
Cross-area dedup — Before filing, also search without the area prefix (e.g., search for "dependency caching" not just "TEST-INFRA: [ci-pipeline-design] dependency caching") to catch issues filed under different areas.

Expected Impact

Reduces TEST-INFRA issue volume by ~70% (eliminating duplicates)
Reduces noise in the issue tracker, making it easier for humans and other agents to find real work
Reduces wasted human review time on duplicate issues
Makes dedup auditable through the "Duplicate Check" section

Risk Assessment

Low risk: These changes only add guardrails to issue filing. No analysis logic is modified.
Potential concern: Overly aggressive keyword matching might prevent filing of genuinely distinct issues. Mitigation: the agent should use 2+ keyword overlap as the threshold, not single-word matches.

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: Prompt improvement — duplicate issue explosion **Affected Agent**: `ca-test-infra-improver` (Worker Mode) **Evidence**: During the v3.7.0 session and preceding sessions, the test infrastructure improver has created **48+ TEST-INFRA issues**, with massive duplication across focus areas. Specific duplicate clusters: | Topic | Duplicate Issues | Count | |---|---|---| | Dependency caching | #1589, #1639, #1652, #1654, #1662, #1667 | 6 | | Matrix builds / Python versions | #1646, #1657, #1665, #1672, #1676, #1712, #1731 | 7 | | Parallelize CI jobs | #1650, #1679, #1703, #1730, #1741 | 5 | | Redundant setup elimination | #1604, #1658, #1701 | 3 | | Unable to clone repository | #1673, #1686, #1691, #1694, #1699, #1713, #1732 | 7 | **Root Cause**: The agent's "Duplicate Avoidance" section is too vague. It says: > "Before filing any issue: 1. Search Forgejo for existing issues with 'TEST-INFRA:' prefix. 2. Check for similar titles/descriptions. 3. If potential duplicate found, skip." This guidance is insufficient because: 1. The agent doesn't perform rigorous keyword matching — it files issues with slightly different wording about the same topic 2. Workers dispatched in parallel don't coordinate — they each independently analyze and file, creating duplicates across sessions 3. The agent doesn't list what existing issues it found during dedup, making it impossible to verify dedup was attempted ### Proposed Change Modify the **Duplicate Avoidance** section and **Issue Filing** section in `ca-test-infra-improver.md` to: 1. **Mandatory keyword search before filing** — Before creating any issue, the agent MUST search Forgejo for ALL open issues containing the key nouns from the proposed title (e.g., if proposing "Implement dependency caching", search for "dependency caching", "cache", "caching"). If ANY issue with overlapping keywords exists, the agent MUST NOT file a new issue. 2. **Dedup proof in issue body** — Every filed issue MUST include a "Duplicate Check" section listing the search queries used and the existing issues found (or "none found"). This makes dedup auditable. 3. **Broader search scope** — Search must include BOTH open AND closed issues, since closed duplicates indicate the topic has already been addressed. 4. **Cross-area dedup** — Before filing, also search without the area prefix (e.g., search for "dependency caching" not just "TEST-INFRA: [ci-pipeline-design] dependency caching") to catch issues filed under different areas. ### Expected Impact - Reduces TEST-INFRA issue volume by ~70% (eliminating duplicates) - Reduces noise in the issue tracker, making it easier for humans and other agents to find real work - Reduces wasted human review time on duplicate issues - Makes dedup auditable through the "Duplicate Check" section ### Risk Assessment - **Low risk**: These changes only add guardrails to issue filing. No analysis logic is modified. - **Potential concern**: Overly aggressive keyword matching might prevent filing of genuinely distinct issues. Mitigation: the agent should use 2+ keyword overlap as the threshold, not single-word matches. --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo referenced this issue

2026-04-02 23:55:39 +00:00

[Automated] Product Build Session State #1576

freemo referenced this issue

2026-04-03 00:28:44 +00:00

[Automated] Product Build Session State #1576

freemo commented

2026-04-03 18:35:04 +00:00

Author

Owner

I dont like the idea of having a hard cap of 3 issues per focus area. Otherwise this ticket looks good. Once you update that in the description leave a comment for me and tag me and I will approve this ticket at that time (and thus remove "needs feedback" so you can proceed).

freemo commented

2026-04-03 18:43:03 +00:00

Author

Owner

@freemo Updated the proposal per your feedback — removed the hard cap of 3 issues per focus area. The proposal now focuses on:

Mandatory keyword search before filing
Dedup proof in issue body (auditable)
Broader search scope (open + closed issues)
Cross-area dedup (search without area prefix)

No artificial caps. Ready for your review when convenient.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

@freemo Updated the proposal per your feedback — removed the hard cap of 3 issues per focus area. The proposal now focuses on: 1. Mandatory keyword search before filing 2. Dedup proof in issue body (auditable) 3. Broader search scope (open + closed issues) 4. Cross-area dedup (search without area prefix) No artificial caps. Ready for your review when convenient. --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo referenced this issue

2026-04-03 18:50:15 +00:00

[Automated] Product Build Session State — v3.7.0 Production Sprint #2006

freemo commented

2026-04-03 19:42:52 +00:00

Author

Owner

Approved, continue.

freemo referenced this issue from a commit

2026-04-05 06:49:21 +00:00

chore(agents): improve ca-test-infra-improver — prevent massive duplicate issue creation

freemo referenced this issue from a pull request that will close it,