cleveragents/cleveragents-core

Fork 3

Proposal: improve continuous-pr-reviewer — add recently-reviewed SHA tracking to prevent re-dispatching reviewers to already-reviewed PRs #4957

New issue

Open

opened 2026-04-08 23:40:11 +00:00 by freemo · 2 comments

freemo commented

2026-04-08 23:40:11 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: workflow_fix
Affected Agent: continuous-pr-reviewer
Evidence:

PR #3466 accumulated 46 reviews (45 COMMENT-type from the same bot) because the pool supervisor kept re-dispatching pr-self-reviewer instances to the same PR every 30 seconds.
The continuous-pr-reviewer loop checks recently_reviewed dict to avoid re-dispatching, but this dict is in-memory only — it resets every time the supervisor restarts or a new session is created.
The supervisor also dispatches to PRs that have REQUEST_CHANGES reviews but no new commits, causing reviewers to post the same feedback repeatedly.
The active_reviews dict tracks in-flight sessions but doesn't persist across restarts, so after any restart, all PRs appear unreviewed.

Proposed Change

Enhance the continuous-pr-reviewer's deduplication logic with persistent SHA-based tracking:

Persist review state to Forgejo: After dispatching a reviewer, post a lightweight tracking comment on the PR (or update a tracking issue) recording the HEAD SHA that was reviewed. On the next cycle, check this persisted state before dispatching.
Check actual review recency via API: Before dispatching a reviewer, query the Forgejo reviews API for the PR and check if there's already a review from the bot account posted after the current HEAD SHA was pushed. If yes, skip dispatch.
Increase the "stale review" threshold: The current threshold is 24 hours. Given that PRs can take days to get fixes, increase this to 48 hours to reduce unnecessary re-reviews.
Add a "max reviews per PR" guard: If a PR already has more than 5 reviews from the bot account, stop dispatching new reviewers until the PR gets new commits. This is a safety backstop.

Expected Impact

Eliminates the root cause of review spam (supervisor keeps re-dispatching to same PR)
Reduces wasted compute from redundant reviewer sessions
Makes the review system more predictable and less noisy
Complements the pr-self-reviewer deduplication guard (defense in depth)

Risk Assessment

Low risk: The change only affects dispatch decisions, not review quality.
Potential downside: A PR might wait longer for a re-review if the threshold is too high. Mitigation: still allow re-review when HEAD SHA changes.
Implementation complexity: Persisting state to Forgejo adds API calls. Mitigation: use a simple comment-based approach that's already supported.

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: workflow_fix **Affected Agent**: `continuous-pr-reviewer` **Evidence**: - PR #3466 accumulated 46 reviews (45 COMMENT-type from the same bot) because the pool supervisor kept re-dispatching `pr-self-reviewer` instances to the same PR every 30 seconds. - The `continuous-pr-reviewer` loop checks `recently_reviewed` dict to avoid re-dispatching, but this dict is **in-memory only** — it resets every time the supervisor restarts or a new session is created. - The supervisor also dispatches to PRs that have `REQUEST_CHANGES` reviews but no new commits, causing reviewers to post the same feedback repeatedly. - The `active_reviews` dict tracks in-flight sessions but doesn't persist across restarts, so after any restart, all PRs appear unreviewed. ### Proposed Change Enhance the `continuous-pr-reviewer`'s deduplication logic with **persistent SHA-based tracking**: 1. **Persist review state to Forgejo**: After dispatching a reviewer, post a lightweight tracking comment on the PR (or update a tracking issue) recording the HEAD SHA that was reviewed. On the next cycle, check this persisted state before dispatching. 2. **Check actual review recency via API**: Before dispatching a reviewer, query the Forgejo reviews API for the PR and check if there's already a review from the bot account posted after the current HEAD SHA was pushed. If yes, skip dispatch. 3. **Increase the "stale review" threshold**: The current threshold is 24 hours. Given that PRs can take days to get fixes, increase this to 48 hours to reduce unnecessary re-reviews. 4. **Add a "max reviews per PR" guard**: If a PR already has more than 5 reviews from the bot account, stop dispatching new reviewers until the PR gets new commits. This is a safety backstop. ### Expected Impact - Eliminates the root cause of review spam (supervisor keeps re-dispatching to same PR) - Reduces wasted compute from redundant reviewer sessions - Makes the review system more predictable and less noisy - Complements the `pr-self-reviewer` deduplication guard (defense in depth) ### Risk Assessment - **Low risk**: The change only affects dispatch decisions, not review quality. - **Potential downside**: A PR might wait longer for a re-review if the threshold is too high. Mitigation: still allow re-review when HEAD SHA changes. - **Implementation complexity**: Persisting state to Forgejo adds API calls. Mitigation: use a simple comment-based approach that's already supported. --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: agent-evolver

freemo added the

labels

2026-04-08 23:40:14 +00:00

freemo referenced this issue

2026-04-08 23:40:45 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4940

freemo commented

2026-04-08 23:42:39 +00:00

Author

Owner

This issue has the "Needs Feedback" label and is a proposal awaiting human review. I will not modify its state — a human must approve or reject it.

Summary of the proposal:
The agent-evolver has identified that continuous-pr-reviewer keeps re-dispatching reviewers to already-reviewed PRs because the recently_reviewed dict is in-memory only and resets on restart. PR #3466 accumulated 46 reviews (45 COMMENT-type from the same bot) as a result.

Proposed fix: Add persistent SHA-based tracking to continuous-pr-reviewer so it doesn't re-dispatch reviewers to PRs that have already been reviewed at the current HEAD commit.

To approve: Remove the "Needs Feedback" label and add "State/Verified", or comment with explicit approval.

Automated by CleverAgents Bot
Supervisor: Human Liaison | Agent: human-liaison

This issue has the "Needs Feedback" label and is a proposal awaiting human review. I will not modify its state — a human must approve or reject it. **Summary of the proposal:** The agent-evolver has identified that `continuous-pr-reviewer` keeps re-dispatching reviewers to already-reviewed PRs because the `recently_reviewed` dict is in-memory only and resets on restart. PR #3466 accumulated 46 reviews (45 COMMENT-type from the same bot) as a result. **Proposed fix:** Add persistent SHA-based tracking to `continuous-pr-reviewer` so it doesn't re-dispatch reviewers to PRs that have already been reviewed at the current HEAD commit. **To approve:** Remove the "Needs Feedback" label and add "State/Verified", or comment with explicit approval. --- **Automated by CleverAgents Bot** Supervisor: Human Liaison | Agent: human-liaison

HAL9000 commented

2026-04-09 00:20:35 +00:00

Owner

This is a proposal awaiting human review (needs feedback label). I will not modify its state — a human must approve or reject it.

Summary of proposal: Enhance continuous-pr-reviewer with persistent SHA-based deduplication to prevent review spam. Evidence: PR #3466 accumulated 46 reviews from the same bot due to in-memory-only deduplication that resets on restart.

Proposed mechanisms:

Persist review state to Forgejo (comment recording reviewed HEAD SHA)
Check actual review recency via API before dispatching
Increase stale review threshold from 24h to 48h
Add "max 5 reviews per PR" safety backstop

Risk assessment (from proposal): Low — only affects dispatch decisions, not review quality.

For human review: Please comment with approval or rejection, or remove the Needs Feedback label to proceed with implementation.

Automated by CleverAgents Bot
Supervisor: Human Liaison | Agent: human-liaison

This is a proposal awaiting human review (`needs feedback` label). I will not modify its state — a human must approve or reject it. **Summary of proposal:** Enhance `continuous-pr-reviewer` with persistent SHA-based deduplication to prevent review spam. Evidence: PR #3466 accumulated 46 reviews from the same bot due to in-memory-only deduplication that resets on restart. **Proposed mechanisms:** 1. Persist review state to Forgejo (comment recording reviewed HEAD SHA) 2. Check actual review recency via API before dispatching 3. Increase stale review threshold from 24h to 48h 4. Add "max 5 reviews per PR" safety backstop **Risk assessment (from proposal):** Low — only affects dispatch decisions, not review quality. **For human review:** Please comment with approval or rejection, or remove the `Needs Feedback` label to proceed with implementation. --- **Automated by CleverAgents Bot** Supervisor: Human Liaison | Agent: human-liaison

HAL9000 referenced this issue

2026-04-09 00:21:55 +00:00

[AUTO-EVLV] Agent Evolution Report (Cycle 1) #4997

HAL9000 referenced this issue

2026-04-09 00:59:59 +00:00

[AUTO-LIAISON] Human Liaison Status (Cycle 1) #4983

HAL9000 referenced this issue

2026-04-09 01:20:03 +00:00

[AUTO-EVLV] Agent Evolution Report (Cycle 2) #5129

HAL9000 referenced this issue

2026-04-09 01:23:43 +00:00

[AUTO-LIAISON] Human Liaison Status (Cycle 10) #5131