cleveragents/cleveragents-core

Fork 3

Proposal: Add milestone-based PR prioritization to ca-continuous-pr-reviewer #3069

New issue

Open

opened 2026-04-05 04:52:21 +00:00 by freemo · 2 comments

freemo commented

2026-04-05 04:52:21 +00:00

Owner

Agent Improvement Proposal

Pattern Detected

Type: Workflow improvement — review/merge throughput optimization
Affected Agent: ca-continuous-pr-reviewer
Evidence: During Session 3 (issue #2877, watchdog cycles 1-18):

89 open PRs accumulated across milestones v3.2.0 through v3.7.0
Only 1 PR merged in 90 minutes of operation
PRs are reviewed in arbitrary discovery order (whichever Forgejo returns first)
Earlier milestones (v3.2.0, v3.3.0) are significantly overdue but their PRs compete equally with v3.7.0 PRs for reviewer attention

The current PR discovery logic in Step 1 (Source A) iterates over open_prs in whatever order Forgejo returns them. With 89 open PRs, a reviewer might pick up a v3.7.0 PR while v3.2.0 PRs (which are months overdue) sit unreviewed. This delays milestone completion for the most critical milestones.

Additionally, the reviewer does not prioritize PRs that are already approved and just need a merge retry (Source B) over PRs that need full review (Source A). A merge retry takes seconds; a full review takes minutes. Prioritizing merge retries would increase merge throughput significantly.

Proposed Change

Add a PR prioritization step between Step 1 (Discover work) and Step 3 (Dispatch parallel reviewers) in ca-continuous-pr-reviewer.md:

# After building work_items list, sort by priority:
work_items.sort(key=lambda item: (
    # Priority 1: Merge retries first (already approved, just need merge)
    0 if item.type == "merge_retry" else 1,
    # Priority 2: Earlier milestones first (v3.2.0 before v3.7.0)
    milestone_sort_key(item.pr.milestone),
    # Priority 3: PRs with passing CI before PRs with pending CI
    0 if item.ci_status == "success" else 1,
    # Priority 4: Smaller PRs first (faster to review)
    item.pr.additions + item.pr.deletions,
))

Where milestone_sort_key extracts the version number from the milestone title (e.g., "v3.2.0" → (3, 2, 0)) and sorts ascending, with no-milestone PRs sorted last.

Expected Impact

Merge retries processed first: Approved PRs that just need a merge attempt complete in seconds instead of waiting behind full reviews. This alone could dramatically increase merge throughput.
Earlier milestones complete first: v3.2.0 PRs (months overdue) get reviewed before v3.7.0 PRs, unblocking milestone closure.
CI-passing PRs prioritized: PRs that can merge immediately get reviewed before PRs that will need CI fixes, reducing wasted review cycles.
Smaller PRs first: Quick wins are processed first, increasing the merge count per cycle.

Risk Assessment

Low risk: The change only reorders the work queue. It does not change any review logic, merge behavior, or claim protocol.
Potential concern: v3.7.0 PRs could be starved if there's a constant stream of earlier-milestone PRs. Mitigated by the fact that earlier milestones have fewer remaining issues and will eventually complete, at which point v3.7.0 PRs get full attention.
Edge case: PRs without a milestone are sorted last. This is intentional — milestone-less PRs are typically agent improvement PRs or docs PRs that are lower priority than milestone work.

This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the needs feedback label, add State/Verified, or comment with approval.

Automated by CleverAgents Bot
Supervisor: Agent Evolver | Agent: ca-agent-evolver

## Agent Improvement Proposal ### Pattern Detected **Type**: Workflow improvement — review/merge throughput optimization **Affected Agent**: `ca-continuous-pr-reviewer` **Evidence**: During Session 3 (issue #2877, watchdog cycles 1-18): - **89 open PRs** accumulated across milestones v3.2.0 through v3.7.0 - **Only 1 PR merged** in 90 minutes of operation - PRs are reviewed in arbitrary discovery order (whichever Forgejo returns first) - Earlier milestones (v3.2.0, v3.3.0) are significantly overdue but their PRs compete equally with v3.7.0 PRs for reviewer attention The current PR discovery logic in Step 1 (Source A) iterates over `open_prs` in whatever order Forgejo returns them. With 89 open PRs, a reviewer might pick up a v3.7.0 PR while v3.2.0 PRs (which are months overdue) sit unreviewed. This delays milestone completion for the most critical milestones. Additionally, the reviewer does not prioritize PRs that are already approved and just need a merge retry (Source B) over PRs that need full review (Source A). A merge retry takes seconds; a full review takes minutes. Prioritizing merge retries would increase merge throughput significantly. ### Proposed Change Add a **PR prioritization step** between Step 1 (Discover work) and Step 3 (Dispatch parallel reviewers) in `ca-continuous-pr-reviewer.md`: ``` # After building work_items list, sort by priority: work_items.sort(key=lambda item: ( # Priority 1: Merge retries first (already approved, just need merge) 0 if item.type == "merge_retry" else 1, # Priority 2: Earlier milestones first (v3.2.0 before v3.7.0) milestone_sort_key(item.pr.milestone), # Priority 3: PRs with passing CI before PRs with pending CI 0 if item.ci_status == "success" else 1, # Priority 4: Smaller PRs first (faster to review) item.pr.additions + item.pr.deletions, )) ``` Where `milestone_sort_key` extracts the version number from the milestone title (e.g., "v3.2.0" → (3, 2, 0)) and sorts ascending, with no-milestone PRs sorted last. ### Expected Impact - **Merge retries processed first**: Approved PRs that just need a merge attempt complete in seconds instead of waiting behind full reviews. This alone could dramatically increase merge throughput. - **Earlier milestones complete first**: v3.2.0 PRs (months overdue) get reviewed before v3.7.0 PRs, unblocking milestone closure. - **CI-passing PRs prioritized**: PRs that can merge immediately get reviewed before PRs that will need CI fixes, reducing wasted review cycles. - **Smaller PRs first**: Quick wins are processed first, increasing the merge count per cycle. ### Risk Assessment - **Low risk**: The change only reorders the work queue. It does not change any review logic, merge behavior, or claim protocol. - **Potential concern**: v3.7.0 PRs could be starved if there's a constant stream of earlier-milestone PRs. Mitigated by the fact that earlier milestones have fewer remaining issues and will eventually complete, at which point v3.7.0 PRs get full attention. - **Edge case**: PRs without a milestone are sorted last. This is intentional — milestone-less PRs are typically agent improvement PRs or docs PRs that are lower priority than milestone work. --- *This is a proposal from the agent evolver. A human must approve this issue before the change will be implemented. To approve: remove the `needs feedback` label, add `State/Verified`, or comment with approval.* --- **Automated by CleverAgents Bot** Supervisor: Agent Evolver | Agent: ca-agent-evolver

freemo added the

labels

2026-04-05 04:53:50 +00:00

freemo commented

2026-04-05 04:58:58 +00:00

Author

Owner

approved

freemo commented

2026-04-05 05:08:21 +00:00

Author

Owner

Transition Forgejo issue #3069 from State/Unverified to State/Verified.

Preconditions:

Current state: State/Unverified
Not Paused; no blocker conditions apply for this transition

Actions performed:

Removed labels: State/Unverified
Removed label: needs feedback (if present; not present on this issue)
Added label: State/Verified

Status: Transition performed according to Ticket Lifecycle. If the issue already had State/Verified, no changes would be made.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-state-updater

Transition Forgejo issue #3069 from State/Unverified to State/Verified. Preconditions: - Current state: State/Unverified - Not Paused; no blocker conditions apply for this transition Actions performed: - Removed labels: State/Unverified - Removed label: needs feedback (if present; not present on this issue) - Added label: State/Verified Status: Transition performed according to Ticket Lifecycle. If the issue already had State/Verified, no changes would be made. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-state-updater

freemo referenced this issue from a commit

2026-04-05 06:18:50 +00:00

chore(agents): add milestone-based PR prioritization to reviewer

freemo referenced this issue from a pull request that will close it,

2026-04-05 06:19:40 +00:00

chore(agents): add milestone-based PR prioritization to ca-continuous-pr-reviewer #3111

freemo referenced this issue

2026-04-05 06:21:51 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3060

freemo referenced this issue

2026-04-05 06:58:31 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3060