chore(agents): improve ca-subtask-loop — add meaningful-change verification
CI / lint (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 48s
CI / security (pull_request) Successful in 1m3s
CI / build (pull_request) Successful in 24s
CI / helm (pull_request) Successful in 23s
CI / quality (pull_request) Successful in 3m44s
CI / unit_tests (pull_request) Successful in 9m56s
CI / e2e_tests (pull_request) Successful in 16m41s
CI / integration_tests (pull_request) Successful in 22m54s
CI / docker (pull_request) Successful in 11s
CI / coverage (pull_request) Successful in 11m18s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m47s
CI / lint (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 48s
CI / security (pull_request) Successful in 1m3s
CI / build (pull_request) Successful in 24s
CI / helm (pull_request) Successful in 23s
CI / quality (pull_request) Successful in 3m44s
CI / unit_tests (pull_request) Successful in 9m56s
CI / e2e_tests (pull_request) Successful in 16m41s
CI / integration_tests (pull_request) Successful in 22m54s
CI / docker (pull_request) Successful in 11s
CI / coverage (pull_request) Successful in 11m18s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m47s
Approved proposal: #2443 Pattern: workflow_fix Evidence: PR #1513 contained only a single comment line as its entire diff, yet passed all quality gates (lint, typecheck, tests, coverage) and was submitted for review, consuming 8 independent reviewer slots. All quality gates verify code quality, not code existence — a change that adds only comments passes every gate. Fix: Add Step 1.5 (Meaningful Change Verification) between the implementer invocation and test-writing step. This checks git diff output for functional code changes and rejects attempts that produce empty, comment-only, or trivially small diffs (< 3 functional lines), immediately escalating to the next tier without wasting quality gate and reviewer capacity. ISSUES CLOSED: #2443
This commit is contained in:
@@ -109,6 +109,38 @@ Loop FOREVER:
|
||||
Pass: working directory, ref summary, subtask description,
|
||||
spec context, issue details, attempt_log (if attempt > 1)
|
||||
|
||||
# Step 1.5: Meaningful Change Verification
|
||||
# Before running expensive quality gates, verify the implementer
|
||||
# actually produced functional code changes — not just comments,
|
||||
# whitespace, or empty diffs.
|
||||
diff_output = run `git diff --stat` in working directory
|
||||
full_diff = run `git diff` in working directory
|
||||
|
||||
reject_attempt = false
|
||||
if diff_output is empty:
|
||||
reject_attempt = true # No changes at all
|
||||
else:
|
||||
# Analyze the diff content for meaningful changes
|
||||
functional_lines = count lines in full_diff that:
|
||||
- Are additions (start with "+") or deletions (start with "-")
|
||||
- Are NOT comment-only lines (e.g., lines where the only content
|
||||
after +/- is a Python comment starting with #)
|
||||
- Are NOT whitespace-only changes
|
||||
- Are NOT empty lines
|
||||
- Are NOT import-only additions with no corresponding usage
|
||||
if functional_lines < 3:
|
||||
reject_attempt = true # Trivially small or comment-only change
|
||||
|
||||
if reject_attempt:
|
||||
Record in attempt_log:
|
||||
"Attempt <N> rejected: implementer produced no meaningful code
|
||||
changes (empty diff, comment-only, or fewer than 3 functional
|
||||
lines). Skipping quality gates and escalating."
|
||||
# Reset the working directory to avoid polluting the next attempt
|
||||
run `git checkout -- .` in working directory
|
||||
# Continue to next attempt (escalation) WITHOUT running quality gates
|
||||
continue
|
||||
|
||||
# Step 2: Write / Review ALL Tests IN PARALLEL
|
||||
# Launch ALL test writers simultaneously — do not serialize.
|
||||
invoke ALL of the following IN PARALLEL:
|
||||
|
||||
Reference in New Issue
Block a user