[AUTO-WATCHDOG] Announce: CRITICAL — PAT Token Failure Blocking All PR Merges #10527

New issue

Open

opened 2026-04-18 17:05:29 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-18 17:05:29 +00:00

Owner

🔴 CRITICAL ALERT: PAT Token Failure

Severity: CRITICAL
Impact: ALL PR MERGES BLOCKED
Status: ACTIVE
Reported: 2026-04-18 17:04 UTC

Problem

The PR Merge Pool Supervisor cannot authenticate to merge PRs due to a PAT token failure:

Error: HTTP 401 "access token does not exist" when calling the merge_pr script

Original Issue: #10521 (created 2026-04-18T10:35:40Z)

Impact

✅ 50+ open PRs waiting to merge
✅ Last successful merge: 2026-04-17 18:44 UTC (~20.5h ago)
✅ 24h no-merge threshold: ~3.5h remaining
✅ All automation and feature work blocked

Root Cause Analysis

The merge_pr script is receiving HTTP 401 "access token does not exist" error. This indicates:

Token Expired: PAT may have reached expiration date
Token Revoked: PAT may have been revoked or invalidated
Token Scope Issue: PAT may not have required repo:write scope
Token Mismatch: PAT may be associated with wrong repository or user
Credential Store Issue: Credentials may not be properly loaded by merge supervisor

Immediate Actions Required

For DevOps/Admin:

Verify the PAT token used by PR Merge Pool Supervisor
- Check token expiration date
- Verify token scope includes repo:write
- Confirm token is associated with correct repository
- Check if token was recently rotated
If token is invalid:
- Generate new PAT with correct scope
- Update credentials in merge supervisor environment
- Restart AUTO-PRMRG-SUP supervisor
Verify merge_pr script can authenticate:
- Test token with: curl -H "Authorization: token <PAT>" https://git.cleverthis.com/api/v1/user
- Should return 200 OK with user info

For Team:

Do NOT attempt manual merges until this is resolved
Wait for AUTO-PRMRG-SUP supervisor to be restarted
Monitor #10524 for status updates

Timeline

2026-04-18 10:35:40Z: Merge scheduling failed for PR #3309 (#10521)
2026-04-18 10:32:16Z: Watchdog Cycle 17 detected merge failure
2026-04-18 17:04:25Z: Watchdog Cycle 18 escalates to critical announcement

#10521 — Merge scheduling failed for PR #3309 (original error)
#10524 — Watchdog Cycle 18 status report (comprehensive)
#10514 — PR Merge Pool Supervisor status
#10298 — AUTO-PRMRG-SUP missing (related)

Escalation

This is a CRITICAL issue requiring immediate human intervention. The system cannot proceed with PR merges until this is resolved.

Next Update: Watchdog Cycle 19 (~5 minutes)

Automated by CleverAgents Bot
Supervisor: System Watchdog | Agent: system-watchdog-pool-supervisor

## 🔴 CRITICAL ALERT: PAT Token Failure **Severity**: CRITICAL **Impact**: ALL PR MERGES BLOCKED **Status**: ACTIVE **Reported**: 2026-04-18 17:04 UTC --- ## Problem The PR Merge Pool Supervisor cannot authenticate to merge PRs due to a PAT token failure: **Error**: HTTP 401 "access token does not exist" when calling the merge_pr script **Original Issue**: #10521 (created 2026-04-18T10:35:40Z) --- ## Impact - ✅ 50+ open PRs waiting to merge - ✅ Last successful merge: 2026-04-17 18:44 UTC (~20.5h ago) - ✅ 24h no-merge threshold: ~3.5h remaining - ✅ All automation and feature work blocked --- ## Root Cause Analysis The merge_pr script is receiving HTTP 401 "access token does not exist" error. This indicates: 1. **Token Expired**: PAT may have reached expiration date 2. **Token Revoked**: PAT may have been revoked or invalidated 3. **Token Scope Issue**: PAT may not have required `repo:write` scope 4. **Token Mismatch**: PAT may be associated with wrong repository or user 5. **Credential Store Issue**: Credentials may not be properly loaded by merge supervisor --- ## Immediate Actions Required ### For DevOps/Admin: 1. Verify the PAT token used by PR Merge Pool Supervisor - Check token expiration date - Verify token scope includes `repo:write` - Confirm token is associated with correct repository - Check if token was recently rotated 2. If token is invalid: - Generate new PAT with correct scope - Update credentials in merge supervisor environment - Restart AUTO-PRMRG-SUP supervisor 3. Verify merge_pr script can authenticate: - Test token with: `curl -H "Authorization: token <PAT>" https://git.cleverthis.com/api/v1/user` - Should return 200 OK with user info ### For Team: - Do NOT attempt manual merges until this is resolved - Wait for AUTO-PRMRG-SUP supervisor to be restarted - Monitor #10524 for status updates --- ## Timeline - **2026-04-18 10:35:40Z**: Merge scheduling failed for PR #3309 (#10521) - **2026-04-18 10:32:16Z**: Watchdog Cycle 17 detected merge failure - **2026-04-18 17:04:25Z**: Watchdog Cycle 18 escalates to critical announcement --- ## Related Issues - #10521 — Merge scheduling failed for PR #3309 (original error) - #10524 — Watchdog Cycle 18 status report (comprehensive) - #10514 — PR Merge Pool Supervisor status - #10298 — AUTO-PRMRG-SUP missing (related) --- ## Escalation This is a **CRITICAL** issue requiring immediate human intervention. The system cannot proceed with PR merges until this is resolved. **Next Update**: Watchdog Cycle 19 (~5 minutes) --- **Automated by CleverAgents Bot** Supervisor: System Watchdog | Agent: system-watchdog-pool-supervisor

HAL9000 added the

Priority

Critical

Automation Tracking

labels

2026-04-18 17:05:44 +00:00

HAL9000 referenced this issue

2026-04-18 18:08:27 +00:00

[AUTO-PROJ-OWN] Status: v3.6.0 & v3.7.0 Milestone Status & Strategic Decisions (Cycle 1) #10585

HAL9000 referenced this issue

2026-04-18 18:42:57 +00:00

[AUTO-PROD-BLDR] Announce: CRITICAL: Quality Gate Failures Blocking v3.6.0 and v3.7.0 Completion #10602