Add Retry Logic for HuggingFace Transient Errors #57

Open
opened 2026-01-27 14:38:27 +00:00 by aditya · 0 comments
Member

As a data engineer uploading large RDF datasets to HuggingFace, I want the upload script to automatically retry on transient server errors (503, 502, 500, network timeouts), so that multi-day uploads don't fail and require manual intervention when HuggingFace experiences temporary service issues.

Acceptance Criteria:

  • Script retries on HTTP 5xx errors with exponential backoff
  • README uploads include retry logic (max 10 attempts)
  • Detailed error logging shows retry attempts and error types
As a data engineer uploading large RDF datasets to HuggingFace, I want the upload script to automatically retry on transient server errors (503, 502, 500, network timeouts), so that multi-day uploads don't fail and require manual intervention when HuggingFace experiences temporary service issues. **Acceptance Criteria:** - Script retries on HTTP 5xx errors with exponential backoff - README uploads include retry logic (max 10 attempts) - Detailed error logging shows retry attempts and error types
aditya added the
Priority
High
Points
3
labels 2026-01-27 14:38:27 +00:00
aditya self-assigned this 2026-01-27 14:38:27 +00:00
aditya added this to the Sprint Jan 1st- 14th project 2026-01-27 14:38:27 +00:00
aditya added the
Type
Bug
label 2026-01-27 14:38:47 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: cleverdatasets/dataset-uploader#57