dataset-uploader

#59 opened 2026-01-29 11:48:23 +00:00 by aditya

Implement RDF/XML streaming functionality for robust large-scale dataset processing

Priority

High

Type

Feature

#58 opened 2026-01-29 11:34:08 +00:00 by aditya 0 / 4

Add Retry Logic for HuggingFace Transient Errors

#57 opened 2026-01-27 14:38:27 +00:00 by aditya Sprint Jan 1st- 14th

CleverDatasets needs a way to override Skipping... Already exists on.

#55 opened 2026-01-26 23:40:27 +00:00 by brent.edwards

CleverDatasets does not handle dates before 1 AD.

#54 opened 2026-01-20 20:08:37 +00:00 by brent.edwards

Optimize RDF Streaming with Description Skipping and Parallel Parsing

#52 opened 2026-01-16 14:39:03 +00:00 by aditya Sprint Jan 1st- 14th

Integrate Google Sheets for Dataset Processing Status Tracking

#51 opened 2026-01-16 14:27:50 +00:00 by aditya Sprint Jan 1st- 14th

Implement Streaming Pipeline for Large RDF Datasets

#50 opened 2026-01-16 14:26:09 +00:00 by aditya Sprint Jan 1st- 14th

Add multiple files to the dataset_config.

#36 opened 2025-12-16 19:24:16 +00:00 by brent.edwards

Add xz decompression

#35 opened 2025-12-16 19:20:05 +00:00 by brent.edwards

Fix download resume functionality in rdf_dataset_downloader.py

#32 opened 2025-12-10 09:04:52 +00:00 by aditya

How are temporary files used?

#30 opened 2025-12-09 22:07:04 +00:00 by brent.edwards

Allow download via ftp.

#27 opened 2025-12-07 17:59:38 +00:00 by brent.edwards

Command-line option to turn off convert step

#26 opened 2025-12-07 17:57:09 +00:00 by brent.edwards

Fix upload_all_datasets.2.py: streaming converter, default behavior, and dry-run mode

#20 opened 2025-11-28 13:46:01 +00:00 by aditya

Rewrite upload_all_datasets with more clarity and fewer bugs.

Points

8

#19 opened 2025-11-24 21:46:23 +00:00 by brent.edwards

1

The --rm parameter should remove cached files.

#18 opened 2025-11-22 00:03:07 +00:00 by brent.edwards

Add proper error messages and logging.

#16 opened 2025-11-20 21:47:32 +00:00 by brent.edwards

Refactor 5 RDF converter scripts to unified strategy pattern

#15 opened 2025-11-20 10:07:56 +00:00 by aditya

1

Create proper CLI entry points.

#14 opened 2025-11-20 10:03:15 +00:00 by aditya

1