Commit Graph

30 Commits

Author SHA1 Message Date
brent.edwards 3022ce4e9e Corrections to code and printing missing fields. 2026-02-01 14:53:19 -08:00
brent.edwards 8393fa3c5d Removed a circular dependency. 2026-01-29 17:29:56 -08:00
brent.edwards 06d499e4e9 Adding assertions to turn off pyright warnings. 2026-01-29 14:18:54 -08:00
brent.edwards 49f6a288d9 Thank you, Aditya, for pointing out missing "not"s. 2026-01-29 11:56:33 -08:00
brent.edwards cc1df798db Changes made to correctly use Bloom filters. 2026-01-28 18:55:23 -08:00
brent.edwards 6ad7cb2e90 Changes requested by Aditya. 2026-01-27 18:20:03 -08:00
brent.edwards b250bb79a8 feat: add string extraction and validation for datasets
Co-authored-by: aider (openrouter/openai/gpt-5.2-codex) <aider@aider.chat>
2026-01-26 18:51:08 -08:00
brent.edwards 9cee285cca refactor: move REQUIRED_COLUMNS definition in dataset_validator.py 2026-01-26 18:50:57 -08:00
brent.edwards 597267f4b0 fix: escape braces in f-strings within documentation strings
Co-authored-by: aider (openrouter/openai/gpt-5.2-codex) <aider@aider.chat>
2026-01-23 15:16:02 -08:00
brent.edwards 41a6f1a8ce refactor: update validation to return download path and handle multiple parquet files
Co-authored-by: aider (openrouter/openai/gpt-5.2-codex) <aider@aider.chat>
2026-01-23 15:14:11 -08:00
aditya c21be4fe14 fix: fix sheet updation logic; update README.md for pycharm devcontainer 2026-01-14 19:33:01 +05:30
aditya 42cadec23f fix: fix issues as per review feedback 2026-01-14 19:33:01 +05:30
aditya b11910103a feat: integrate excel integration using gcp 2026-01-14 19:32:56 +05:30
brent.edwards f8a7739016 Minor changes. 2025-12-30 18:06:34 -08:00
aditya 2b1537d4f9 fix: fix ruff check errors 2025-12-29 18:49:57 +05:30
aditya 1bb26305ce fix: fix path too long issue; fix pickling issue in unified converter script 2025-12-26 18:21:02 +05:30
aditya 5b215aff76 fix: complete the unified script to add add the new features as streaming parallel; update the upload_all_dataset.py to use the unified script 2025-12-24 21:00:32 +05:30
aditya d564739f72 Merge branch '35-xz-compression' into stream-download-convert-upload 2025-12-23 20:20:36 +05:30
brent.edwards ff3939d937 fix: improve RDF format detection and parsing for UniProt dataset
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-16 19:38:30 -08:00
brent.edwards 0784e3e275 Fixes suggested by Aider. 2025-12-16 18:21:25 -08:00
brent.edwards 4da8793b18 THIS DOES NOT WORK. 2025-12-11 21:26:52 -08:00
brent.edwards 18a724eef9 More changes. 2025-12-09 20:01:45 -08:00
brent.edwards bac71ff1cc Minor improvements. 2025-12-09 13:28:30 -08:00
brent.edwards 37b3ee6e91 Final version; what was used to send wikidata-truthy. 2025-12-07 09:47:04 -08:00
brent.edwards a6928d0ce2 Changes requested by Kyle; thank you! 2025-12-04 14:11:41 -08:00
brent.edwards 9534336c15 Replacing upload_all_datasets.py with the previous
upload_all_datasets.2.py
2025-12-02 14:40:15 -08:00
brent.edwards 6b73289c24 One-half of the conversion is done, and the structure is written. 2025-11-24 18:59:44 -08:00
brent.edwards c776025b0f Changes made to make the downloader work. 2025-11-23 16:25:31 -08:00
brent.edwards e3bee8af4f feat: enhance error handling and reporting in dataset processing scripts
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-11-20 17:13:26 -08:00
brent.edwards 8784b798a6 Initial checkin with scripts. 2025-11-18 13:18:15 -08:00