81 lines
2.7 KiB
Gherkin
81 lines
2.7 KiB
Gherkin
Feature: Complete Processing Workflow
|
|
As a data engineer
|
|
I want to process datasets through the complete workflow
|
|
So that they are uploaded to HuggingFace
|
|
|
|
Background:
|
|
Given a base directory for processing
|
|
And sheets tracking is enabled
|
|
|
|
Scenario: Process dataset with all steps
|
|
Given a dataset "wordnet" is available
|
|
When I process "wordnet" with all steps
|
|
Then the download step should complete
|
|
And the conversion step should complete
|
|
And the upload step should complete
|
|
And the sheet status should be "DONE"
|
|
|
|
Scenario: Skip download step
|
|
Given a dataset "wordnet" is already downloaded
|
|
When I process "wordnet" with "--skip-download"
|
|
Then the download step should be skipped
|
|
And the conversion step should complete
|
|
|
|
Scenario: Skip conversion step
|
|
Given a dataset "wordnet" is already converted
|
|
When I process "wordnet" with "--skip-convert"
|
|
Then the conversion step should be skipped
|
|
And the upload step should complete
|
|
|
|
Scenario: Skip upload step
|
|
When I process "wordnet" with "--skip-upload"
|
|
Then the upload step should be skipped
|
|
And the dataset should be converted locally
|
|
|
|
Scenario: Dry run mode
|
|
When I process "wordnet" with "--dry-run"
|
|
Then no actual processing should occur
|
|
And the output should show what would be done
|
|
|
|
Scenario: Handle download failure
|
|
Given download will fail for "wordnet"
|
|
When I process "wordnet"
|
|
Then the processing should fail
|
|
And the sheet status should be "NOT STARTED"
|
|
And the error message should be recorded
|
|
|
|
Scenario: Handle conversion failure
|
|
Given conversion will fail for "wordnet"
|
|
When I process "wordnet"
|
|
Then the processing should fail
|
|
And the sheet status should be "NOT STARTED"
|
|
And the upload error message should contain "Conversion failed"
|
|
|
|
Scenario: Handle upload failure
|
|
Given upload will fail for "wordnet"
|
|
When I process "wordnet"
|
|
Then the processing should fail
|
|
And the sheet status should be "NOT STARTED"
|
|
And the upload error message should contain "Upload failed"
|
|
|
|
Scenario: Remove downloaded files after processing
|
|
When I process "wordnet" with "--rm"
|
|
Then the downloaded files should be removed after success
|
|
|
|
Scenario: Retry failed download
|
|
Given download fails on first attempt
|
|
But succeeds on second attempt
|
|
When I process "wordnet" with "--repeat 3"
|
|
Then the download should succeed after retry
|
|
|
|
Scenario: Process special dataset FB15k-237
|
|
When I process "fb15k-237"
|
|
Then the specialized converter should be used
|
|
And the dataset should be uploaded successfully
|
|
|
|
Scenario: Process special dataset ConceptNet
|
|
When I process "conceptnet"
|
|
Then the ConceptNet converter should be used
|
|
And CSV files should be processed
|
|
|