96 lines
4.6 KiB
Gherkin
96 lines
4.6 KiB
Gherkin
Feature: Checkpoint Operations for Resume Functionality
|
|
As a data engineer
|
|
I want to save and retrieve checkpoint data
|
|
So that I can resume processing from where it left off
|
|
|
|
Background:
|
|
Given an authenticated sheets tracker
|
|
And a sheet with the following datasets
|
|
| Dataset Name | Status | Error | Last Shard | Total Rows | Skip Descriptions | Last Updated |
|
|
| wordnet | IN PROGRESS | | 5 | 1000 | 50 | 2026-01-30 |
|
|
| yago | NOT STARTED | | | | | |
|
|
|
|
Scenario: Get checkpoint data for dataset with checkpoint
|
|
When I get checkpoint for "wordnet"
|
|
Then the checkpoint should contain last_completed_shard 5
|
|
And the checkpoint should contain total_rows 1000
|
|
And the checkpoint should contain skip_descriptions 50
|
|
And the checkpoint should contain timestamp "2026-01-30"
|
|
|
|
Scenario: Get checkpoint for dataset without checkpoint data
|
|
When I get checkpoint for "yago"
|
|
Then the checkpoint should be empty or None
|
|
|
|
Scenario: Get checkpoint for non-existent dataset
|
|
When I get checkpoint for "nonexistent"
|
|
Then the checkpoint should be None
|
|
|
|
Scenario: Get checkpoint when tracker not authenticated
|
|
Given an unauthenticated sheets tracker
|
|
When I get checkpoint for "wordnet"
|
|
Then the checkpoint should be None
|
|
|
|
Scenario: Update checkpoint data for existing dataset
|
|
When I update checkpoint for "yago" with shard 3, rows 500, descriptions 25
|
|
Then the update should succeed
|
|
When I get checkpoint for "yago"
|
|
Then the checkpoint should contain last_completed_shard 3
|
|
And the checkpoint should contain total_rows 500
|
|
And the checkpoint should contain skip_descriptions 25
|
|
And the checkpoint should have timestamp
|
|
|
|
Scenario: Update checkpoint for non-existent dataset
|
|
When I update checkpoint for "nonexistent" with shard 1, rows 100, descriptions 10
|
|
Then the update should return False
|
|
|
|
Scenario: Update checkpoint when tracker not authenticated
|
|
Given an unauthenticated sheets tracker
|
|
When I update checkpoint for "wordnet" with shard 10, rows 2000, descriptions 100
|
|
Then the update should return False
|
|
|
|
Scenario: Checkpoint columns with alternate names
|
|
Given a sheet with headers "Dataset Name, Status, Error, Last Completed Shard, Total Rows, Skip Descriptions, Last Updated"
|
|
And a sheet with the following datasets
|
|
| Dataset Name | Status | Error | Last Completed Shard | Total Rows | Skip Descriptions | Last Updated |
|
|
| wordnet | IN PROGRESS | | 10 | 5000 | 250 | 2026-01-30 |
|
|
When the tracker loads headers
|
|
Then last_shard_col should be identified correctly
|
|
And total_rows_col should be identified correctly
|
|
And skip_descriptions_col should be identified correctly
|
|
And last_updated_col should be identified correctly
|
|
|
|
Scenario: Checkpoint with missing columns
|
|
Given a sheet with headers "Dataset Name, Status, Error, Last Shard"
|
|
And a sheet with the following datasets
|
|
| Dataset Name | Status | Error | Last Shard |
|
|
| wordnet | IN PROGRESS | | 7 |
|
|
When I get checkpoint for "wordnet"
|
|
Then the checkpoint should contain last_completed_shard 7
|
|
And the checkpoint should not contain total_rows
|
|
And the checkpoint should not contain skip_descriptions
|
|
|
|
Scenario: Update checkpoint with only some columns present
|
|
Given a sheet with headers "Dataset Name, Status, Last Shard, Total Rows"
|
|
And a sheet with the following datasets
|
|
| Dataset Name | Status | Last Shard | Total Rows |
|
|
| wordnet | IN PROGRESS | 2 | 200 |
|
|
When I update checkpoint for "wordnet" with shard 5, rows 500, descriptions 50
|
|
Then the update should succeed
|
|
And only last_shard and total_rows should be updated
|
|
|
|
Scenario: Checkpoint with empty values
|
|
Given a sheet with the following datasets
|
|
| Dataset Name | Status | Error | Last Shard | Total Rows | Skip Descriptions |
|
|
| wordnet | IN PROGRESS | | | | |
|
|
When I get checkpoint for "wordnet"
|
|
Then the checkpoint values should default to 0
|
|
|
|
Scenario: Update checkpoint automatically sets timestamp
|
|
Given a sheet with headers "Dataset Name, Status, Last Updated"
|
|
And a sheet with the following datasets
|
|
| Dataset Name | Status | Last Updated |
|
|
| wordnet | IN PROGRESS | |
|
|
When I update checkpoint for "wordnet" with shard 1, rows 100, descriptions 10
|
|
Then the last_updated column should be populated with current timestamp
|
|
|