freemo
e1f5c95bad
CI / lint (pull_request) Successful in 26s
CI / typecheck (pull_request) Successful in 51s
CI / security (pull_request) Successful in 58s
CI / quality (pull_request) Successful in 34s
CI / build (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 22s
CI / unit_tests (pull_request) Successful in 6m48s
CI / e2e_tests (pull_request) Successful in 15m40s
CI / integration_tests (pull_request) Successful in 22m59s
CI / coverage (pull_request) Successful in 10m59s
CI / docker (pull_request) Successful in 1m24s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 58m1s
Move ctx.plans.get_all_for_project(project.id) outside the plan iteration loop in LegacyDataMigrator.migrate_project_data to eliminate an N+1 database query pattern. Previously, the method fetched all plans from the database on every iteration of the plans loop, resulting in O(N) queries where N is the number of plans in the legacy plans.json file. For projects with many plans, this caused significant unnecessary database load during migration. The fix fetches all existing plans once before the loop begins, then uses the in-memory list for duplicate detection on each iteration. Also verified no other similar N+1 patterns exist in LegacyDataMigrator. Added a new Behave scenario 'get_all_for_project is called only once for multiple plans' that patches the repository method to count invocations and asserts exactly one call regardless of how many plans are migrated. ISSUES CLOSED: #3047
213 lines
9.2 KiB
Gherkin
213 lines
9.2 KiB
Gherkin
Feature: Legacy Data Migrator Coverage
|
|
As a developer maintaining the CleverAgents system
|
|
I want comprehensive tests for the legacy data migrator
|
|
So that migration from JSON files to SQLite works reliably
|
|
|
|
Background:
|
|
Given I have a temporary test directory for legacy migration
|
|
And I have a Unit of Work instance for testing
|
|
|
|
Scenario: Migrate project with all legacy JSON files
|
|
Given I have a legacy project with all JSON files
|
|
When I run the legacy data migration
|
|
Then the migration should return true
|
|
And the JSON files should be backed up
|
|
And the database should contain the migrated project data
|
|
And the database should contain the migrated plans
|
|
And the database should contain the migrated contexts
|
|
And the database should contain the migrated changes
|
|
|
|
Scenario: No migration needed when no cleveragents directory
|
|
Given I have a project path without .cleveragents directory
|
|
When I run the legacy data migration
|
|
Then the migration should return false
|
|
|
|
Scenario: No migration needed when no JSON files exist
|
|
Given I have a project with .cleveragents directory but no JSON files
|
|
When I run the legacy data migration
|
|
Then the migration should return false
|
|
|
|
Scenario: Migrate project with only plans.json
|
|
Given I have a legacy project with only plans.json file
|
|
When I run the legacy data migration
|
|
Then the migration should return true
|
|
And the plans.json file should be backed up
|
|
And the database should contain the migrated plans
|
|
|
|
Scenario: Migrate project with contexts.json and minimal plan
|
|
Given I have a legacy project with only contexts.json file for current plan
|
|
When I run the legacy data migration
|
|
Then the migration should return true
|
|
And the plans.json file should be backed up
|
|
And the contexts.json file should be backed up
|
|
And the database should contain the migrated contexts
|
|
|
|
Scenario: Migrate project with changes.json and minimal plan
|
|
Given I have a legacy project with only changes.json file for current plan
|
|
When I run the legacy data migration
|
|
Then the migration should return true
|
|
And the plans.json file should be backed up
|
|
And the changes.json file should be backed up
|
|
And the database should contain the migrated changes
|
|
|
|
Scenario: Handle existing project in database
|
|
Given I have an existing project in the database for legacy migration
|
|
And I have a legacy project with matching name
|
|
When I run the legacy data migration
|
|
Then the migration should use the existing project
|
|
And the migration should return true
|
|
|
|
Scenario: Create new project when not in database
|
|
Given I have a legacy project without database entry
|
|
When I run the legacy data migration
|
|
Then a new project should be created in the database
|
|
And the migration should return true
|
|
|
|
Scenario: Handle project without project.name file
|
|
Given I have a legacy project without project.name file
|
|
When I run the legacy data migration
|
|
Then the project name should use the directory name
|
|
And the migration should return true
|
|
|
|
Scenario: Skip existing plans during migration
|
|
Given I have a legacy project with plans already in database
|
|
When I run the legacy data migration
|
|
Then existing plans should not be duplicated
|
|
And the migration should return true
|
|
|
|
Scenario: Map plan status values correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I map plan status "OVERLOADED" to enum
|
|
Then the migrator result should be PlanStatus.PENDING
|
|
When I map plan status "pending" to enum
|
|
Then the migrator result should be PlanStatus.PENDING
|
|
When I map plan status "building" to enum
|
|
Then the migrator result should be PlanStatus.BUILDING
|
|
When I map plan status "built" to enum
|
|
Then the migrator result should be PlanStatus.BUILT
|
|
When I map plan status "failed" to enum
|
|
Then the migrator result should be PlanStatus.ERROR
|
|
When I map plan status "applied" to enum
|
|
Then the migrator result should be PlanStatus.APPLIED
|
|
When I map plan status "unknown" to enum
|
|
Then the migrator result should be PlanStatus.PENDING
|
|
|
|
Scenario: Map plan build data correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I map plan build data with valid dictionary
|
|
Then the migrator result should be None
|
|
When I map plan build data with None
|
|
Then the migrator result should be None
|
|
When I map plan build data with empty dictionary
|
|
Then the migrator result should be None
|
|
When I map plan build data with non-dict value
|
|
Then the migrator result should be None
|
|
|
|
Scenario: Map context type values correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I map context type "file" to enum
|
|
Then the migrator result should be ContextType.FILE
|
|
When I map context type "directory" to enum
|
|
Then the migrator result should be ContextType.DIRECTORY
|
|
When I map context type "url" to enum
|
|
Then the migrator result should be ContextType.URL
|
|
When I map context type "document" to enum
|
|
Then the migrator result should be ContextType.NOTE
|
|
When I map context type "unknown" to enum
|
|
Then the migrator result should be ContextType.FILE
|
|
|
|
Scenario: Map operation values correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I map operation "create" to enum
|
|
Then the migrator result should be OperationType.CREATE
|
|
When I map operation "modify" to enum
|
|
Then the migrator result should be OperationType.MODIFY
|
|
When I map operation "delete" to enum
|
|
Then the migrator result should be OperationType.DELETE
|
|
When I map operation "rename" to enum
|
|
Then the migrator result should be OperationType.MOVE
|
|
When I map operation "unknown" to enum
|
|
Then the migrator result should be OperationType.CREATE
|
|
|
|
Scenario: Parse datetime strings correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I parse datetime string "2024-01-15T10:30:45"
|
|
Then the migrator result should be a valid datetime object
|
|
When I parse datetime string "2024-01-15 10:30:45"
|
|
Then the migrator result should be a valid datetime object
|
|
When I parse datetime string None
|
|
Then the migrator result should be None
|
|
When I parse datetime string "invalid-date"
|
|
Then the migrator result should be None
|
|
When I parse datetime string ""
|
|
Then the migrator result should be None
|
|
|
|
Scenario: Set current plan during migration
|
|
Given I have a legacy project with multiple plans
|
|
And one plan is marked as current
|
|
When I run the legacy data migration
|
|
Then the current plan should be set in the project
|
|
And the migration should return true
|
|
|
|
Scenario: Handle contexts without current plan
|
|
Given I have a legacy project with contexts but no current plan
|
|
When I run the legacy data migration
|
|
Then the contexts should not be migrated
|
|
And the migration should return true
|
|
|
|
Scenario: Handle changes without current plan
|
|
Given I have a legacy project with changes but no current plan
|
|
When I run the legacy data migration
|
|
Then the changes should not be migrated
|
|
And the migration should return true
|
|
|
|
Scenario: Migrate contexts successfully
|
|
Given I have a legacy project with contexts to migrate
|
|
When I run the legacy data migration
|
|
Then the contexts should be migrated successfully
|
|
And the migration should return true
|
|
|
|
Scenario: Handle plan with all optional fields
|
|
Given I have a legacy project with plan containing all fields
|
|
When I run the legacy data migration
|
|
Then all plan fields should be migrated correctly
|
|
And the migration should return true
|
|
|
|
Scenario: Handle plan with minimal fields
|
|
Given I have a legacy project with plan containing minimal fields
|
|
When I run the legacy data migration
|
|
Then the plan should be migrated with defaults
|
|
And the migration should return true
|
|
|
|
Scenario: Use check_and_migrate_legacy_data function
|
|
Given I have a legacy project with JSON files
|
|
When I call check_and_migrate_legacy_data function
|
|
Then the function should return true
|
|
And the migration should be performed
|
|
|
|
Scenario: Map uppercase status values correctly
|
|
Given I have a LegacyDataMigrator instance
|
|
When I map plan status "PENDING" to enum
|
|
Then the migrator result should be PlanStatus.PENDING
|
|
When I map plan status "BUILDING" to enum
|
|
Then the migrator result should be PlanStatus.BUILDING
|
|
When I map plan status "BUILT" to enum
|
|
Then the migrator result should be PlanStatus.BUILT
|
|
|
|
Scenario: Handle invalid JSON files gracefully
|
|
Given I have a legacy project with invalid JSON files
|
|
When I run the legacy data migration
|
|
Then the migration should handle errors gracefully
|
|
And the migration should return false
|
|
|
|
Scenario: Handle file system errors during backup
|
|
Given I have a legacy project with read-only JSON files
|
|
When I run the legacy data migration
|
|
Then the migration should handle backup errors gracefully
|
|
Scenario: get_all_for_project is called only once for multiple plans
|
|
Given I have a legacy project with multiple plans to migrate efficiently
|
|
When I run the legacy data migration with query tracking
|
|
Then get_all_for_project should be called exactly once
|
|
And the migration should return true
|
|
And all plans should be migrated correctly
|