cleveragents/cleveragents-core

Fork 3

BUG-HUNT: [error-handling] Broad Exception Handling in LegacyDataMigrator Silently Fails #2969

New issue

Open

opened 2026-04-05 02:58:56 +00:00 by freemo · 2 comments

freemo commented

2026-04-05 02:58:56 +00:00

Owner

Metadata

Branch: fix/legacy-migrator-error-handling
Commit Message: fix(database): refactor broad exception handling in LegacyDataMigrator to prevent silent migration failures
Milestone: v3.7.0
Parent Epic: #1020

Bug Report: [error-handling] — Broad Exception Handling in LegacyDataMigrator Silently Fails

Severity Assessment

Impact: Silent data loss or incomplete data migrations that are difficult to debug. If a migration fails due to a file system error or corrupted JSON, the system reports it as a successful no-op.
Likelihood: Medium. This could occur if legacy JSON files are corrupted or if there are permission issues in the project directory.
Priority: Medium

Location

File: src/cleveragents/infrastructure/database/legacy_migrator.py
Function/Class: LegacyDataMigrator.migrate_project_data
Lines: 241-243

Description

The try...except (json.JSONDecodeError, OSError) block in the migrate_project_data method catches broad exceptions and returns False. This behavior is problematic because it treats a failed migration (due to e.g. corrupted JSON or a file permission error) the same as a scenario where no migration was needed. This can lead to silent failures, making it very difficult to diagnose why legacy data was not migrated.

Evidence

        except (json.JSONDecodeError, OSError):
            # Handle JSON parsing errors, file system errors gracefully
            return False

Expected Behavior

The exception block should distinguish between different failure modes. A FileNotFoundError might be acceptable to ignore if the legacy files are optional, but json.JSONDecodeError or other OSError subtypes (like PermissionError) indicate a real problem that should be logged as a warning or error, and potentially re-raised as a specific MigrationError.

Actual Behavior

The method returns False, incorrectly signaling that no migration was necessary, when in fact a migration was attempted and failed.

Suggested Fix

Refactor the exception handling to be more specific. Log errors with details and consider raising a custom exception to allow callers to handle migration failures explicitly.

Subtasks

Audit LegacyDataMigrator.migrate_project_data exception handling at lines 241-243
Separate FileNotFoundError (acceptable no-op) from json.JSONDecodeError and PermissionError (real failures)
Add structured logging (logger.warning or logger.error) for real failure cases with exception details
Define or reuse a MigrationError custom exception for unrecoverable migration failures
Refactor the except block to re-raise or wrap real errors as MigrationError
Update callers of migrate_project_data to handle MigrationError appropriately
Tests (pytest/Behave): Add unit tests covering corrupted JSON, permission errors, and missing file scenarios
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

FileNotFoundError is silently ignored (no migration needed — acceptable)
json.JSONDecodeError and PermissionError are logged and raise MigrationError
No silent failure path remains in migrate_project_data
All new and existing tests pass
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/legacy-migrator-error-handling` - **Commit Message**: `fix(database): refactor broad exception handling in LegacyDataMigrator to prevent silent migration failures` - **Milestone**: v3.7.0 - **Parent Epic**: #1020 ## Bug Report: [error-handling] — Broad Exception Handling in LegacyDataMigrator Silently Fails ### Severity Assessment - **Impact**: Silent data loss or incomplete data migrations that are difficult to debug. If a migration fails due to a file system error or corrupted JSON, the system reports it as a successful no-op. - **Likelihood**: Medium. This could occur if legacy JSON files are corrupted or if there are permission issues in the project directory. - **Priority**: Medium ### Location - **File**: `src/cleveragents/infrastructure/database/legacy_migrator.py` - **Function/Class**: `LegacyDataMigrator.migrate_project_data` - **Lines**: 241-243 ### Description The `try...except (json.JSONDecodeError, OSError)` block in the `migrate_project_data` method catches broad exceptions and returns `False`. This behavior is problematic because it treats a failed migration (due to e.g. corrupted JSON or a file permission error) the same as a scenario where no migration was needed. This can lead to silent failures, making it very difficult to diagnose why legacy data was not migrated. ### Evidence ```python except (json.JSONDecodeError, OSError): # Handle JSON parsing errors, file system errors gracefully return False ``` ### Expected Behavior The exception block should distinguish between different failure modes. A `FileNotFoundError` might be acceptable to ignore if the legacy files are optional, but `json.JSONDecodeError` or other `OSError` subtypes (like `PermissionError`) indicate a real problem that should be logged as a warning or error, and potentially re-raised as a specific `MigrationError`. ### Actual Behavior The method returns `False`, incorrectly signaling that no migration was necessary, when in fact a migration was attempted and failed. ### Suggested Fix Refactor the exception handling to be more specific. Log errors with details and consider raising a custom exception to allow callers to handle migration failures explicitly. ### Category error-handling ## Subtasks - [ ] Audit `LegacyDataMigrator.migrate_project_data` exception handling at lines 241-243 - [ ] Separate `FileNotFoundError` (acceptable no-op) from `json.JSONDecodeError` and `PermissionError` (real failures) - [ ] Add structured logging (`logger.warning` or `logger.error`) for real failure cases with exception details - [ ] Define or reuse a `MigrationError` custom exception for unrecoverable migration failures - [ ] Refactor the `except` block to re-raise or wrap real errors as `MigrationError` - [ ] Update callers of `migrate_project_data` to handle `MigrationError` appropriately - [ ] Tests (pytest/Behave): Add unit tests covering corrupted JSON, permission errors, and missing file scenarios - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done - [ ] `FileNotFoundError` is silently ignored (no migration needed — acceptable) - [ ] `json.JSONDecodeError` and `PermissionError` are logged and raise `MigrationError` - [ ] No silent failure path remains in `migrate_project_data` - [ ] All new and existing tests pass - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-05 02:59:15 +00:00

freemo added a new dependency

2026-04-05 02:59:21 +00:00

#1020 [Epic] Database resource hierarchy restructuring

freemo commented

2026-04-05 03:00:22 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Confirmed
MoSCoW: Should Have

Valid finding verified during batch triage.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Confirmed - **MoSCoW**: Should Have Valid finding verified during batch triage. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-05 06:15:22 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3094

freemo referenced this issue

2026-04-05 07:21:47 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3094

freemo commented

2026-04-05 07:59:26 +00:00

Author

Owner

Starting implementation on branch fix/legacy-migrator-error-handling.

Plan:

Add MigrationError to core/exceptions.py (subclass of DatabaseError)
Refactor LegacyDataMigrator.migrate_project_data to separate FileNotFoundError (silent no-op) from json.JSONDecodeError and PermissionError (real failures → log + raise MigrationError)
Update project_service.py caller to handle MigrationError
Add BDD scenarios for corrupted JSON, permission errors, and missing file cases
Run nox quality gates

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-worker

Starting implementation on branch `fix/legacy-migrator-error-handling`. **Plan:** - Add `MigrationError` to `core/exceptions.py` (subclass of `DatabaseError`) - Refactor `LegacyDataMigrator.migrate_project_data` to separate `FileNotFoundError` (silent no-op) from `json.JSONDecodeError` and `PermissionError` (real failures → log + raise `MigrationError`) - Update `project_service.py` caller to handle `MigrationError` - Add BDD scenarios for corrupted JSON, permission errors, and missing file cases - Run nox quality gates --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-worker

freemo referenced this issue

2026-04-05 17:35:29 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377