# Database Migration Management with `agents db` ## Overview CleverAgents uses [Alembic](https://alembic.sqlalchemy.org/) to manage its SQLite database schema. The `agents db` command group exposes all migration operations directly from the command line, giving administrators full control over schema inspection, upgrades, and rollbacks — without needing to touch Alembic directly. This example walks through the complete database migration lifecycle: checking the current state of a fresh database, viewing the full migration history, applying all pending migrations, rolling back a single step, and upgrading again to the latest revision. ## Prerequisites - CleverAgents installed (`pip install cleveragents`) - Python 3.13 or higher ## What You'll Learn - How to **inspect the current migration state** of any database - How to **browse the full migration history** (41 revisions across the full schema evolution) - How to **apply pending migrations** to bring a database up to `head` - How to **roll back** one or more revisions safely - How to **target a specific revision** for precise upgrade or downgrade - How to use **`--format json`** to integrate migration status into scripts and CI/CD pipelines --- ## Part 1: Exploring the Commands ### Step 1: View available `db` subcommands ```bash $ agents db --help ``` **Output:** ``` Usage: agents db [OPTIONS] COMMAND [ARGS]... Database migration management (Alembic) ╭─ Options ────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰──────────────────────────────────────────────────────────────────────────────╯ ╭─ Commands ───────────────────────────────────────────────────────────────────╮ │ migrate Generate a new Alembic migration revision (autogenerate). │ │ upgrade Apply pending database migrations up to REVISION (default: head). │ │ downgrade Roll the database back to REVISION. │ │ current Show the current migration revision of the database. │ │ history Show the migration revision history. │ ╰──────────────────────────────────────────────────────────────────────────────╯ ``` **What's Happening:** The `agents db` group provides five subcommands covering the full migration lifecycle. All commands accept a `--format` flag (`rich`, `json`, `yaml`, `plain`, `table`) for flexible output. --- ## Part 2: Inspecting a Fresh Database ### Step 2: Check current revision on an uninitialized database Point CleverAgents at a brand-new SQLite file and check its migration state: ```bash $ export CLEVERAGENTS_DATABASE_URL="sqlite:////tmp/my-project.db" $ agents db current ``` **Output:** ``` Current revision : (none) Pending migrations: 41 - 001_initial_schema - 4b518923afb2 - c3d9b3d0cf3e - a5_001_actions_v3 - a5_002_lifecycle_plans - a5_003_spec_aligned_actions - a5_004_spec_aligned_plans - b1_001_resource_registry - b1_001_resource_links - a5_005_rebaseline_plan_phases - b0_001_projects - c1_001_tool_registry - a7_001_session_persistence - a6_001_automation_profiles - 71cd40eb661f - c0_001_skill_registry - a7_002_merge_heads - c3_001_actor_registry - a6_002_drop_automation_level - c0_002_merge_skill_registry - m4_001_concurrency_locks - d0_001_changeset_artifacts - d0_002_merge_changeset_and_locks - m4_001_decision_tables - m4_002_skill_flattened_tools - m6_001_checkpoint_metadata - c4_001_safety_profile_column - m6_002_merge_safety_and_checkpoint - m6_003_async_jobs_table - m7_001_repo_indexing_tables - m6_004_resource_type_inherits - m6_004_container_metadata_column - m6_005_profile_guards_json - m4_003_plan_env_columns - m8_001_align_plans_schema - m8_001_correction_attempts - m5_001_rename_profile_fields - m8_002_merge_profile_rename_and_corrections - m6_006_estimation_report_json - m9_001_session_name_column - m9_002_plan_resume_fields ``` **What's Happening:** On a fresh database, `current revision` is `(none)` because no migrations have been applied yet. All 41 revisions are listed as pending. The revision IDs follow a naming convention that encodes the feature milestone (e.g., `a5_001_actions_v3` = milestone A5, migration 001, feature "actions v3"). --- ## Part 3: Browsing Migration History ### Step 3: View the full migration history ```bash $ agents db history ``` **Output (most recent first):** ``` m9_001_session_name_column -> m9_002_plan_resume_fields (head), Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans. m6_006_estimation_report_json -> m9_001_session_name_column, Add name column to sessions table. m8_002_merge_profile_rename_and_corrections -> m6_006_estimation_report_json, Add estimation_report_json column to v3_plans. m5_001_rename_profile_fields, m8_001_correction_attempts, m8_001_align_plans_schema -> m8_002_merge_profile_rename_and_corrections (mergepoint), Merge profile-rename, correction-attempts, and plans-schema heads. m4_003_plan_env_columns -> m5_001_rename_profile_fields, Rename automation profile task-type threshold columns. m4_003_plan_env_columns -> m8_001_correction_attempts, Create correction_attempts table. m4_003_plan_env_columns -> m8_001_align_plans_schema, Align v3_plans table schema with specification DDL. m6_005_profile_guards_json -> m4_003_plan_env_columns (branchpoint), Add execution_environment and execution_env_priority columns to v3_plans. m6_004_container_metadata_column -> m6_005_profile_guards_json, Add safety_json and guards_json columns to automation_profiles table. m6_004_resource_type_inherits -> m6_004_container_metadata_column, Add container_metadata_json column to tool_invocations table. m7_001_repo_indexing_tables -> m6_004_resource_type_inherits, Add inherits column to resource_types for type inheritance (ADR-042). m6_003_async_jobs_table -> m7_001_repo_indexing_tables, Add repo_indexes and indexed_files tables for repository indexing. m6_002_merge_safety_and_checkpoint -> m6_003_async_jobs_table, Add async_jobs table for background job execution. c4_001_safety_profile_column, m6_001_checkpoint_metadata -> m6_002_merge_safety_and_checkpoint (mergepoint), Merge safety profile and checkpoint metadata heads. c0_002_merge_skill_registry -> c4_001_safety_profile_column, Add safety_profile_json column to actions table. m4_002_skill_flattened_tools -> m6_001_checkpoint_metadata, Create checkpoint_metadata table. m4_001_decision_tables -> m4_002_skill_flattened_tools, Add flattened tool persistence columns to skills table. d0_002_merge_changeset_and_locks -> m4_001_decision_tables, Add decision tree tables. d0_001_changeset_artifacts, m4_001_concurrency_locks -> d0_002_merge_changeset_and_locks (mergepoint), Merge changeset artifacts and concurrency locks heads. c0_002_merge_skill_registry -> d0_001_changeset_artifacts, Add changeset_entries and tool_invocations tables. c0_002_merge_skill_registry -> m4_001_concurrency_locks, Add concurrency locks table. a6_002_drop_automation_level, c0_001_skill_registry -> c0_002_merge_skill_registry (branchpoint) (mergepoint), Merge skill registry and actor/automation heads. c3_001_actor_registry -> a6_002_drop_automation_level, Drop automation_level column from v3_plans. a7_002_merge_heads -> c3_001_actor_registry, Add yaml_text, schema_version, compiled_metadata to actors table. 71cd40eb661f, a7_001_session_persistence -> a7_002_merge_heads (mergepoint), Merge session persistence and resource/automation heads. 71cd40eb661f -> c0_001_skill_registry, Add skill registry tables (skills, skill_items). a6_001_automation_profiles, b1_001_resource_links -> 71cd40eb661f (branchpoint) (mergepoint), Merge resource_links and automation_profiles branches c1_001_tool_registry -> a6_001_automation_profiles, Add automation_profiles table. c1_001_tool_registry -> a7_001_session_persistence, Add sessions and session_messages tables. b0_001_projects -> c1_001_tool_registry (branchpoint), Add tool registry tables (tools, tool_resource_bindings, validation_attachments). a5_005_rebaseline_plan_phases -> b0_001_projects, Add ns_projects and project_resource_links tables. b1_001_resource_registry -> a5_005_rebaseline_plan_phases, Rebaseline plan phase/state enums for Action + Apply terminal states. b1_001_resource_registry -> b1_001_resource_links, Add resource_links table for validated DAG parent-child links. a5_004_spec_aligned_plans -> b1_001_resource_registry (branchpoint), Add resource registry tables (resource_types, resources, resource_edges). a5_003_spec_aligned_actions -> a5_004_spec_aligned_plans, Create spec-aligned plans and related tables. a5_002_lifecycle_plans -> a5_003_spec_aligned_actions, Replace actions_v3 with spec-aligned actions table. a5_001_actions_v3 -> a5_002_lifecycle_plans, Add lifecycle_plans table for v3 plan lifecycle. c3d9b3d0cf3e -> a5_001_actions_v3, Add actions_v3 table for v3 plan lifecycle. 4b518923afb2 -> c3d9b3d0cf3e, Add actors table for actor registry. 001_initial_schema -> 4b518923afb2, add_debug_attempts_table -> 001_initial_schema, Initial database schema with all core tables. ``` **What's Happening:** Each line shows ` -> , `. The `(head)` marker identifies the latest revision. `(branchpoint)` and `(mergepoint)` labels show where the migration DAG diverges and converges — CleverAgents uses parallel migration branches to allow independent feature development without blocking each other. ### Step 4: Get history as JSON for scripting ```bash $ agents db history --format json ``` **Output (excerpt):** ```json { "command": "", "status": "ok", "exit_code": 0, "data": [ { "revision": "m9_002_plan_resume_fields", "down_revision": "m9_001_session_name_column", "description": "Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans." }, { "revision": "m9_001_session_name_column", "down_revision": "m6_006_estimation_report_json", "description": "Add name column to sessions table." }, { "revision": "m6_006_estimation_report_json", "down_revision": "m8_002_merge_profile_rename_and_corrections", "description": "Add estimation_report_json column to v3_plans." }, { "revision": "m8_002_merge_profile_rename_and_corrections", "down_revision": [ "m5_001_rename_profile_fields", "m8_001_correction_attempts", "m8_001_align_plans_schema" ], "description": "Merge profile-rename, correction-attempts, and plans-schema heads." } ], "timing": { "duration_ms": 0 }, "messages": [ {"level": "ok", "text": "ok"} ] } ``` **What's Happening:** The JSON output exposes the full DAG structure: `down_revision` is a list when a revision has multiple parents (a merge point). This is useful for tooling that needs to programmatically inspect the migration graph. --- ## Part 4: Applying Migrations ### Step 5: Upgrade to the latest revision (`head`) ```bash $ agents db upgrade ``` **Output (stderr shows Alembic progress, stdout shows result):** ``` INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Running upgrade -> 001_initial_schema, Initial database schema with all core tables. INFO [alembic.runtime.migration] Running upgrade 001_initial_schema -> 4b518923afb2, add_debug_attempts_table INFO [alembic.runtime.migration] Running upgrade 4b518923afb2 -> c3d9b3d0cf3e, Add actors table for actor registry. INFO [alembic.runtime.migration] Running upgrade c3d9b3d0cf3e -> a5_001_actions_v3, Add actions_v3 table for v3 plan lifecycle. INFO [alembic.runtime.migration] Running upgrade a5_001_actions_v3 -> a5_002_lifecycle_plans, Add lifecycle_plans table for v3 plan lifecycle. INFO [alembic.runtime.migration] Running upgrade a5_002_lifecycle_plans -> a5_003_spec_aligned_actions, Replace actions_v3 with spec-aligned actions table. INFO [alembic.runtime.migration] Running upgrade a5_003_spec_aligned_actions -> a5_004_spec_aligned_plans, Create spec-aligned plans and related tables. INFO [alembic.runtime.migration] Running upgrade a5_004_spec_aligned_plans -> b1_001_resource_registry, Add resource registry tables (resource_types, resources, resource_edges). INFO [alembic.runtime.migration] Running upgrade b1_001_resource_registry -> b1_001_resource_links, Add resource_links table for validated DAG parent-child links. INFO [alembic.runtime.migration] Running upgrade b1_001_resource_registry -> a5_005_rebaseline_plan_phases, Rebaseline plan phase/state enums for Action + Apply terminal states. INFO [alembic.runtime.migration] Running upgrade a5_005_rebaseline_plan_phases -> b0_001_projects, Add ns_projects and project_resource_links tables. INFO [alembic.runtime.migration] Running upgrade b0_001_projects -> c1_001_tool_registry, Add tool registry tables (tools, tool_resource_bindings, validation_attachments). INFO [alembic.runtime.migration] Running upgrade c1_001_tool_registry -> a7_001_session_persistence, Add sessions and session_messages tables. INFO [alembic.runtime.migration] Running upgrade c1_001_tool_registry -> a6_001_automation_profiles, Add automation_profiles table. INFO [alembic.runtime.migration] Running upgrade a6_001_automation_profiles, b1_001_resource_links -> 71cd40eb661f, Merge resource_links and automation_profiles branches INFO [alembic.runtime.migration] Running upgrade 71cd40eb661f -> c0_001_skill_registry, Add skill registry tables (skills, skill_items). INFO [alembic.runtime.migration] Running upgrade 71cd40eb661f, a7_001_session_persistence -> a7_002_merge_heads, Merge session persistence and resource/automation heads. INFO [alembic.runtime.migration] Running upgrade a7_002_merge_heads -> c3_001_actor_registry, Add yaml_text, schema_version, compiled_metadata to actors table. INFO [alembic.runtime.migration] Running upgrade c3_001_actor_registry -> a6_002_drop_automation_level, Drop automation_level column from v3_plans. INFO [alembic.runtime.migration] Running upgrade a6_002_drop_automation_level, c0_001_skill_registry -> c0_002_merge_skill_registry, Merge skill registry and actor/automation heads. INFO [alembic.runtime.migration] Running upgrade c0_002_merge_skill_registry -> m4_001_concurrency_locks, Add concurrency locks table. INFO [alembic.runtime.migration] Running upgrade c0_002_merge_skill_registry -> d0_001_changeset_artifacts, Add changeset_entries and tool_invocations tables. INFO [alembic.runtime.migration] Running upgrade d0_001_changeset_artifacts, m4_001_concurrency_locks -> d0_002_merge_changeset_and_locks, Merge changeset artifacts and concurrency locks heads. INFO [alembic.runtime.migration] Running upgrade d0_002_merge_changeset_and_locks -> m4_001_decision_tables, Add decision tree tables. INFO [alembic.runtime.migration] Running upgrade m4_001_decision_tables -> m4_002_skill_flattened_tools, Add flattened tool persistence columns to skills table. INFO [alembic.runtime.migration] Running upgrade m4_002_skill_flattened_tools -> m6_001_checkpoint_metadata, Create checkpoint_metadata table. INFO [alembic.runtime.migration] Running upgrade c0_002_merge_skill_registry -> c4_001_safety_profile_column, Add safety_profile_json column to actions table. INFO [alembic.runtime.migration] Running upgrade c4_001_safety_profile_column, m6_001_checkpoint_metadata -> m6_002_merge_safety_and_checkpoint, Merge safety profile and checkpoint metadata heads. INFO [alembic.runtime.migration] Running upgrade m6_002_merge_safety_and_checkpoint -> m6_003_async_jobs_table, Add async_jobs table for background job execution. INFO [alembic.runtime.migration] Running upgrade m6_003_async_jobs_table -> m7_001_repo_indexing_tables, Add repo_indexes and indexed_files tables for repository indexing. INFO [alembic.runtime.migration] Running upgrade m7_001_repo_indexing_tables -> m6_004_resource_type_inherits, Add inherits column to resource_types for type inheritance (ADR-042). INFO [alembic.runtime.migration] Running upgrade m6_004_resource_type_inherits -> m6_004_container_metadata_column, Add container_metadata_json column to tool_invocations table. INFO [alembic.runtime.migration] Running upgrade m6_004_container_metadata_column -> m6_005_profile_guards_json, Add safety_json and guards_json columns to automation_profiles table. INFO [alembic.runtime.migration] Running upgrade m6_005_profile_guards_json -> m4_003_plan_env_columns, Add execution_environment and execution_env_priority columns to v3_plans. INFO [alembic.runtime.migration] Running upgrade m4_003_plan_env_columns -> m8_001_align_plans_schema, Align v3_plans table schema with specification DDL. INFO [alembic.runtime.migration] Running upgrade m4_003_plan_env_columns -> m8_001_correction_attempts, Create correction_attempts table. INFO [alembic.runtime.migration] Running upgrade m4_003_plan_env_columns -> m5_001_rename_profile_fields, Rename automation profile task-type threshold columns. INFO [alembic.runtime.migration] Running upgrade m5_001_rename_profile_fields, m8_001_correction_attempts, m8_001_align_plans_schema -> m8_002_merge_profile_rename_and_corrections, Merge profile-rename, correction-attempts, and plans-schema heads. INFO [alembic.runtime.migration] Running upgrade m8_002_merge_profile_rename_and_corrections -> m6_006_estimation_report_json, Add estimation_report_json column to v3_plans. INFO [alembic.runtime.migration] Running upgrade m6_006_estimation_report_json -> m9_001_session_name_column, Add name column to sessions table. INFO [alembic.runtime.migration] Running upgrade m9_001_session_name_column -> m9_002_plan_resume_fields, Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans. Database upgraded to revision: m9_002_plan_resume_fields ``` **What's Happening:** Alembic applies all 41 pending migrations in topological order, respecting the DAG structure. Parallel branches (e.g., `a6_001_automation_profiles` and `b1_001_resource_links`) are applied before their merge point (`71cd40eb661f`). The final line confirms the database is now at the `head` revision: `m9_002_plan_resume_fields`. ### Step 6: Verify the database is fully up to date ```bash $ agents db current ``` **Output:** ``` Current revision : m9_002_plan_resume_fields Pending migrations: 0 ``` **What's Happening:** `Pending migrations: 0` confirms the database is at `head`. No further migrations need to be applied. ### Step 7: Get current state as JSON (for CI/CD integration) ```bash $ agents db current --format json ``` **Output:** ```json { "command": "", "status": "ok", "exit_code": 0, "data": { "current_revision": "m9_002_plan_resume_fields", "pending_count": 0, "pending_revisions": [] }, "timing": { "duration_ms": 0 }, "messages": [ {"level": "ok", "text": "ok"} ] } ``` **What's Happening:** The JSON output is ideal for CI/CD pipelines. You can parse `pending_count` to gate deployments: if `pending_count > 0`, run `agents db upgrade` before starting the application. --- ## Part 5: Rolling Back Migrations ### Step 8: Downgrade one step (relative rollback) ```bash $ agents db downgrade -- -1 ``` > **Note:** The `--` separator is required when passing negative revision > identifiers like `-1` so the shell does not interpret them as flags. **Output:** ``` INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Running downgrade m9_002_plan_resume_fields -> m9_001_session_name_column, Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans. Database downgraded to revision: m9_001_session_name_column ``` **What's Happening:** The `-1` relative identifier rolls back exactly one migration. The database is now at `m9_001_session_name_column`. The Alembic log shows the description of the migration that was reversed. ### Step 9: Check state after rollback ```bash $ agents db current ``` **Output:** ``` Current revision : m9_001_session_name_column Pending migrations: 1 - m9_002_plan_resume_fields ``` **What's Happening:** The database is now one revision behind `head`. The single pending migration (`m9_002_plan_resume_fields`) is the one that was just rolled back. --- ## Part 6: Targeted Upgrade and Downgrade ### Step 10: Downgrade to a specific revision by ID ```bash $ agents db downgrade m9_001_session_name_column ``` **Output:** ``` INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Running downgrade m9_002_plan_resume_fields -> m9_001_session_name_column, Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans. Database downgraded to revision: m9_001_session_name_column ``` ### Step 11: Upgrade to a specific revision by ID ```bash $ agents db upgrade m9_002_plan_resume_fields ``` **Output:** ``` INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Running upgrade m9_001_session_name_column -> m9_002_plan_resume_fields, Add reversion_count, last_completed_step, last_checkpoint_id columns to v3_plans. Database upgraded to revision: m9_002_plan_resume_fields ``` **What's Happening:** Both `upgrade` and `downgrade` accept explicit revision IDs. This is useful when you need to pin a database to a known-good state, for example during staged rollouts or when bisecting a regression. ### Step 12: Upgrade with JSON output (for scripting) ```bash $ agents db upgrade --format json 2>/dev/null ``` **Output (when already at head — idempotent):** ```json { "command": "", "status": "ok", "exit_code": 0, "data": { "status": "ok", "current_revision": "m9_002_plan_resume_fields" }, "timing": { "duration_ms": 0 }, "messages": [ {"level": "ok", "text": "ok"} ] } ``` **What's Happening:** `agents db upgrade` is **idempotent**: running it when the database is already at `head` is a no-op. The JSON output confirms `status: ok` and the current revision. The `2>/dev/null` suppresses the Alembic INFO log lines from stderr when you only want the structured result. --- ## Complete Interaction Log
Click to see the full session transcript ```bash # Set up a fresh test database $ export CLEVERAGENTS_DATABASE_URL="sqlite:////tmp/my-project.db" # 1. View available commands $ agents db --help Usage: agents db [OPTIONS] COMMAND [ARGS]... Database migration management (Alembic) ... # 2. Check state of fresh database $ agents db current Current revision : (none) Pending migrations: 41 - 001_initial_schema - 4b518923afb2 ... (41 revisions listed) # 3. Browse migration history $ agents db history m9_001_session_name_column -> m9_002_plan_resume_fields (head), ... ... -> 001_initial_schema, Initial database schema with all core tables. # 4. Apply all migrations $ agents db upgrade INFO [alembic.runtime.migration] Running upgrade -> 001_initial_schema, ... ... (41 migrations applied) Database upgraded to revision: m9_002_plan_resume_fields # 5. Confirm fully up to date $ agents db current Current revision : m9_002_plan_resume_fields Pending migrations: 0 # 6. Get current state as JSON $ agents db current --format json {"data": {"current_revision": "m9_002_plan_resume_fields", "pending_count": 0, ...}} # 7. Roll back one step $ agents db downgrade -- -1 INFO [alembic.runtime.migration] Running downgrade m9_002_plan_resume_fields -> m9_001_session_name_column, ... Database downgraded to revision: m9_001_session_name_column # 8. Check state after rollback $ agents db current Current revision : m9_001_session_name_column Pending migrations: 1 - m9_002_plan_resume_fields # 9. Upgrade back to head $ agents db upgrade head INFO [alembic.runtime.migration] Running upgrade m9_001_session_name_column -> m9_002_plan_resume_fields, ... Database upgraded to revision: m9_002_plan_resume_fields ```
--- ## Key Takeaways 1. **`agents db current`** is your go-to command for checking database health. It shows both the current revision and the count of pending migrations. 2. **`agents db history`** reveals the full migration DAG, including branchpoints and mergepoints. CleverAgents uses parallel migration branches to allow independent feature development. 3. **`agents db upgrade`** (no arguments) always upgrades to `head`. It is **idempotent** — safe to run on every application startup. 4. **`agents db downgrade -- -1`** rolls back exactly one step. Use `--` to separate the negative relative identifier from CLI flags. 5. **All commands support `--format json`** for scripting and CI/CD integration. Parse `pending_count` to gate deployments. 6. **The `CLEVERAGENTS_DATABASE_URL` environment variable** overrides the default database path. This is the recommended way to point commands at a specific database file. --- ## Try It Yourself ```bash # Initialize a fresh workspace and inspect its migration state mkdir ~/my-agents-project && cd ~/my-agents-project agents project init my-agents-project --yes # The project init automatically runs all migrations. # Verify the database is at head: agents db current # Browse the schema evolution history: agents db history # Simulate a rollback scenario: agents db downgrade -- -1 agents db current # Shows 1 pending migration agents db upgrade # Re-applies the rolled-back migration agents db current # Back to head, 0 pending ``` --- *This example was automatically generated and verified by the CleverAgents UAT system.* *Feature area: Database migration management | Test cycle: 1* --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester