fix(database): include alembic files in package distribution

Move alembic configuration and migration files from repository root into the
Python package structure to ensure they are included in the wheel distribution.

This fix resolves the FileNotFoundError when running `agents init` in Docker
containers or any environment using the built wheel distribution.

Changes:
- Move alembic/ directory from repo root to
  src/cleveragents/infrastructure/database/migrations/
- Move alembic.ini to the same new location and update script_location setting
- Update MigrationRunner._find_alembic_ini() to search from the new canonical
  location within the package
- Update create_template_db.py to point to the new alembic.ini location
- Update documentation references to reflect new migration file locations
- Create __init__.py for migrations package

- The env.py file is imported when running tests that verify all modules can be
  imported without errors. However, context.config is only available when alembic
  is actually running migrations, not during normal module imports. This caused
  an AttributeError when the test tried to import the migrations.env module.

- Fix by using getattr() with a default value to safely access context.config,
  and guard all code that uses config with None checks. This allows the module
  to be safely imported while still functioning correctly during migrations.

Testing:
- Verified MigrationRunner can locate alembic.ini in new location
- Tested agents init succeeds in creating project with database
- Template database creation works correctly
- All migration tests should pass without changes

Alembic files now follow standard Python packaging conventions, making them
automatically included in wheel distributions without special configuration.

ISSUES CLOSED: #4180

CHANGELOG.md

@@ -16,6 +16,17 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   `"3"`, `"3.0"`, `"3.0.0"`). Configs with `type: null` are not treated as v3.
   Invalid v3 actors are rejected with clear error messages before registration.
 
+- **Alembic Files Missing from Wheel Distribution** (#4180): Alembic configuration
+  (`alembic.ini`) and migration files are now part of the Python package structure
+  at `src/cleveragents/infrastructure/database/migrations/`. Previously, when
+  `agents init` was run in Docker containers or any wheel-based installation,
+  `FileNotFoundError` was raised because alembic files were stored at the
+  repository root and excluded from the wheel distribution. Now alembic files
+  follow standard Python packaging conventions and are automatically included.
+  `MigrationRunner._find_alembic_ini()` has been updated to search the new package
+  location as the primary anchor point. This fix enables `agents init` to work
+  correctly in all deployment modes: Docker containers, local pip installs
+  (wheel or editable), and development environments.
 - **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
   `features/environment.py` now emits its non-assertion exception guard warning to
   both the structured logger and `stderr` via a new `_warning_with_stderr` helper.

docs/development/review_playbook.md

@@ -14,7 +14,7 @@ blocking findings.
 | Area | Reason |
 | ---- | ------ |
 | Domain model changes (`src/cleveragents/domain/`) | Core business logic; regressions break everything |
-| Database migrations (`alembic/`) | Irreversible in production; data loss risk |
+| Database migrations (`src/cleveragents/infrastructure/database/migrations/`) | Irreversible in production; data loss risk |
 | Security-sensitive code (auth, secrets, server stubs) | Direct attack surface |
 | CLI entry points (`src/cleveragents/cli/`) | User-facing; breaking changes affect all users |
 | Public API contracts (schemas, configs) | Backward-compatibility obligations |
@@ -166,7 +166,7 @@ within the SLA window, assign the secondary.
 | CLI tests | `features/*cli*`, `robot/*cli*` | Brent | Rui | CLI review |
 | Behave tests & fixtures | `features/`, `features/steps/` | Brent | Rui | Test review |
 | Robot tests & helpers | `robot/` | Brent | Rui | Test review |
-| DB migrations | `alembic/` | Hamza | Luis | DB migration review |
+| DB migrations | `src/cleveragents/infrastructure/database/migrations/` | Hamza | Luis | DB migration review |
 | Actor/Skill/Tool configs | `src/cleveragents/*/actor*`, `src/cleveragents/*/skill*`, `src/cleveragents/*/tool*` | Aditya | Jeff | Architecture review |
 | MCP adapter | `src/cleveragents/infrastructure/mcp/` | Aditya | Jeff | Architecture review |
 | Security changes | `**/security*`, `**/auth*`, `**/secrets*` | Jeff | Brent | Security review |

docs/reference/database_schema.md

@@ -379,4 +379,4 @@ check constraint.
 
 - **Revision**: `c1_001_tool_registry`
 - **Depends on**: `b0_001_projects`
-- **File**: `alembic/versions/c1_001_tool_registry.py`
+- **File**: `src/cleveragents/infrastructure/database/migrations/versions/c1_001_tool_registry.py`

docs/reference/resources.md

@@ -206,6 +206,6 @@ schema.
 - Domain models: `src/cleveragents/domain/models/core/resource.py`,
   `resource_type.py`
 - Service: `src/cleveragents/application/services/resource_registry_service.py`
-- Migration: `alembic/versions/b1_001_resource_registry_tables.py`
+- Migration: `src/cleveragents/infrastructure/database/migrations/versions/b1_001_resource_registry_tables.py`
 - CLI: `src/cleveragents/cli/commands/resource.py`
 - Specification: `docs/specification.md` — Resource Registry sections

features/db_migration_lifecycle.feature

@@ -48,3 +48,18 @@ Feature: Database migration lifecycle
   Scenario: init_database creates a valid schema
     When I call init_database with an in-memory URL
     Then the resulting engine should have a valid schema
+
+  @issue_4180 @alembic_packaging
+  Scenario: MigrationRunner locates alembic.ini from packaged location
+    Given the MigrationRunner class is available
+    When I call _find_alembic_ini() to locate the configuration file
+    Then it should find alembic.ini at the package location
+    And the alembic.ini should be readable and valid
+
+  @issue_4180 @alembic_packaging
+  Scenario: Database initialization succeeds with packaged alembic files
+    Given a fresh in-memory database for migration lifecycle testing
+    When I run all migrations forward to head
+    Then the database should have all expected tables
+    And the database should be stamped with alembic_version
+    And the database revision should be at head

features/steps/migration_runner_steps.py

@@ -657,3 +657,44 @@ def step_when_prompt_errors(context) -> None:
 def step_then_prompt_auto_after_error(context) -> None:
     assert context.prompt_result is True
     assert len(getattr(context, "prompt_error_confirm_calls", [])) == 1
+
+
+@given("the MigrationRunner class is available")
+def step_given_migration_runner_available(context) -> None:
+    """Make MigrationRunner available in context for testing alembic.ini location."""
+    context.migration_runner_class = MigrationRunner
+
+
+@when("I call _find_alembic_ini() to locate the configuration file")
+def step_when_find_alembic_ini(context) -> None:
+    """Call the _find_alembic_ini method to locate alembic.ini from package."""
+    try:
+        context.alembic_ini_path = MigrationRunner._find_alembic_ini()
+        context.find_alembic_error = None
+    except FileNotFoundError as exc:
+        context.find_alembic_error = exc
+        context.alembic_ini_path = None
+
+
+@then("it should find alembic.ini at the package location")
+def step_then_alembic_ini_found_at_package(context) -> None:
+    """Verify alembic.ini is found in the package location."""
+    assert context.find_alembic_error is None, (
+        f"Expected to find alembic.ini but got error: {context.find_alembic_error}"
+    )
+    assert context.alembic_ini_path is not None
+    assert context.alembic_ini_path.exists(), (
+        f"alembic.ini path {context.alembic_ini_path} does not exist"
+    )
+    assert "migrations" in str(context.alembic_ini_path).lower(), (
+        f"Expected alembic.ini in migrations directory, got {context.alembic_ini_path}"
+    )
+
+
+@then("the alembic.ini should be readable and valid")
+def step_then_alembic_ini_readable(context) -> None:
+    """Verify alembic.ini is readable and contains expected configuration."""
+    assert context.alembic_ini_path.is_file()
+    content = context.alembic_ini_path.read_text(encoding="utf-8")
+    assert "[loggers]" in content, "alembic.ini missing loggers section"
+    assert "script_location" in content, "alembic.ini missing script_location"

pyproject.toml

@@ -108,7 +108,12 @@ agents = "cleveragents.cli:main"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/cleveragents"]
-include = ["src/cleveragents/py.typed"]
+include = [
+    "src/cleveragents/py.typed",
+    "src/cleveragents/infrastructure/database/migrations/alembic.ini",
+    "src/cleveragents/infrastructure/database/migrations/script.py.mako",
+    "src/cleveragents/infrastructure/database/migrations/README",
+]
 
 [tool.ruff]
 line-length = 88

robot/cli_init_fresh_environment.robot

@@ -0,0 +1,78 @@
+*** Settings ***
+Documentation     Integration test for agents init in fresh environment (issue #4180).
+...               Verifies that agents init properly initializes the database and
+...               creates all required directories in a clean, containerized context.
+Resource          ${CURDIR}/common.resource
+Library           Process
+Library           OperatingSystem
+Library           String
+Suite Setup       Setup Test Environment
+Suite Teardown    Cleanup Test Environment
+
+*** Test Cases ***
+Init In Fresh Environment Exits Successfully
+    [Documentation]    agents init should complete with exit code 0 in a fresh environment
+    [Tags]    issue_4180    fresh_environment
+    ${tmpdir}=    Evaluate    __import__('tempfile').mkdtemp(prefix='init_fresh_')
+    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    init    --yes
+    ...    timeout=120s    on_timeout=kill    cwd=${tmpdir}
+    Should Be Equal As Integers    ${result.rc}    0
+    ...    msg=Expected exit code 0 in fresh environment, got ${result.rc}. stderr: ${result.stderr}
+    [Teardown]    Remove Directory    ${tmpdir}    recursive=True
+
+Init In Fresh Environment Initializes Database
+    [Documentation]    agents init should properly initialize the database in fresh environment
+    [Tags]    issue_4180    fresh_environment
+    ${tmpdir}=    Evaluate    __import__('tempfile').mkdtemp(prefix='init_fresh_db_')
+    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    init    --yes
+    ...    timeout=120s    on_timeout=kill    cwd=${tmpdir}
+    Should Be Equal As Integers    ${result.rc}    0
+    ...    msg=Init should succeed in fresh environment
+    Should Contain    ${result.stdout}    Database:
+    ...    msg=Output should indicate database was initialized
+    [Teardown]    Remove Directory    ${tmpdir}    recursive=True
+
+Init In Fresh Environment Creates Required Directories
+    [Documentation]    agents init should create all required directories for packaged/containerized context
+    [Tags]    issue_4180    fresh_environment
+    ${tmpdir}=    Evaluate    __import__('tempfile').mkdtemp(prefix='init_fresh_dirs_')
+    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    init    --yes
+    ...    timeout=120s    on_timeout=kill    cwd=${tmpdir}
+    Should Be Equal As Integers    ${result.rc}    0
+    Should Contain    ${result.stdout}    Directories:
+    ...    msg=Output should indicate directories were created
+    # Verify required directories exist
+    ${config_dir}=    Evaluate    __import__('os').path.join('${tmpdir}', '.cleveragents')
+    Directory Should Exist    ${config_dir}
+    ...    msg=.cleveragents config directory should exist
+    [Teardown]    Remove Directory    ${tmpdir}    recursive=True
+
+Init In Fresh Environment Reports All Init Output Fields
+    [Documentation]    agents init should produce all spec-required output fields for fresh environment
+    [Tags]    issue_4180    fresh_environment
+    ${tmpdir}=    Evaluate    __import__('tempfile').mkdtemp(prefix='init_fresh_out_')
+    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    init    --yes
+    ...    timeout=120s    on_timeout=kill    cwd=${tmpdir}
+    Should Be Equal As Integers    ${result.rc}    0
+    # All spec-required output fields must be present
+    Should Contain    ${result.stdout}    Data Dir:
+    ...    msg=Output should contain Data Dir field per spec
+    Should Contain    ${result.stdout}    Config:
+    ...    msg=Output should contain Config field per spec
+    Should Contain    ${result.stdout}    Database:
+    ...    msg=Output should contain Database field per spec
+    Should Contain    ${result.stdout}    Directories:
+    ...    msg=Output should contain Directories field per spec
+    [Teardown]    Remove Directory    ${tmpdir}    recursive=True
+
+Init Handles Packaged Alembic Files Correctly
+    [Documentation]    agents init should find and use alembic files from packaged location in fresh environment
+    [Tags]    issue_4180    fresh_environment    alembic_packaging
+    ${tmpdir}=    Evaluate    __import__('tempfile').mkdtemp(prefix='init_fresh_alembic_')
+    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    init    --yes
+    ...    timeout=120s    on_timeout=kill    cwd=${tmpdir}
+    Should Be Equal As Integers    ${result.rc}    0
+    ...    msg=Init should succeed with packaged alembic files. stderr: ${result.stderr}
+    Should Contain    ${result.stdout}    Database:
+    ...    msg=Database should be initialized from packaged alembic files
+    [Teardown]    Remove Directory    ${tmpdir}    recursive=True

scripts/create_template_db.py

@@ -48,7 +48,15 @@ def create_template(output_path: str = "build/.template-migrated.db") -> None:
 
     # Stamp alembic_version with the head revision so MigrationRunner
     # sees the database as fully migrated (no pending migrations).
-    alembic_ini = Path(__file__).resolve().parent.parent / "alembic.ini"
+    alembic_ini = (
+        Path(__file__).resolve().parent.parent
+        / "src"
+        / "cleveragents"
+        / "infrastructure"
+        / "database"
+        / "migrations"
+        / "alembic.ini"
+    )
     cfg = Config(str(alembic_ini))
     sd = ScriptDirectory.from_config(cfg)
     head = sd.get_current_head()

src/cleveragents/infrastructure/database/migration_runner.py

@@ -35,57 +35,61 @@ class MigrationRunner:
     def _find_alembic_ini() -> Path:
         """Locate the ``alembic.ini`` configuration file.
 
-        Searches for ``alembic.ini`` using two strategies:
+        Searches for ``alembic.ini`` using multiple strategies:
 
-        1. **Package-relative**: Walk upward from the top-level
-           ``cleveragents`` package directory.  This is the most reliable
-           anchor because ``cleveragents.__file__`` is always resolvable
-           regardless of the current working directory.
-        2. **Module-relative**: Walk upward from *this* source file as a
-           fallback for non-standard layouts.
+        1. **Package-relative direct path**: Look for alembic.ini in the
+           standard location at
+           ``cleveragents/infrastructure/database/migrations/alembic.ini``.
+           This is the canonical location in the installed package.
+        2. **Upward search from module**: Walk upward from *this* source
+           file as a fallback for non-standard layouts.
 
-        Only the presence of ``alembic.ini`` is checked (not an
-        ``alembic/`` sibling directory) because the ini file's own
-        ``script_location`` setting already tells Alembic where to find
-        the migration scripts.
+        The first strategy is the primary path used in all standard
+        deployment modes (wheel installs, editable installs, Docker).
 
         Raises:
             FileNotFoundError: If ``alembic.ini`` cannot be located.
         """
-        search_roots: list[Path] = []
-
-        # Strategy 1 - start from the cleveragents package root.
-        # In an editable install this is e.g. /app/src/cleveragents/;
-        # walking up two levels reaches /app/ where alembic.ini lives.
+        # Strategy 1 - Direct path from cleveragents package root.
+        # In all installations (editable, wheel, Docker), alembic.ini is at:
+        # <cleveragents_root>/infrastructure/database/migrations/alembic.ini
         try:
             import cleveragents as _pkg
 
             pkg_init = getattr(_pkg, "__file__", None)
             if pkg_init is not None:
-                search_roots.append(Path(pkg_init).resolve().parent)
+                pkg_root = Path(pkg_init).resolve().parent
+                candidate = (
+                    pkg_root
+                    / "infrastructure"
+                    / "database"
+                    / "migrations"
+                    / "alembic.ini"
+                )
+                if candidate.exists():
+                    return candidate
         except Exception:  # pragma: no cover - defensive
             pass
 
-        # Strategy 2 - start from *this* module's directory.
-        search_roots.append(Path(__file__).resolve().parent)
+        # Strategy 2 - Fallback: Walk upward from *this* module's directory.
+        # This handles non-standard layouts where the package structure
+        # differs.
+        current = Path(__file__).resolve().parent
+        for _ in range(10):
+            candidate = current / "alembic.ini"
+            if candidate.exists():
+                return candidate
+            parent = current.parent
+            if parent == current:
+                # Reached filesystem root
+                break
+            current = parent
 
-        for start in search_roots:
-            current = start
-            # Walk up at most 10 levels to avoid runaway traversal
-            for _ in range(10):
-                candidate = current / "alembic.ini"
-                if candidate.exists():
-                    return candidate
-                parent = current.parent
-                if parent == current:
-                    # Reached filesystem root
-                    break
-                current = parent
-
-        searched = ", ".join(str(s) for s in search_roots)
         raise FileNotFoundError(
             "Alembic configuration file (alembic.ini) not found. "
-            f"Searched upward from: {searched}"
+            "Expected location: "
+            "<cleveragents_package_root>/infrastructure/database/"
+            "migrations/alembic.ini"
         )
 
     @property

src/cleveragents/infrastructure/database/migrations/alembic.ini

@@ -5,7 +5,7 @@
 # this is typically a path given in POSIX (e.g. forward slashes)
 # format, relative to the token %(here)s which refers to the location of this
 # ini file
-script_location = %(here)s/alembic
+script_location = %(here)s
 
 # template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
 # Uncomment the line below if you want the files to be prepended with date and time

src/cleveragents/infrastructure/database/migrations/env.py

@@ -1,3 +1,4 @@
+import logging
 import os
 from logging.config import fileConfig
 from pathlib import Path
@@ -8,30 +9,34 @@ from sqlalchemy import engine_from_config, pool
 # Import our models
 from cleveragents.infrastructure.database.models import Base
 
-# this is the Alembic Config object, which provides
-# access to the values within the .ini file in use.
-config = context.config
-
-# Interpret the config file for Python logging.
-# This line sets up loggers basically.
-# NOTE: disable_existing_loggers=False prevents fileConfig from disabling
-# application loggers that were created before this point, which causes
-# test failures when log capture handlers are attached to those loggers.
-if config.config_file_name is not None:
-    fileConfig(config.config_file_name, disable_existing_loggers=False)
+logger = logging.getLogger(__name__)
 
 # add your model's MetaData object here
 # for 'autogenerate' support
 target_metadata = Base.metadata
 
-# Override the database URL from environment or use default
-# This allows flexible configuration based on deployment
-# Default matches Settings.database_url: ~/.cleveragents/cleveragents.db
-database_url = os.getenv(
-    "CLEVERAGENTS_DATABASE_URL",
-    f"sqlite:///{Path.home() / '.cleveragents' / 'cleveragents.db'}",
-)
-config.set_main_option("sqlalchemy.url", database_url)
+# this is the Alembic Config object, which provides
+# access to the values within the .ini file in use.
+# Only access context.config when alembic is actually running
+config = getattr(context, "config", None)
+
+# Interpret the config file for Python logging and set database URL
+# This is only needed when migrations are actually running
+if config is not None:
+    # NOTE: disable_existing_loggers=False prevents fileConfig from disabling
+    # application loggers that were created before this point, which causes
+    # test failures when log capture handlers are attached to those loggers.
+    if config.config_file_name is not None:
+        fileConfig(config.config_file_name, disable_existing_loggers=False)
+
+    # Override the database URL from environment or use default
+    # This allows flexible configuration based on deployment
+    # Default matches Settings.database_url: ~/.cleveragents/cleveragents.db
+    database_url = os.getenv(
+        "CLEVERAGENTS_DATABASE_URL",
+        f"sqlite:///{Path.home() / '.cleveragents' / 'cleveragents.db'}",
+    )
+    config.set_main_option("sqlalchemy.url", database_url)
 
 # other values from the config, defined by the needs of env.py,
 # can be acquired:
@@ -51,6 +56,9 @@ def run_migrations_offline() -> None:
     script output.
 
     """
+    if config is None:
+        msg = "Config is required for offline migrations"
+        raise RuntimeError(msg)
     url = config.get_main_option("sqlalchemy.url")
     context.configure(
         url=url,
@@ -70,6 +78,9 @@ def run_migrations_online() -> None:
     and associate a connection with the context.
 
     """
+    if config is None:
+        msg = "Config is required for online migrations"
+        raise RuntimeError(msg)
     # Check if a connection/engine is already provided (e.g., for in-memory SQLite)
     # This allows tests to reuse the same in-memory database
     connectable = config.attributes.get("connection", None)
@@ -96,7 +107,17 @@ def run_migrations_online() -> None:
             context.run_migrations()
 
 
-if context.is_offline_mode():
-    run_migrations_offline()
+# Only run migrations if alembic context is available
+# (i.e., when alembic is actually running)
+if config is not None:
+    if context.is_offline_mode():
+        run_migrations_offline()
+    else:
+        run_migrations_online()
 else:
-    run_migrations_online()
+    logger.warning(
+        "Alembic context (context.config) not available. "
+        "This typically means env.py was imported as a regular module "
+        "rather than being invoked by Alembic. "
+        "Database migrations will not run unless Alembic is properly configured."
+    )