HAL9000
8dde6c81ef
Add PostgreSQL support as the server-mode storage backend alongside existing SQLite for local mode. Verify all ORM models are dialect- agnostic, configure connection pooling for multi-user access, add Docker Compose for local PG development, and wire database URL selection based on deployment mode. Changes: - Add psycopg2-binary dependency to pyproject.toml - Add server_mode, db_pool_size, db_max_overflow, db_pool_recycle settings to Settings with environment variable support - Add resolve_database_url() and is_postgresql() to Settings for mode-aware database URL resolution - Configure UnitOfWork engine creation with pool_size, max_overflow, pool_recycle, and pool_pre_ping for PostgreSQL connections - Update MigrationRunner to handle both SQLite and PostgreSQL backends - Add compare_type=True to Alembic env.py for dialect-aware migrations - Add docker-compose.yml with PostgreSQL 16-alpine for local development - Add Behave BDD feature (14 scenarios) covering settings, pool config, engine creation, ORM dialect compatibility, and migration runner - Add Robot Framework integration tests (12 test cases) for the abstraction layer with requires_postgresql tag for live PG tests Fixes: - Restore require_confirmation parameter to UnitOfWork.__init__ - Add argument validation to UnitOfWork.__init__ (fail-fast) - Add explicit PostgreSQL isolation_level (READ COMMITTED) - Add dispose() method and context manager support to UnitOfWork - Fix MigrationRunner.get_current_revision() to dispose engine - Fix Settings.is_postgresql() to handle ValueError gracefully ISSUES CLOSED: #878
167 lines
8.5 KiB
Plaintext
167 lines
8.5 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for PostgreSQL storage backend abstraction layer.
|
|
... Tests the dialect-agnostic ORM models, connection configuration,
|
|
... and database URL selection logic using SQLite as the test backend.
|
|
... Tests tagged @requires_postgresql are excluded from default CI runs.
|
|
Library OperatingSystem
|
|
Library String
|
|
Library Collections
|
|
Library Process
|
|
Library indentation_library.py
|
|
Resource ${CURDIR}/common.resource
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
|
|
*** Test Cases ***
|
|
|
|
Settings Resolve Database URL Local Mode
|
|
[Documentation] Verify resolve_database_url returns SQLite in local mode
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import os
|
|
... os.environ.pop('CLEVERAGENTS_SERVER_MODE', None)
|
|
... os.environ.pop('CLEVERAGENTS_DATABASE_URL', None)
|
|
... from cleveragents.config.settings import Settings
|
|
... s = Settings()
|
|
... print(s.resolve_database_url())
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Start With ${result.stdout.strip()} sqlite
|
|
|
|
Settings Resolve Database URL Server Mode
|
|
[Documentation] Verify resolve_database_url returns PostgreSQL when server_mode is on
|
|
... and CLEVERAGENTS_DATABASE_URL points to a PostgreSQL instance.
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import os
|
|
... os.environ['CLEVERAGENTS_SERVER_MODE'] = 'true'
|
|
... os.environ['CLEVERAGENTS_DATABASE_URL'] = 'postgresql://localhost/cleveragents'
|
|
... from cleveragents.config.settings import Settings
|
|
... s = Settings()
|
|
... print(s.resolve_database_url())
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Start With ${result.stdout.strip()} postgresql
|
|
|
|
Settings Is PostgreSQL Returns True For PG URL
|
|
[Documentation] Verify is_postgresql detects PostgreSQL URLs
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import os
|
|
... os.environ['CLEVERAGENTS_SERVER_MODE'] = 'true'
|
|
... os.environ['CLEVERAGENTS_DATABASE_URL'] = 'postgresql://localhost/db'
|
|
... from cleveragents.config.settings import Settings
|
|
... s = Settings()
|
|
... print(s.is_postgresql())
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Be Equal As Strings ${result.stdout.strip()} True
|
|
|
|
Settings Is PostgreSQL Returns False For SQLite URL
|
|
[Documentation] Verify is_postgresql returns False for SQLite URLs
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import os
|
|
... os.environ.pop('CLEVERAGENTS_SERVER_MODE', None)
|
|
... os.environ.pop('CLEVERAGENTS_DATABASE_URL', None)
|
|
... from cleveragents.config.settings import Settings
|
|
... s = Settings()
|
|
... print(s.is_postgresql())
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Be Equal As Strings ${result.stdout.strip()} False
|
|
|
|
UnitOfWork Creates SQLite Engine With Pool Parameters
|
|
[Documentation] UnitOfWork stores pool parameters and creates an SQLite engine
|
|
${result}= Run Process ${PYTHON} -c
|
|
... from cleveragents.infrastructure.database.unit_of_work import UnitOfWork; u \= UnitOfWork('sqlite:///:memory:', prompt_for_migration\=lambda m: True, pool_size\=8, max_overflow\=15, pool_recycle\=900); print(u._pool_size, u._max_overflow, u._pool_recycle, u.engine.dialect.name)
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout.strip()} 8 15 900 sqlite
|
|
|
|
ORM Models Use Dialect-Agnostic Types
|
|
[Documentation] All ORM column types are portable across SQLite and PostgreSQL
|
|
${script}= Catenate SEPARATOR=\n
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... PORTABLE = {'VARCHAR','STRING','TEXT','INTEGER','BIGINTEGER','SMALLINTEGER',
|
|
... 'FLOAT','NUMERIC','BOOLEAN','DATE','DATETIME','TIME','TIMESTAMP','JSON',
|
|
... 'BLOB','LARGEBINARY','ENUM'}
|
|
... for t in Base.metadata.tables.values():
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}for c in t.columns:
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}tn = type(c.type).__name__.upper()
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}if tn not in PORTABLE and not any(b in tn for b in PORTABLE):
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}raise AssertionError(f'{t.name}.{c.name} has non-portable type {tn}')
|
|
... print('ALL_PORTABLE')
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout.strip()} ALL_PORTABLE
|
|
|
|
ORM Models Source Contains No SQLite Specific Code
|
|
[Documentation] models.py should not import sqlite3 or use PRAGMAs
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import inspect
|
|
... from cleveragents.infrastructure.database import models
|
|
... src = inspect.getsource(models)
|
|
... for forbidden in ['sqlite3', 'PRAGMA']:
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}assert forbidden not in src, f'Found {forbidden}'
|
|
... print('CLEAN')
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout.strip()} CLEAN
|
|
|
|
Table Names Are Valid PostgreSQL Identifiers
|
|
[Documentation] All table names are lowercase SQL identifiers
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import re
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... pat = re.compile(r'^[a-z_][a-z0-9_]*$')
|
|
... for t in Base.metadata.tables.values():
|
|
... ${SPACE}${SPACE}${SPACE}${SPACE}assert pat.match(t.name), f'Invalid table: {t.name}'
|
|
... print('VALID')
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout.strip()} VALID
|
|
|
|
Docker Compose File Exists For PostgreSQL Development
|
|
[Documentation] docker-compose.yml exists at project root
|
|
File Should Exist ${CURDIR}/../docker-compose.yml
|
|
|
|
Pool Defaults Are Multi-User Appropriate
|
|
[Documentation] Default pool settings: pool_size=5, max_overflow=10, pool_recycle=1800
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import os
|
|
... os.environ['CLEVERAGENTS_SERVER_MODE'] = 'true'
|
|
... os.environ.pop('CLEVERAGENTS_DATABASE_URL', None)
|
|
... from cleveragents.config.settings import Settings
|
|
... s = Settings()
|
|
... print(s.db_pool_size, s.db_max_overflow, s.db_pool_recycle)
|
|
${result}= Run Process ${PYTHON} -c ${script}
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Be Equal As Strings ${result.stdout.strip()} 5 10 1800
|
|
|
|
Migration Runner Accepts PostgreSQL URL
|
|
[Documentation] MigrationRunner can be constructed with a PostgreSQL URL
|
|
${result}= Run Process ${PYTHON} -c
|
|
... from cleveragents.infrastructure.database.migration_runner import MigrationRunner; r \= MigrationRunner('postgresql://localhost/test'); print(r.database_url)
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Start With ${result.stdout.strip()} postgresql
|
|
|
|
PostgreSQL Live Connection Test
|
|
[Documentation] Connect to a real PostgreSQL instance and run migrations.
|
|
... Requires a running PostgreSQL server and CLEVERAGENTS_DATABASE_URL
|
|
... pointing to it. Excluded from default CI runs.
|
|
[Tags] requires_postgresql
|
|
${pg_url}= Get Environment Variable CLEVERAGENTS_DATABASE_URL default=SKIP
|
|
Skip If '${pg_url}' == 'SKIP' PostgreSQL not available
|
|
${result}= Run Process ${PYTHON} -c
|
|
... from cleveragents.infrastructure.database.migration_runner import MigrationRunner; r \= MigrationRunner('${pg_url}'); r.init_or_upgrade(); print('OK')
|
|
... cwd=${CURDIR}/..
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout.strip()} OK
|