Files
cleveragents-core/features/plans_table_schema_alignment.feature
CoreRasurae a4a6b061a6
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 18s
CI / lint (pull_request) Successful in 20s
CI / helm (pull_request) Successful in 22s
CI / typecheck (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 3m45s
CI / integration_tests (pull_request) Successful in 3m59s
CI / security (pull_request) Successful in 4m8s
CI / unit_tests (pull_request) Successful in 5m44s
CI / docker (pull_request) Successful in 1m32s
CI / coverage (pull_request) Successful in 11m50s
CI / e2e_tests (pull_request) Successful in 20m28s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 17s
CI / build (push) Successful in 18s
CI / helm (push) Successful in 22s
CI / quality (push) Successful in 31s
CI / typecheck (push) Successful in 1m1s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m57s
CI / security (push) Successful in 4m3s
CI / docker (push) Successful in 1m21s
CI / integration_tests (push) Successful in 7m11s
CI / coverage (push) Successful in 11m56s
CI / e2e_tests (push) Successful in 20m56s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 28m22s
CI / benchmark-regression (pull_request) Successful in 55m20s
fix(db): align v3_plans schema with specification DDL
Aligned v3_plans table with specification DDL:

1. Added effective_profile_snapshot column (TEXT NOT NULL) for
   storing frozen JSON snapshot of automation profile at plan
   creation time.  Added Pydantic field_validator ensuring the
   value is well-formed JSON.  Validator catches RecursionError
   for deeply nested JSON, consistent with automation_profile
   deserialization hardening.  Validator error message uses
   length-only to avoid potential information disclosure.
   Documented that the default "{}" exists for backward
   compatibility; new plans should explicitly set the snapshot.

2. Made root_plan_id NOT NULL — root plans self-reference their
   own plan_id, child plans reference the root ancestor.  Added
   explicit ondelete="RESTRICT" FK policy for consistency with
   other FKs in the model.  Documented known FK policy drift
   between ORM model (RESTRICT) and migrated databases (retained
   SET NULL) in the migration; data integrity is preserved by
   the NOT NULL constraint regardless.  Moved root_plan_id
   self-reference resolution into a PlanIdentity model_validator
   so the domain model is consistent with the DB NOT NULL
   constraint before and after persistence (previously the
   resolution only happened in from_domain(), creating an
   asymmetry where root_plan_id was None in-memory but non-null
   after round-tripping through the database).

3. Made automation_profile NOT NULL with default "balanced".

4. Documented intentional deviation: phase default is "action"
   (code) vs "strategize" (spec) because the Action phase was
   added as a pre-Strategize setup step.

5. Created Alembic migration with backfill logic for existing
   rows.  Root-ancestor backfill uses level-by-level propagation
   with a parent-readiness guard to correctly resolve plans at
   arbitrary hierarchy depth (3+ levels).  Added safety bound
   (max 100 iterations) with logged error on exhaustion to guard
   against cycles in parent_plan_id.  Merged batch_alter_table
   operations to avoid redundant full-table copies in SQLite
   batch mode.  Migration backfill also handles empty-string
   automation_profile values.  Documented downgrade limitation
   (backfill is not reversible).  Orphan-row fallback now logs
   affected row count at WARNING level.  Migration cycle-detection
   now logs affected plan_id values before the orphan fallback
   overwrites them.  All migration SQL uses sa.text() for
   consistency with SQLAlchemy best practices.

6. Hardened automation_profile deserialization in to_domain() to
   catch ValueError (invalid StrEnum provenance), Pydantic
   ValidationError, and RecursionError (deeply nested JSON) in
   addition to JSONDecodeError and KeyError, preventing
   unreadable plans from corrupted DB rows.  Applied the same
   defensive deserialization pattern to effective_profile_snapshot
   in to_domain(): corrupted JSON falls back to '{}' with a
   WARNING log instead of crashing the read path.  Added TypeError
   to the effective_profile_snapshot exception list in to_domain()
   for consistency with the Pydantic validator.  Logging of
   unparseable values uses length only to avoid potential
   information disclosure.

7. Used explicit None check (is not None) instead of truthiness
   for root_plan_id resolution in from_domain(), for
   effective_profile_snapshot in to_domain(), and in
   _serialize_automation_profile() for consistency.

8. Documented intentional column naming conventions vs spec DDL
   (e.g. automation_profile vs automation_profile_name, *_actor
   vs *_actor_name, processing_state vs state, v3_plans vs
   plans).  Documented the semantic difference: automation_profile
   stores either a bare name or structured JSON with provenance,
   whereas the spec automation_profile_name stores a plain name.

9. Fixed benchmark plan constructors
   (plan_phase_migration_bench.py) that were missing the now-
   required root_plan_id and effective_profile_snapshot fields.

10. Replaced defensive getattr() with direct attribute access for
    effective_profile_snapshot in from_domain() and update(),
    since the field is now defined on the Plan domain model.

11. Fixed Any type annotation in test helper _make_plan() to use
    AutomationProfileRef | None for proper type safety.

12. Added BDD scenarios for PlanIdentity self-reference
    resolution, NULL effective_profile_snapshot constraint
    enforcement, valid-JSON-missing-profile_name-key
    deserialization, invalid-JSON and empty-string snapshot
    rejection by Pydantic validator, and corrupted
    effective_profile_snapshot DB fallback in to_domain().

13. Extracted default automation profile name to a module-level
    constant (DEFAULT_AUTOMATION_PROFILE) to reduce sentinel
    duplication across models.py and repositories.py.

14. Centralised automation-profile serialisation into
    LifecyclePlanModel._serialize_automation_profile() to
    eliminate duplication between from_domain() and
    LifecyclePlanRepository.update().

15. Fixed to_domain() root_plan_id type cast from str | None
    to str, reflecting the NOT NULL column constraint.

16. Added PlanIdentity model_validator that resolves None
    root_plan_id to plan_id at domain construction time, ensuring
    the domain model honours the spec DDL NOT NULL constraint
    regardless of persistence state.  Simplified from_domain()
    root resolution accordingly.

ISSUES CLOSED: #921
2026-03-30 23:40:36 +01:00

140 lines
7.2 KiB
Gherkin

@feature921
Feature: v3_plans table schema alignment with spec DDL
As a database administrator
I want the v3_plans table schema to match the specification DDL
So that nullability, defaults, and columns are correct
Background:
Given a schema-aligned fresh in-memory database
And a schema-aligned prerequisite action "local/schema-test" exists
@feature921
Scenario: Plan creation includes effective_profile_snapshot
Given a schema-aligned plan with ID "01HV000000000000000000SA01"
And the plan effective_profile_snapshot is '{"profile":"balanced","thresholds":{}}'
When I persist the schema-aligned plan
Then the retrieved plan effective_profile_snapshot should be '{"profile":"balanced","thresholds":{}}'
@feature921
Scenario: Plan creation with default effective_profile_snapshot
Given a schema-aligned plan with ID "01HV000000000000000000SA02"
When I persist the schema-aligned plan
Then the retrieved plan effective_profile_snapshot should be '{}'
@feature921
Scenario: Root plan has self-referencing root_plan_id
Given a schema-aligned root plan with ID "01HV000000000000000000SA03"
When I persist the schema-aligned plan
Then the plan root_plan_id should equal the plan_id "01HV000000000000000000SA03"
@feature921
Scenario: Child plan has parent root_plan_id
Given a schema-aligned root plan with ID "01HV000000000000000000SA04"
And a schema-aligned child plan with ID "01HV000000000000000000SA05" parent "01HV000000000000000000SA04" root "01HV000000000000000000SA04"
When I persist both schema-aligned plans
Then the child plan root_plan_id should be "01HV000000000000000000SA04"
And the child plan parent_plan_id should be "01HV000000000000000000SA04"
@feature921
Scenario: automation_profile defaults to balanced for plans without profile
Given a schema-aligned plan with ID "01HV000000000000000000SA06" and no automation profile
When I persist the schema-aligned plan
Then the persisted automation_profile column should be "balanced"
@feature921
Scenario: Bare-string automation_profile deserializes gracefully
Given a schema-aligned plan with ID "01HV000000000000000000SA07"
And the plan automation_profile column is set to bare string "full-auto"
When I load the plan from the database
Then the plan automation_profile ref should be None
@feature921
Scenario: Invalid provenance in automation_profile JSON deserializes gracefully
Given a schema-aligned plan with ID "01HV000000000000000000SA08"
And the plan automation_profile column is set to '{"profile_name":"test","provenance":"bogus"}'
When I load the plan from the database
Then the plan automation_profile ref should be None
@feature921
Scenario: Grandchild plan carries root ancestor plan_id
Given a schema-aligned root plan with ID "01HV000000000000000000SA10"
And a schema-aligned child plan with ID "01HV000000000000000000SA11" parent "01HV000000000000000000SA10" root "01HV000000000000000000SA10"
And a schema-aligned grandchild plan with ID "01HV000000000000000000SA12" parent "01HV000000000000000000SA11" root "01HV000000000000000000SA10"
When I persist root, child, and grandchild schema-aligned plans
Then the grandchild plan root_plan_id should be "01HV000000000000000000SA10"
And the grandchild plan parent_plan_id should be "01HV000000000000000000SA11"
@feature921
Scenario: automation_profile None roundtrips as None through domain model
Given a schema-aligned plan with ID "01HV000000000000000000SA13" and no automation profile
When I persist the schema-aligned plan
Then the persisted automation_profile column should be "balanced"
And the plan automation_profile ref should be None
@feature921
Scenario: Truncated JSON in automation_profile deserializes gracefully
Given a schema-aligned plan with ID "01HV000000000000000000SA14"
And the plan automation_profile column is set to '{"profile_name":"test"'
When I load the plan from the database
Then the plan automation_profile ref should be None
@feature921
Scenario: Updating effective_profile_snapshot persists correctly
Given a schema-aligned plan with ID "01HV000000000000000000SA15"
And the plan effective_profile_snapshot is '{"profile":"balanced","thresholds":{}}'
When I persist the schema-aligned plan
And I update the plan effective_profile_snapshot to '{"profile":"full-auto","thresholds":{"strategy":0.5}}'
Then the retrieved plan effective_profile_snapshot should be '{"profile":"full-auto","thresholds":{"strategy":0.5}}'
@feature921
Scenario: AutomationProfileRef round-trips through persistence
Given a schema-aligned plan "01HV000000000000000000SA16" with profile "supervised" and provenance "plan"
When I persist the schema-aligned plan
Then the plan automation_profile name should be "supervised"
And the plan automation_profile provenance should be "plan"
@feature921
Scenario: NULL root_plan_id is rejected by the database
Given a schema-aligned plan with ID "01HV000000000000000000SA09" forced NULL root_plan_id
When I attempt to persist the plan with NULL root_plan_id
Then the database should reject the insert with an integrity error
@feature921
Scenario: NULL effective_profile_snapshot is rejected by the database
Given a schema-aligned plan with ID "01HV000000000000000000SA17" forced NULL effective_profile_snapshot
When I attempt to persist the plan with NULL effective_profile_snapshot
Then the database should reject the snapshot insert with an integrity error
@feature921
Scenario: PlanIdentity resolves None root_plan_id to plan_id at construction
Given a schema-aligned plan with ID "01HV000000000000000000SA18" and None root_plan_id
Then the domain plan root_plan_id should already equal "01HV000000000000000000SA18"
When I convert the plan via from_domain
Then the ORM model root_plan_id should equal "01HV000000000000000000SA18"
@feature921
Scenario: Valid JSON missing profile_name key deserializes gracefully
Given a schema-aligned plan with ID "01HV000000000000000000SA19"
And the plan automation_profile column is set to '{"provenance":"plan"}'
When I load the plan from the database
Then the plan automation_profile ref should be None
@feature921
Scenario: Corrupted effective_profile_snapshot in DB falls back to empty JSON
Given a schema-aligned plan with ID "01HV000000000000000000SA20"
And the plan effective_profile_snapshot is '{"profile":"balanced"}'
When I persist the schema-aligned plan
And the effective_profile_snapshot column is corrupted to "not-valid-json{"
And I reload the plan from the database
Then the retrieved plan effective_profile_snapshot should be '{}'
@feature921
Scenario: Invalid JSON in effective_profile_snapshot is rejected by Plan model
When I attempt to construct a Plan with effective_profile_snapshot "not-valid-json"
Then the Plan construction should raise a validation error for effective_profile_snapshot
@feature921
Scenario: Empty string effective_profile_snapshot is rejected by Plan model
When I attempt to construct a Plan with an empty effective_profile_snapshot
Then the Plan construction should raise a validation error for effective_profile_snapshot