TDD: Write failing test for #1022 — InvariantService in-memory storage only #1032

Closed
opened 2026-03-17 18:19:00 +00:00 by freemo · 3 comments
Owner

Metadata

  • Commit Message: test: add TDD bug-capture test for #1022 — InvariantService persistence
  • Branch: tdd/m5-invariant-service-persistence

Background and Context

This is the TDD counterpart to bug #1022. Per the project's Test-Driven Development workflow for bugs (see CONTRIBUTING.md > Bug Fix Workflow), the first step in fixing any bug is to write a test that captures the buggy behavior.

See #1022 for full bug details.

Expected Behavior

A new test exists that:

  1. Captures the failure — InvariantService stores invariants only in memory, losing them across CLI process invocations.
  2. Is tagged with @tdd_bug, @tdd_bug_1022, and @tdd_expected_fail.
  3. Passes CI via the expected-failure mechanism.

Acceptance Criteria

  • A test captures the bug behavior described in #1022.
  • Tagged with @tdd_bug, @tdd_bug_1022, and @tdd_expected_fail.
  • CI passes with the tagged test.
  • Tag validation rules pass.
  • PR opened, reviewed, merged to master.

Definition of Done

This issue is complete when all subtasks are done, commit pushed, PR merged.

Subtasks

  • Code: Analyze bug #1022 — invariants are lost across process boundaries.
  • Tests (Behave): Write a scenario verifying invariant persistence across simulated process restarts. Tag with @tdd_bug, @tdd_bug_1022, @tdd_expected_fail.
  • Tests (Robot): Add integration test if cross-process behavior warrants it.
  • Docs: Comment explaining this test captures bug #1022.
  • Quality: Verify CI passes. Verify tag validation. Coverage >=97%. nox passes.
## Metadata - **Commit Message**: `test: add TDD bug-capture test for #1022 — InvariantService persistence` - **Branch**: `tdd/m5-invariant-service-persistence` ## Background and Context This is the TDD counterpart to bug #1022. Per the project's Test-Driven Development workflow for bugs (see `CONTRIBUTING.md` > Bug Fix Workflow), the first step in fixing any bug is to write a test that captures the buggy behavior. See #1022 for full bug details. ## Expected Behavior A new test exists that: 1. Captures the failure — `InvariantService` stores invariants only in memory, losing them across CLI process invocations. 2. Is tagged with `@tdd_bug`, `@tdd_bug_1022`, and `@tdd_expected_fail`. 3. Passes CI via the expected-failure mechanism. ## Acceptance Criteria - [x] A test captures the bug behavior described in #1022. - [x] Tagged with `@tdd_bug`, `@tdd_bug_1022`, and `@tdd_expected_fail`. - [x] CI passes with the tagged test. - [x] Tag validation rules pass. - [ ] PR opened, reviewed, merged to `master`. ## Definition of Done This issue is complete when all subtasks are done, commit pushed, PR merged. ## Subtasks - [x] Code: Analyze bug #1022 — invariants are lost across process boundaries. - [x] Tests (Behave): Write a scenario verifying invariant persistence across simulated process restarts. Tag with `@tdd_bug`, `@tdd_bug_1022`, `@tdd_expected_fail`. - [x] Tests (Robot): Add integration test if cross-process behavior warrants it. - [x] Docs: Comment explaining this test captures bug #1022. - [x] Quality: Verify CI passes. Verify tag validation. Coverage >=97%. `nox` passes.
freemo added this to the v3.4.0 milestone 2026-03-17 18:19:21 +00:00
Author
Owner

Planning Agent — Dependency Note

This TDD issue (#1032) blocks bug fix #1022. Bug #1022 depends on this issue being completed first per the TDD workflow.

Dependency: #1022 depends on → #1032 (this issue).

Note: Forgejo dependency link creation via API returned errors. The dependency relationship is documented here until the link can be created through the Forgejo UI. @brent.edwards or @hamza.khyari — please add this dependency link via the Forgejo issue UI: open #1022, add #1032 under "depends on".

**Planning Agent — Dependency Note** This TDD issue (#1032) blocks bug fix #1022. Bug #1022 depends on this issue being completed first per the TDD workflow. **Dependency**: #1022 depends on → #1032 (this issue). Note: Forgejo dependency link creation via API returned errors. The dependency relationship is documented here until the link can be created through the Forgejo UI. @brent.edwards or @hamza.khyari — please add this dependency link via the Forgejo issue UI: open #1022, add #1032 under "depends on".
Author
Owner

Assigned to @brent.edwards for TDD test writing for bug #1022 (InvariantService in-memory). This TDD counterpart is top priority per project policy — bugs always take precedence over feature work.

Assigned to @brent.edwards for TDD test writing for bug #1022 (InvariantService in-memory). This TDD counterpart is top priority per project policy — bugs always take precedence over feature work.
Member

Implementation Notes — TDD Bug-Capture Tests for #1022

Analysis (Subtask 1)

Bug #1022: InvariantService (at cleveragents.application.services.invariant_service.InvariantService) stores invariants in self._invariants: dict[str, Invariant] — a plain in-memory dict with no database persistence. The CLI helper _get_service() at cleveragents.cli.commands.invariant._get_service creates a module-level singleton InvariantService(), but each CLI process invocation gets its own Python process, so the singleton is per-process. This means:

  1. agents invariant add --project X "text" succeeds and stores in the process-local dict
  2. Process exits → dict is garbage-collected
  3. agents invariant list --project X in a new process → fresh empty dict → "No invariants found"

No InvariantRepository, no InvariantModel ORM entity, no DI registration in container.py.

Behave Tests (Subtask 2)

Created features/tdd_invariant_persistence.feature with 4 scenarios tagged @tdd_expected_fail @tdd_bug @tdd_bug_1022:

  1. Service-level persistence: Add project invariant via instance A, create fresh instance B, list via B → should contain invariant (fails due to bug)
  2. Global invariant persistence: Same pattern with global scope
  3. CLI cross-invocation: Add via CLI invocation 1, list via CLI invocation 2 (fresh service) → should contain text (fails due to bug)
  4. Cross-instance remove: Add in instance A, attempt remove by ID in instance B → should succeed (fails with NotFoundError due to bug)

Step definitions in features/steps/tdd_invariant_persistence_steps.py. All scenarios pass CI via the @tdd_expected_fail inversion mechanism (Behave environment hooks invert fail→pass).

Robot Integration Tests (Subtask 3)

Created robot/tdd_invariant_persistence.robot with 3 test cases tagged tdd_expected_fail tdd_bug tdd_bug_1022:

  1. Add then list project — cross-invocation via helper subprocess
  2. Add then list global — global scope variant
  3. Add then remove cross-instance — remove by ID in fresh instance

Helper script at robot/helper_tdd_invariant_persistence.py runs each scenario as a self-contained check using fresh InvariantService instances.

Quality Gate Results (Subtask 5)

  • lint: passed
  • typecheck: passed (Pyright, 0 errors)
  • unit_tests: 462 features passed, 0 failed, 12234 scenarios passed
  • integration_tests: 1669/1675 passed; 6 pre-existing failures unrelated to this change (plan execution timeout issues)
  • coverage_report: 98.38% (threshold: 97%)
  • e2e_tests: Pre-existing failures (LLM API keys not available in dev environment — not a merge blocker per CI config)

Tag Validation

All tests correctly carry the three required TDD tags:

  • @tdd_bug (generic filter)
  • @tdd_bug_1022 (issue reference)
  • @tdd_expected_fail (behavioral switch — to be removed when #1022 is fixed)
## Implementation Notes — TDD Bug-Capture Tests for #1022 ### Analysis (Subtask 1) Bug #1022: `InvariantService` (at `cleveragents.application.services.invariant_service.InvariantService`) stores invariants in `self._invariants: dict[str, Invariant]` — a plain in-memory dict with no database persistence. The CLI helper `_get_service()` at `cleveragents.cli.commands.invariant._get_service` creates a module-level singleton `InvariantService()`, but each CLI process invocation gets its own Python process, so the singleton is per-process. This means: 1. `agents invariant add --project X "text"` succeeds and stores in the process-local dict 2. Process exits → dict is garbage-collected 3. `agents invariant list --project X` in a new process → fresh empty dict → "No invariants found" No `InvariantRepository`, no `InvariantModel` ORM entity, no DI registration in `container.py`. ### Behave Tests (Subtask 2) Created `features/tdd_invariant_persistence.feature` with 4 scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_1022`: 1. **Service-level persistence**: Add project invariant via instance A, create fresh instance B, list via B → should contain invariant (fails due to bug) 2. **Global invariant persistence**: Same pattern with global scope 3. **CLI cross-invocation**: Add via CLI invocation 1, list via CLI invocation 2 (fresh service) → should contain text (fails due to bug) 4. **Cross-instance remove**: Add in instance A, attempt remove by ID in instance B → should succeed (fails with `NotFoundError` due to bug) Step definitions in `features/steps/tdd_invariant_persistence_steps.py`. All scenarios pass CI via the `@tdd_expected_fail` inversion mechanism (Behave environment hooks invert fail→pass). ### Robot Integration Tests (Subtask 3) Created `robot/tdd_invariant_persistence.robot` with 3 test cases tagged `tdd_expected_fail tdd_bug tdd_bug_1022`: 1. **Add then list project** — cross-invocation via helper subprocess 2. **Add then list global** — global scope variant 3. **Add then remove cross-instance** — remove by ID in fresh instance Helper script at `robot/helper_tdd_invariant_persistence.py` runs each scenario as a self-contained check using fresh `InvariantService` instances. ### Quality Gate Results (Subtask 5) - **lint**: ✅ passed - **typecheck**: ✅ passed (Pyright, 0 errors) - **unit_tests**: ✅ 462 features passed, 0 failed, 12234 scenarios passed - **integration_tests**: ✅ 1669/1675 passed; 6 pre-existing failures unrelated to this change (plan execution timeout issues) - **coverage_report**: ✅ **98.38%** (threshold: 97%) - **e2e_tests**: Pre-existing failures (LLM API keys not available in dev environment — not a merge blocker per CI config) ### Tag Validation All tests correctly carry the three required TDD tags: - `@tdd_bug` (generic filter) - `@tdd_bug_1022` (issue reference) - `@tdd_expected_fail` (behavioral switch — to be removed when #1022 is fixed)
brent.edwards added reference tdd/m5-invariant-service-persistence 2026-03-23 00:12:01 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#1032
No description provided.