TDD: Write failing test for #989 — _to_domain/_from_domain crash on corrupt JSON #1094

Closed
opened 2026-03-22 16:30:07 +00:00 by freemo · 8 comments
Owner

Metadata

  • Commit Message: test: add TDD bug-capture test for #989 — JSON decode crash in persistence
  • Branch: tdd/m5-json-decode-crash

Background and Context

This is the TDD counterpart to bug #989. Per the project's Test-Driven Development workflow for bugs (see CONTRIBUTING.md > Bug Fix Workflow), the first step in fixing any bug is to write a test that captures the buggy behavior. The test is tagged with @tdd_bug, @tdd_bug_989, and @tdd_expected_fail so that it passes CI while the bug is still unfixed. Once the fix is implemented in #989, the @tdd_expected_fail tag will be removed and the test will run normally.

See #989 for full bug details.

Expected Behavior

A new test exists that:

  1. Captures the exact failure described in #989.
  2. Is tagged with @tdd_bug, @tdd_bug_989, and @tdd_expected_fail.
  3. Passes CI via the expected-failure mechanism (the underlying assertion fails, confirming the bug exists, but the tag inversion causes the test to pass).
  4. Would fail CI if the bug were fixed without removing the @tdd_expected_fail tag.

Acceptance Criteria

  • A test is written that captures the bug behavior described in #989.
  • The test is tagged with @tdd_bug, @tdd_bug_989, and @tdd_expected_fail.
  • The @tdd_expected_fail tag causes the test to pass CI (the underlying assertion fails as expected, proving the bug exists).
  • The test is specific enough that it will pass normally (without the tag) only when the bug is genuinely fixed.
  • Tag validation rules pass: @tdd_bug_989 has corresponding @tdd_bug, and @tdd_expected_fail has both.
  • A pull request is opened from the branch to master, CI passes, and the PR is merged through the normal merge process.

Definition of Done

This issue is complete when:

  • All subtasks below are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the test and what bug behavior it captures.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, CI passes, and the PR is merged before this issue is marked done.

Subtasks

  • Code: Analyze bug #989 to identify the exact failure condition, including the inputs, state, and code path that trigger the bug.
  • Code: Determine the appropriate test type (Behave unit test, Robot integration test, or both) and file location for the reproducing test.
  • Tests (Behave): Write a Behave scenario in features/ that captures the bug. Tag the scenario with @tdd_bug, @tdd_bug_989, and @tdd_expected_fail. The scenario must exercise the specific code path that triggers the bug and assert the correct expected behavior (which currently fails due to the bug). Name the scenario descriptively to indicate it is a bug regression test.
  • Tests (Robot): N/A — this defect is in persistence-domain mapping (AutomationProfileRepository._to_domain) and is most precise as a unit-level Behave regression test; integration-level Robot coverage for this bug path is not required to prove the defect.
  • Docs: Add a comment in the test file explaining this test captures bug #989 and uses @tdd_expected_fail until the fix is merged.
  • Quality: Verify CI passes with the tagged test. Confirm the underlying assertion fails for the correct reason.
  • Quality: Verify tag validation rules pass.
  • Quality: Verify coverage >=97% via nox -s coverage_report. If coverage is <97% then review the current unit test coverage report at build/coverage.xml and use it to write new Behave based unit tests to improve code coverage.
  • Quality: Run nox (all default sessions), fix any errors if needed ensuring nox passes across entire code base.
## Metadata - **Commit Message**: `test: add TDD bug-capture test for #989 — JSON decode crash in persistence` - **Branch**: `tdd/m5-json-decode-crash` ## Background and Context This is the TDD counterpart to bug #989. Per the project's Test-Driven Development workflow for bugs (see `CONTRIBUTING.md` > Bug Fix Workflow), the first step in fixing any bug is to write a test that captures the buggy behavior. The test is tagged with `@tdd_bug`, `@tdd_bug_989`, and `@tdd_expected_fail` so that it passes CI while the bug is still unfixed. Once the fix is implemented in #989, the `@tdd_expected_fail` tag will be removed and the test will run normally. See #989 for full bug details. ## Expected Behavior A new test exists that: 1. Captures the exact failure described in #989. 2. Is tagged with `@tdd_bug`, `@tdd_bug_989`, and `@tdd_expected_fail`. 3. Passes CI via the expected-failure mechanism (the underlying assertion fails, confirming the bug exists, but the tag inversion causes the test to pass). 4. Would fail CI if the bug were fixed without removing the `@tdd_expected_fail` tag. ## Acceptance Criteria - [ ] A test is written that captures the bug behavior described in #989. - [ ] The test is tagged with `@tdd_bug`, `@tdd_bug_989`, and `@tdd_expected_fail`. - [ ] The `@tdd_expected_fail` tag causes the test to pass CI (the underlying assertion fails as expected, proving the bug exists). - [ ] The test is specific enough that it will pass normally (without the tag) only when the bug is genuinely fixed. - [ ] Tag validation rules pass: `@tdd_bug_989` has corresponding `@tdd_bug`, and `@tdd_expected_fail` has both. - [ ] A pull request is opened from the branch to `master`, CI passes, and the PR is merged through the normal merge process. ## Definition of Done This issue is complete when: - All subtasks below are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the test and what bug behavior it captures. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, CI passes, and the PR is **merged** before this issue is marked done. ## Subtasks - [x] Code: Analyze bug #989 to identify the exact failure condition, including the inputs, state, and code path that trigger the bug. - [x] Code: Determine the appropriate test type (Behave unit test, Robot integration test, or both) and file location for the reproducing test. - [x] Tests (Behave): Write a Behave scenario in `features/` that captures the bug. Tag the scenario with `@tdd_bug`, `@tdd_bug_989`, and `@tdd_expected_fail`. The scenario must exercise the specific code path that triggers the bug and assert the correct expected behavior (which currently fails due to the bug). Name the scenario descriptively to indicate it is a bug regression test. - [x] Tests (Robot): N/A — this defect is in persistence-domain mapping (`AutomationProfileRepository._to_domain`) and is most precise as a unit-level Behave regression test; integration-level Robot coverage for this bug path is not required to prove the defect. - [x] Docs: Add a comment in the test file explaining this test captures bug #989 and uses `@tdd_expected_fail` until the fix is merged. - [x] Quality: Verify CI passes with the tagged test. Confirm the underlying assertion fails for the correct reason. - [x] Quality: Verify tag validation rules pass. - [x] Quality: Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. - [x] Quality: Run `nox` (all default sessions), fix any errors if needed ensuring nox passes across **entire** code base.
freemo added this to the v3.5.0 milestone 2026-03-22 16:30:07 +00:00
Member

Phase 1/2 update:

  • Reviewed issue #1094, bug #989, and relevant guidance in CONTRIBUTING.md (Bug Fix Workflow, TDD bug tags, tag validation rules, Behave-first unit testing policy) plus docs/specification.md automation profile/safety profile sections and docs/timeline.md M6 context.
  • Confirmed branch metadata target: tdd/m5-json-decode-crash.
  • Confirmed issue state transition to State/In Progress.

Implementation plan:

  1. Add a new Behave bug-capture feature under features/ with required tags: @tdd_bug @tdd_bug_989 @tdd_expected_fail.
  2. Reproduce the concrete failure path by persisting an AutomationProfileModel row with malformed safety_json and calling AutomationProfileRepository.get_by_name(...), which routes through _to_domain and currently uses bare json.loads.
  3. Assert expected behavior (domain-specific corruption handling, not raw json.JSONDecodeError) so the assertion fails today and is inverted by @tdd_expected_fail.
  4. Mark Robot as N/A with rationale (this defect is persistence-domain mapping behavior and is best isolated at unit/Behave level).
  5. Run full quality gates (nox -s coverage_report and full nox) before commit/PR.
Phase 1/2 update: - Reviewed issue #1094, bug #989, and relevant guidance in `CONTRIBUTING.md` (Bug Fix Workflow, TDD bug tags, tag validation rules, Behave-first unit testing policy) plus `docs/specification.md` automation profile/safety profile sections and `docs/timeline.md` M6 context. - Confirmed branch metadata target: `tdd/m5-json-decode-crash`. - Confirmed issue state transition to `State/In Progress`. Implementation plan: 1. Add a new Behave bug-capture feature under `features/` with required tags: `@tdd_bug @tdd_bug_989 @tdd_expected_fail`. 2. Reproduce the concrete failure path by persisting an `AutomationProfileModel` row with malformed `safety_json` and calling `AutomationProfileRepository.get_by_name(...)`, which routes through `_to_domain` and currently uses bare `json.loads`. 3. Assert expected behavior (domain-specific corruption handling, not raw `json.JSONDecodeError`) so the assertion fails today and is inverted by `@tdd_expected_fail`. 4. Mark Robot as N/A with rationale (this defect is persistence-domain mapping behavior and is best isolated at unit/Behave level). 5. Run full quality gates (`nox -s coverage_report` and full `nox`) before commit/PR.
Member

Implementation update (Phase 2/3):

What was implemented

  • Added new Behave TDD bug-capture feature:
    • features/tdd_json_decode_crash_persistence.feature
  • Added dedicated step definitions:
    • features/steps/tdd_json_decode_crash_persistence_steps.py

Bug path captured

The new scenario persists an AutomationProfileModel row with malformed safety_json and executes AutomationProfileRepository.get_by_name(...).

This specifically exercises the AutomationProfileRepository._to_domain JSON decode path (in src/cleveragents/infrastructure/database/repositories.py) where bare json.loads(...) currently leaks raw json.JSONDecodeError for corrupt data.

Assertion design

The scenario asserts the expected post-fix behavior:

  • an error is raised,
  • it is not json.JSONDecodeError, and
  • it is corruption-specific (type/message contains corrupt).

Current behavior leaks JSONDecodeError, so the scenario’s core assertion fails for the correct reason and is inverted by @tdd_expected_fail.

TDD tags and docs

  • Scenario uses all required tags: @tdd_bug @tdd_bug_989 @tdd_expected_fail.
  • Feature file includes explicit comments documenting that this is the bug #989 capture and that @tdd_expected_fail must be removed by the bugfix branch.

Robot test decision

  • Marked Robot subtask as N/A with justification: this defect is a persistence-domain mapping bug and is most directly and deterministically captured at the Behave unit level.

Additional quality-gate stabilization work

While running required full quality gates, integration tests consistently failed in Robot.Resource Dag (Cycle Detection Rejects A To B To A) due session visibility/transaction behavior in the inline script fixture. To keep quality gates green, I made a minimal deterministic stabilization in:

  • robot/resource_dag.robot

Change: use a shared SQLAlchemy session object for ResourceTypeRepository and ResourceRepository in the suite’s inline scripts, removing nondeterministic visibility of freshly flushed-but-not-committed type rows.

This was small, directly related to satisfying required project quality gates for this branch.

Implementation update (Phase 2/3): ### What was implemented - Added new Behave TDD bug-capture feature: - `features/tdd_json_decode_crash_persistence.feature` - Added dedicated step definitions: - `features/steps/tdd_json_decode_crash_persistence_steps.py` ### Bug path captured The new scenario persists an `AutomationProfileModel` row with malformed `safety_json` and executes `AutomationProfileRepository.get_by_name(...)`. This specifically exercises the `AutomationProfileRepository._to_domain` JSON decode path (in `src/cleveragents/infrastructure/database/repositories.py`) where bare `json.loads(...)` currently leaks raw `json.JSONDecodeError` for corrupt data. ### Assertion design The scenario asserts the expected post-fix behavior: - an error is raised, - it is **not** `json.JSONDecodeError`, and - it is corruption-specific (type/message contains `corrupt`). Current behavior leaks `JSONDecodeError`, so the scenario’s core assertion fails for the correct reason and is inverted by `@tdd_expected_fail`. ### TDD tags and docs - Scenario uses all required tags: `@tdd_bug @tdd_bug_989 @tdd_expected_fail`. - Feature file includes explicit comments documenting that this is the bug #989 capture and that `@tdd_expected_fail` must be removed by the bugfix branch. ### Robot test decision - Marked Robot subtask as N/A with justification: this defect is a persistence-domain mapping bug and is most directly and deterministically captured at the Behave unit level. ### Additional quality-gate stabilization work While running required full quality gates, integration tests consistently failed in `Robot.Resource Dag` (`Cycle Detection Rejects A To B To A`) due session visibility/transaction behavior in the inline script fixture. To keep quality gates green, I made a minimal deterministic stabilization in: - `robot/resource_dag.robot` Change: use a shared SQLAlchemy session object for `ResourceTypeRepository` and `ResourceRepository` in the suite’s inline scripts, removing nondeterministic visibility of freshly flushed-but-not-committed type rows. This was small, directly related to satisfying required project quality gates for this branch.
Member

Phase 4/5 completion update:

  • Commit pushed on tdd/m5-json-decode-crash: 045bfd03f394b558f8a979979e407bf0a1762570
  • PR opened: #1166
  • Issue moved to State/In Review.

Quality gates (final runs):

  • nox -s lint
  • nox -s typecheck
  • nox -s unit_tests
  • nox -s integration_tests
  • nox -s e2e_tests
  • nox -s coverage_report
  • nox (full default suite)
  • Coverage: 97.67% (build/coverage.xml)

Traceability:

  • TDD capture feature: features/tdd_json_decode_crash_persistence.feature
  • TDD step implementations: features/steps/tdd_json_decode_crash_persistence_steps.py
  • Integration quality-gate stabilization: robot/resource_dag.robot (shared-session fixture update)

Note:

  • I set PR milestone to match the issue milestone (v3.5.0 / id 108) and applied Type/Testing label to the PR.
Phase 4/5 completion update: - Commit pushed on `tdd/m5-json-decode-crash`: `045bfd03f394b558f8a979979e407bf0a1762570` - PR opened: https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/1166 - Issue moved to `State/In Review`. Quality gates (final runs): - `nox -s lint` ✅ - `nox -s typecheck` ✅ - `nox -s unit_tests` ✅ - `nox -s integration_tests` ✅ - `nox -s e2e_tests` ✅ - `nox -s coverage_report` ✅ - `nox` (full default suite) ✅ - Coverage: **97.67%** (`build/coverage.xml`) Traceability: - TDD capture feature: `features/tdd_json_decode_crash_persistence.feature` - TDD step implementations: `features/steps/tdd_json_decode_crash_persistence_steps.py` - Integration quality-gate stabilization: `robot/resource_dag.robot` (shared-session fixture update) Note: - I set PR milestone to match the issue milestone (v3.5.0 / id 108) and applied `Type/Testing` label to the PR.
freemo self-assigned this 2026-04-02 06:13:54 +00:00
Author
Owner

PR #1166 Review Update: Code reviewed and APPROVED by independent reviewer. The TDD test correctly captures bug #989 with proper tags, well-structured assertions, and clean code quality.

However, merge is blocked by a conflict in robot/resource_dag.robot — the session-sharing fix in the PR was already merged to master via commit b6c31696 (PR for #762) with additional improvements. The branch needs a rebase onto master, dropping the Robot file changes (they're already superseded on master). After rebase, the PR can be merged.

**PR #1166 Review Update**: Code reviewed and **APPROVED** by independent reviewer. The TDD test correctly captures bug #989 with proper tags, well-structured assertions, and clean code quality. However, **merge is blocked by a conflict** in `robot/resource_dag.robot` — the session-sharing fix in the PR was already merged to master via commit `b6c31696` (PR for #762) with additional improvements. The branch needs a rebase onto master, dropping the Robot file changes (they're already superseded on master). After rebase, the PR can be merged.
Author
Owner

PR #1166 Review Update

PR #1166 has been approved — the TDD bug-capture test for #989 is well-designed and meets all quality criteria.

However, the PR has a merge conflict in robot/resource_dag.robot (the session-sharing changes were already merged to master via another PR). A rebase is needed before the PR can be merged.

Waiting on the implementor to rebase the branch onto master.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

## PR #1166 Review Update PR #1166 has been **approved** — the TDD bug-capture test for #989 is well-designed and meets all quality criteria. However, the PR has a **merge conflict** in `robot/resource_dag.robot` (the session-sharing changes were already merged to master via another PR). A rebase is needed before the PR can be merged. Waiting on the implementor to rebase the branch onto master. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer
Author
Owner

PR #1166 Review Status

PR #1166 has been approved by independent code review. The TDD bug-capture test for #989 is well-designed and correctly captures the defect.

Merge is currently blocked by a conflict in robot/resource_dag.robot. The session-sharing changes in the PR branch have already been merged to master via a separate PR with improvements. The branch needs a rebase onto master, dropping the Robot file changes.

Once the rebase is completed and CI passes, the PR can be merged immediately.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

## PR #1166 Review Status PR #1166 has been **approved** by independent code review. The TDD bug-capture test for #989 is well-designed and correctly captures the defect. **Merge is currently blocked by a conflict** in `robot/resource_dag.robot`. The session-sharing changes in the PR branch have already been merged to master via a separate PR with improvements. The branch needs a rebase onto master, dropping the Robot file changes. Once the rebase is completed and CI passes, the PR can be merged immediately. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer
Author
Owner

PR #1166 has been merged successfully. Issue should now be resolved.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-continuous-pr-reviewer

PR #1166 has been merged successfully. Issue should now be resolved. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-continuous-pr-reviewer
Author
Owner

PR #1166 reviewed, approved, and merged.

The TDD bug-capture test for #989 has been merged to master. The test correctly captures the JSON decode crash in persistence with proper @tdd_bug @tdd_bug_989 @tdd_expected_fail tags and a well-designed three-stage assertion.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: ca-pr-self-reviewer

PR #1166 reviewed, approved, and merged. The TDD bug-capture test for #989 has been merged to master. The test correctly captures the JSON decode crash in persistence with proper `@tdd_bug @tdd_bug_989 @tdd_expected_fail` tags and a well-designed three-stage assertion. --- **Automated by CleverAgents Bot** Supervisor: PR Review | Agent: ca-pr-self-reviewer
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1094
No description provided.