TEST-INFRA: [test-data-quality] Introduce a fixture factory for more realistic and robust test data #5671

New issue

Open

opened 2026-04-09 08:30:18 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 08:30:18 +00:00

Owner

Metadata

Branch: test/test-data-quality-fixture-factory
Commit Message: test(infra): introduce factory_boy fixture factory for realistic domain model test data
Milestone: Backlog (no milestone — see backlog note below)
Parent Epic: #5407

Description

Our current test suite primarily uses hardcoded, simple data within the step definitions. This approach has several drawbacks:

Lack of Realism: The test data does not always reflect the complexity of real-world data, which can lead to bugs that only surface in production.
Limited Edge Case Coverage: It is cumbersome to test edge cases (e.g., empty strings, long strings, special characters, null values) with the current approach.
High Maintenance: Changes to data models require updating many hardcoded values across numerous step definition files.

Introducing a fixture factory library, such as factory_boy, would address these issues by providing a centralized and flexible way to generate realistic and varied test data for core domain models (Plan, Action, Resource, etc.). This complements the dynamic data generation approach (see #5434) by providing structured, model-aware factories rather than raw data generation.

Subtasks

Research and select a suitable fixture factory library (primary candidate: factory_boy).
Integrate the chosen library into the test dependencies via nox.
Create initial factories for core domain models (e.g., Plan, Action, Resource).
Refactor a few existing Behave step definitions to use the new factories as a proof of concept.
Document the usage of the fixture factory in CONTRIBUTING.md.
Ensure all nox stages pass after integration.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A fixture factory library is integrated into the project via nox (no direct installs).
Factories for core domain models (Plan, Action, Resource) are created in features/.
At least one existing Behave test suite is refactored to use the new factories.
Documentation for using the fixture factory is added to CONTRIBUTING.md.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e coverage_report).
Coverage >= 97%.

Duplicate Check

Search Queries: fixture factory, factory_boy
Results: 0 open or closed issues found.
Justification: No existing issues address the introduction of a factory_boy-style fixture factory for domain model test data generation. The related issue #5434 covers dynamic data generation with Faker but does not address structured model factories.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.7.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `test/test-data-quality-fixture-factory` - **Commit Message**: `test(infra): introduce factory_boy fixture factory for realistic domain model test data` - **Milestone**: Backlog (no milestone — see backlog note below) - **Parent Epic**: #5407 ## Description Our current test suite primarily uses hardcoded, simple data within the step definitions. This approach has several drawbacks: - **Lack of Realism:** The test data does not always reflect the complexity of real-world data, which can lead to bugs that only surface in production. - **Limited Edge Case Coverage:** It is cumbersome to test edge cases (e.g., empty strings, long strings, special characters, null values) with the current approach. - **High Maintenance:** Changes to data models require updating many hardcoded values across numerous step definition files. Introducing a fixture factory library, such as `factory_boy`, would address these issues by providing a centralized and flexible way to generate realistic and varied test data for core domain models (`Plan`, `Action`, `Resource`, etc.). This complements the dynamic data generation approach (see #5434) by providing structured, model-aware factories rather than raw data generation. ## Subtasks - [ ] Research and select a suitable fixture factory library (primary candidate: `factory_boy`). - [ ] Integrate the chosen library into the test dependencies via `nox`. - [ ] Create initial factories for core domain models (e.g., `Plan`, `Action`, `Resource`). - [ ] Refactor a few existing Behave step definitions to use the new factories as a proof of concept. - [ ] Document the usage of the fixture factory in `CONTRIBUTING.md`. - [ ] Ensure all nox stages pass after integration. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A fixture factory library is integrated into the project via `nox` (no direct installs). - Factories for core domain models (`Plan`, `Action`, `Resource`) are created in `features/`. - At least one existing Behave test suite is refactored to use the new factories. - Documentation for using the fixture factory is added to `CONTRIBUTING.md`. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e coverage_report`). - Coverage >= 97%. ### Duplicate Check - **Search Queries:** `fixture factory`, `factory_boy` - **Results:** 0 open or closed issues found. - **Justification:** No existing issues address the introduction of a `factory_boy`-style fixture factory for domain model test data generation. The related issue #5434 covers dynamic data generation with Faker but does not address structured model factories. > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.7.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

HAL9000 added a new dependency

2026-04-09 08:31:16 +00:00

#5407 EPIC: Testing Infrastructure Improvements — Coverage, CI Pipeline, Dependencies & Test Levels

HAL9000 added the

Priority

Backlog

Rows
Columns