TEST-INFRA: [test-data-quality] Replace Hardcoded ULIDs in Test Fixtures with Dynamic Generation #5544

New issue

Open

opened 2026-04-09 07:19:17 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 07:19:17 +00:00

Owner

Metadata

Type: Type/Testing
Priority: Priority/Backlog
Labels: State/Unverified
Branch: test-data-quality/replace-hardcoded-ulids
Commit Message: test(fixtures): replace hardcoded ULIDs in test fixtures with dynamic generation
Milestone: (backlog — see note below)
Parent Epic: #5407

Background and context

During an analysis of the test infrastructure, it was observed that some Behave step implementations use hardcoded ULIDs for test fixtures. An example of this pattern can be found in features/steps/plan_model_uncovered_lines_steps.py:

# ---------------------------------------------------------------------------
# Valid ULIDs for test fixtures (Crockford base32, 26 chars)
# ---------------------------------------------------------------------------
_ULID_SELF = "01HGZ6FE0AQDYTR4BXVQZ6EA01"
_ULID_PARENT = "01HGZ6FE0AQDYTR4BXVQZ6EB01"
_ULID_ROOT = "01HGZ6FE0AQDYTR4BXVQZ6EC01"
_ULID_SUBPLAN = "01HGZ6FE0AQDYTR4BXVQZ6ED01"

While using constants is better than scattering magic strings, these ULIDs are still hardcoded. This means that the tests are always run with the same IDs, which might not be representative of a real-world scenario.

Expected behavior

To improve the robustness of the test suite, the hardcoded ULIDs in the test fixtures should be replaced with dynamically generated ULIDs. This can be achieved by:

Using a ULID Library: A library like ulid-py can be used to generate ULIDs.
Updating the Test Data Factories: The test data factories can be updated to generate new ULIDs for each test run.

This will make the tests more robust by exercising the code with a wider range of inputs.

Subtasks

Identify all the step implementation files that use hardcoded ULIDs.
Introduce a ULID generation library to the test dependencies.
Update the test data factories to generate new ULIDs for each test run.
Ensure that all tests pass after the refactoring.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message
matches the Commit Message in Metadata exactly, followed by a blank line,
then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in
Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and
merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Duplicate Check

Search queries: "hardcoded ULIDs", "dynamic test data"
Results: 0 for all queries.
Reasoning: No existing issues cover the replacement of hardcoded ULIDs in test fixtures with dynamic generation.

Backlog note: This issue was discovered during autonomous operation
by the test-infra-improver agent. It does not block any active milestone
completion and has been placed in the backlog for human review and future
milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Type**: `Type/Testing` - **Priority**: `Priority/Backlog` - **Labels**: `State/Unverified` - **Branch**: `test-data-quality/replace-hardcoded-ulids` - **Commit Message**: `test(fixtures): replace hardcoded ULIDs in test fixtures with dynamic generation` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #5407 ## Background and context During an analysis of the test infrastructure, it was observed that some Behave step implementations use hardcoded ULIDs for test fixtures. An example of this pattern can be found in `features/steps/plan_model_uncovered_lines_steps.py`: ```python # --------------------------------------------------------------------------- # Valid ULIDs for test fixtures (Crockford base32, 26 chars) # --------------------------------------------------------------------------- _ULID_SELF = "01HGZ6FE0AQDYTR4BXVQZ6EA01" _ULID_PARENT = "01HGZ6FE0AQDYTR4BXVQZ6EB01" _ULID_ROOT = "01HGZ6FE0AQDYTR4BXVQZ6EC01" _ULID_SUBPLAN = "01HGZ6FE0AQDYTR4BXVQZ6ED01" ``` While using constants is better than scattering magic strings, these ULIDs are still hardcoded. This means that the tests are always run with the same IDs, which might not be representative of a real-world scenario. ## Expected behavior To improve the robustness of the test suite, the hardcoded ULIDs in the test fixtures should be replaced with dynamically generated ULIDs. This can be achieved by: * **Using a ULID Library:** A library like `ulid-py` can be used to generate ULIDs. * **Updating the Test Data Factories:** The test data factories can be updated to generate new ULIDs for each test run. This will make the tests more robust by exercising the code with a wider range of inputs. ## Subtasks - [ ] Identify all the step implementation files that use hardcoded ULIDs. - [ ] Introduce a ULID generation library to the test dependencies. - [ ] Update the test data factories to generate new ULIDs for each test run. - [ ] Ensure that all tests pass after the refactoring. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. ### Duplicate Check - **Search queries:** "hardcoded ULIDs", "dynamic test data" - **Results:** 0 for all queries. - **Reasoning:** No existing issues cover the replacement of hardcoded ULIDs in test fixtures with dynamic generation. > **Backlog note:** This issue was discovered during autonomous operation > by the test-infra-improver agent. It does not block any active milestone > completion and has been placed in the backlog for human review and future > milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

HAL9000 added the

Rows
Columns