test(plan): replace ad-hoc plan IDs with valid ULIDs #9634

New issue

Open

opened 2026-04-15 00:54:03 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-15 00:54:03 +00:00

Owner

Summary

Behave and Robot plan fixtures hardcode human-readable plan IDs that are not valid Crockford ULIDs (e.g., "01PLANTEST000000000000001", "01JBG969PLANID0000000000").
The new ULID validation guard exercised in features/steps/plan_ulid_validation_steps.py expects real ULIDs; these constants will fail once validation is enforced globally and tests currently bypass that reality.
The lack of a shared helper means each step file repeats the same invalid literals, so inconsistencies slip in unnoticed.

Impact

Upcoming tightening of plan ID validation will cause wide swaths of behaviour and CLI regression tests to fail at collection time.
Because tests rely on invalid IDs, they cannot surface regressions where ULID parsing or normalization goes wrong in production code.

Evidence

features/steps/plan_apply_service_coverage_steps.py: _PLAN_ID = "01PLANTEST000000000000001" (contains the disallowed letter "L").
features/mocks/tdd_plan_correct_plan_id_fixtures.py: PLAN_ID: str = "01JBG969PLANID0000000000" (includes "I"/"L" which ULIDs forbid).
features/steps/plan_ulid_validation_steps.py defines _VALID_ULID = "01HXM8C2ZK4Q7C2B3F2R4VYV6J", showing the expected alphabet.

Proposal

Add a shared test helper (e.g., features.mocks.plan_id_factory) that returns str(ULID()) and update all plan fixtures/steps to consume it.
Add a lightweight assertion utility to fail fast when a test hardcodes an ID containing disallowed characters so future regressions are prevented.

Duplicate Check

Search open issues for "ulid"
Search closed issues for "ulid"
Search open issues for "plan id"
Search closed issues for "plan id"
Reviewed closed issue #8332; it addressed hardcoded timestamps/IDs broadly but not the invalid ULID alphabet gap.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - Behave and Robot plan fixtures hardcode human-readable plan IDs that are not valid Crockford ULIDs (e.g., "01PLANTEST000000000000001", "01JBG969PLANID0000000000"). - The new ULID validation guard exercised in `features/steps/plan_ulid_validation_steps.py` expects real ULIDs; these constants will fail once validation is enforced globally and tests currently bypass that reality. - The lack of a shared helper means each step file repeats the same invalid literals, so inconsistencies slip in unnoticed. ## Impact - Upcoming tightening of plan ID validation will cause wide swaths of behaviour and CLI regression tests to fail at collection time. - Because tests rely on invalid IDs, they cannot surface regressions where ULID parsing or normalization goes wrong in production code. ## Evidence - `features/steps/plan_apply_service_coverage_steps.py`: `_PLAN_ID = "01PLANTEST000000000000001"` (contains the disallowed letter "L"). - `features/mocks/tdd_plan_correct_plan_id_fixtures.py`: `PLAN_ID: str = "01JBG969PLANID0000000000"` (includes "I"/"L" which ULIDs forbid). - `features/steps/plan_ulid_validation_steps.py` defines `_VALID_ULID = "01HXM8C2ZK4Q7C2B3F2R4VYV6J"`, showing the expected alphabet. ## Proposal - Add a shared test helper (e.g., `features.mocks.plan_id_factory`) that returns `str(ULID())` and update all plan fixtures/steps to consume it. - Add a lightweight assertion utility to fail fast when a test hardcodes an ID containing disallowed characters so future regressions are prevented. ### Duplicate Check - [Search open issues for "ulid"](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=ulid&state=open) - [Search closed issues for "ulid"](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=ulid&state=closed) - [Search open issues for "plan id"](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=plan%20id&state=open) - [Search closed issues for "plan id"](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=plan%20id&state=closed) - Reviewed closed issue [#8332](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/8332); it addressed hardcoded timestamps/IDs broadly but not the invalid ULID alphabet gap. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

Rows
Columns