freemo
300a5d6ddc
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 18s
CI / helm (pull_request) Successful in 23s
CI / lint (pull_request) Failing after 26s
CI / quality (pull_request) Successful in 34s
CI / security (pull_request) Failing after 46s
CI / typecheck (pull_request) Failing after 50s
CI / coverage (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Has been skipped
CI / unit_tests (pull_request) Failing after 1m46s
CI / docker (pull_request) Has been skipped
CI / e2e_tests (pull_request) Failing after 14m9s
CI / integration_tests (pull_request) Failing after 20m57s
CI / status-check (pull_request) Failing after 1s
Add ULID format validation to all v3 plan commands to prevent users from
accidentally mixing legacy ('agents tell') and v3 ('agents plan use')
workflows. The two systems use separate storage backends and cannot be
mixed; this change detects the mismatch early and provides actionable
error messages.
Changes:
- Add _validate_plan_ulid() with proper Crockford Base32 regex
(^[0-9A-HJKMNP-TV-Z]{26}$, re.IGNORECASE) that correctly rejects
invalid characters (I, L, O, U), hyphens, and wrong-length strings
- Add _PLAN_ULID_RE compiled regex and _ULID_VALIDATION_ERROR_MSG constant
with actionable guidance explaining the legacy/v3 incompatibility
- Apply ULID validation to all v3 commands: execute_plan,
_lifecycle_apply_with_id, lifecycle_apply_plan, plan_status,
plan_errors, cancel_plan (only on user-provided IDs, not auto-discovered)
- Update _LEGACY_DEPRECATION_MSG and tell/build command warnings to
explicitly state that the two workflows are INCOMPATIBLE and cannot be mixed
- Add comprehensive BDD tests in features/plan_ulid_validation.feature
with step definitions using Typer CLI runner (not subprocess)
- Update CONTRIBUTING.md with 'Workflow Choice: Legacy vs. v3 Plan
Lifecycle' section documenting the incompatibility and migration path
ISSUES CLOSED: #1560
142 lines
7.1 KiB
Gherkin
142 lines
7.1 KiB
Gherkin
Feature: ULID validation for v3 plan commands
|
|
As a CleverAgents user
|
|
I want clear, actionable error messages when I accidentally mix legacy and v3 plan workflows
|
|
So that I understand the incompatibility and know how to fix it
|
|
|
|
# ===================================================================
|
|
# _validate_plan_ulid unit tests
|
|
# ===================================================================
|
|
|
|
Scenario: _validate_plan_ulid accepts a valid ULID
|
|
When I call _validate_plan_ulid with "01HXM8C2ZK4Q7C2B3F2R4VYV6J"
|
|
Then ulid-val the validation should succeed and return "01HXM8C2ZK4Q7C2B3F2R4VYV6J"
|
|
|
|
Scenario: _validate_plan_ulid accepts a valid ULID in lowercase
|
|
When I call _validate_plan_ulid with "01hxm8c2zk4q7c2b3f2r4vyv6j"
|
|
Then ulid-val the validation should succeed and return "01hxm8c2zk4q7c2b3f2r4vyv6j"
|
|
|
|
Scenario: _validate_plan_ulid rejects a legacy plan name
|
|
When I call _validate_plan_ulid with "64-bit port plan"
|
|
Then ulid-val the validation should raise ValidationError
|
|
And ulid-val the error message should contain "not found"
|
|
And ulid-val the error message should contain "ULID"
|
|
And ulid-val the error message should contain "legacy"
|
|
And ulid-val the error message should contain "agents plan use"
|
|
|
|
Scenario: _validate_plan_ulid rejects a plain string that is not a ULID
|
|
When I call _validate_plan_ulid with "not-a-ulid"
|
|
Then ulid-val the validation should raise ValidationError
|
|
And ulid-val the error message should contain "ULID"
|
|
|
|
Scenario: _validate_plan_ulid rejects a string that is too short
|
|
When I call _validate_plan_ulid with "01ARZ3NDEK"
|
|
Then ulid-val the validation should raise ValidationError
|
|
And ulid-val the error message should contain "ULID"
|
|
|
|
Scenario: _validate_plan_ulid rejects a string that is too long
|
|
When I call _validate_plan_ulid with "01ARZ3NDEKTSV4RRFFQ69G5FAAEXTRA"
|
|
Then ulid-val the validation should raise ValidationError
|
|
And ulid-val the error message should contain "ULID"
|
|
|
|
Scenario: _validate_plan_ulid rejects a ULID with invalid characters (I, L, O, U)
|
|
When I call _validate_plan_ulid with "01ARZ3NDEKTSV4RRFFQ69G5FII"
|
|
Then ulid-val the validation should raise ValidationError
|
|
And ulid-val the error message should contain "ULID"
|
|
|
|
# ===================================================================
|
|
# agents plan execute — ULID validation
|
|
# ===================================================================
|
|
|
|
Scenario: plan execute rejects a legacy plan name with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan execute with "my-legacy-plan"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "not found"
|
|
And the ulid-validation output should contain "ULID"
|
|
And the ulid-validation output should contain "legacy"
|
|
And the ulid-validation output should contain "agents plan use"
|
|
|
|
Scenario: plan execute rejects a non-ULID string with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan execute with "not-a-valid-id"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
|
|
Scenario: plan execute accepts a valid ULID and proceeds normally
|
|
Given a ulid-validation CLI runner
|
|
And a mocked lifecycle service for ulid-validation execute
|
|
When I invoke ulid-validation plan execute with "01HXM8C2ZK4Q7C2B3F2R4VYV6J"
|
|
Then the ulid-validation command should not abort due to ULID validation
|
|
|
|
# ===================================================================
|
|
# agents plan apply — ULID validation
|
|
# ===================================================================
|
|
|
|
Scenario: plan apply rejects a legacy plan name with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan apply with "my-legacy-plan"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "not found"
|
|
And the ulid-validation output should contain "ULID"
|
|
And the ulid-validation output should contain "legacy"
|
|
|
|
Scenario: plan apply rejects a non-ULID string with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan apply with "not-a-valid-id"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
|
|
# ===================================================================
|
|
# agents plan status — ULID validation
|
|
# ===================================================================
|
|
|
|
Scenario: plan status rejects a legacy plan name with actionable error
|
|
Given a ulid-validation CLI runner
|
|
And a mocked lifecycle service for ulid-validation status
|
|
When I invoke ulid-validation plan status with "my-legacy-plan"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
And the ulid-validation output should contain "legacy"
|
|
|
|
Scenario: plan status rejects a non-ULID string with actionable error
|
|
Given a ulid-validation CLI runner
|
|
And a mocked lifecycle service for ulid-validation status
|
|
When I invoke ulid-validation plan status with "not-a-valid-id"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
|
|
# ===================================================================
|
|
# agents plan errors — ULID validation
|
|
# ===================================================================
|
|
|
|
Scenario: plan errors rejects a legacy plan name with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan errors with "my-legacy-plan"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
|
|
# ===================================================================
|
|
# agents plan cancel — ULID validation
|
|
# ===================================================================
|
|
|
|
Scenario: plan cancel rejects a legacy plan name with actionable error
|
|
Given a ulid-validation CLI runner
|
|
When I invoke ulid-validation plan cancel with "my-legacy-plan"
|
|
Then the ulid-validation command should abort
|
|
And the ulid-validation output should contain "ULID"
|
|
|
|
# ===================================================================
|
|
# Legacy deprecation warning content
|
|
# ===================================================================
|
|
|
|
Scenario: Legacy tell command deprecation warning explains workflow incompatibility
|
|
When I call tell_command programmatically for ulid-validation
|
|
Then the ulid-validation deprecation warning should mention "incompatible"
|
|
And the ulid-validation deprecation warning should mention "agents plan use"
|
|
And the ulid-validation deprecation warning should not suggest simple command swap
|
|
|
|
Scenario: Legacy build command deprecation warning explains workflow incompatibility
|
|
When I call build_command programmatically for ulid-validation
|
|
Then the ulid-validation deprecation warning should mention "incompatible"
|
|
And the ulid-validation deprecation warning should mention "agents plan use"
|