1 Commits

Author SHA1 Message Date
freemo 36b3212607 refactor(tests): improve data variation in existing tests using factory and fixture system
CI / typecheck (pull_request) Successful in 1m2s
CI / security (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 46s
CI / lint (pull_request) Successful in 3m21s
CI / build (pull_request) Successful in 19s
CI / helm (pull_request) Successful in 23s
CI / unit_tests (pull_request) Successful in 7m6s
CI / e2e_tests (pull_request) Successful in 18m1s
CI / integration_tests (pull_request) Successful in 22m52s
CI / docker (pull_request) Successful in 21s
CI / coverage (pull_request) Successful in 10m58s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m25s
Audited 587 existing Behave feature files to identify scenarios with poor data variation (hardcoded or repetitive single values). Prioritized high-impact candidates: ULID validation, NamespacedName validation, project name validation, and skill schema name validation. Created 5 new feature files using Behave's native Scenario Outline + Examples tables as the data variation mechanism (since blocking issues #2760 TestDataFactory and #2765 Centralized Fixture System are not yet implemented):
- features/data_variation_plan_ulid.feature: 33 scenarios covering valid/invalid ULID formats, boundary lengths, illegal characters (I/L/O/U), legacy names, and CLI command validation
- features/data_variation_namespaced_name.feature: 32 scenarios covering valid names, special characters in namespace/name components, boundary lengths
- features/data_variation_project_name.feature: 35 scenarios covering invalid special characters, valid formats, path resolution
- features/data_variation_skill_name.feature: 39 scenarios covering invalid names, tool refs, MCP transports, include names
- features/data_variation_edge_cases.feature: 81 scenarios covering empty/null values, boundary lengths, special characters, and invalid input types across all four domains

All 592 features pass (14636 scenarios), typecheck passes with 0 errors

Key design decisions:
- Used Behave Scenario Outline + Examples tables as the Behave-native data variation approach (pending #2760 and #2765)
- Created new additive feature files rather than modifying existing ones to avoid breaking existing tests
- Verified each scenario against actual implementation behavior before including in Examples tables
- Removed pipe characters from table cells (Behave table delimiter conflict)
- Used "a ValueError should be raised" for Pydantic ValidationError assertions (Pydantic ValidationError IS a ValueError)

Impact:
- Improves data variation coverage in critical test domains without restructuring the existing test suite
- Keeps changes isolated and additive to minimize risk to current tests
- Maintains alignment with ongoing infrastructure work (factory/fixture system) while providing immediate gains

ISSUES CLOSED: #2772
2026-04-05 04:22:46 +00:00