Commit Graph

1 Commits

Author SHA1 Message Date
freemo 36b3212607 refactor(tests): improve data variation in existing tests using factory and fixture system
Audited 587 existing Behave feature files to identify scenarios with poor data variation (hardcoded or repetitive single values). Prioritized high-impact candidates: ULID validation, NamespacedName validation, project name validation, and skill schema name validation. Created 5 new feature files using Behave's native Scenario Outline + Examples tables as the data variation mechanism (since blocking issues #2760 TestDataFactory and #2765 Centralized Fixture System are not yet implemented):
- features/data_variation_plan_ulid.feature: 33 scenarios covering valid/invalid ULID formats, boundary lengths, illegal characters (I/L/O/U), legacy names, and CLI command validation
- features/data_variation_namespaced_name.feature: 32 scenarios covering valid names, special characters in namespace/name components, boundary lengths
- features/data_variation_project_name.feature: 35 scenarios covering invalid special characters, valid formats, path resolution
- features/data_variation_skill_name.feature: 39 scenarios covering invalid names, tool refs, MCP transports, include names
- features/data_variation_edge_cases.feature: 81 scenarios covering empty/null values, boundary lengths, special characters, and invalid input types across all four domains

All 592 features pass (14636 scenarios), typecheck passes with 0 errors

Key design decisions:
- Used Behave Scenario Outline + Examples tables as the Behave-native data variation approach (pending #2760 and #2765)
- Created new additive feature files rather than modifying existing ones to avoid breaking existing tests
- Verified each scenario against actual implementation behavior before including in Examples tables
- Removed pipe characters from table cells (Behave table delimiter conflict)
- Used "a ValueError should be raised" for Pydantic ValidationError assertions (Pydantic ValidationError IS a ValueError)

Impact:
- Improves data variation coverage in critical test domains without restructuring the existing test suite
- Keeps changes isolated and additive to minimize risk to current tests
- Maintains alignment with ongoing infrastructure work (factory/fixture system) while providing immediate gains

ISSUES CLOSED: #2772
2026-04-05 04:22:46 +00:00