freemo
36b3212607
CI / typecheck (pull_request) Successful in 1m2s
CI / security (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 46s
CI / lint (pull_request) Successful in 3m21s
CI / build (pull_request) Successful in 19s
CI / helm (pull_request) Successful in 23s
CI / unit_tests (pull_request) Successful in 7m6s
CI / e2e_tests (pull_request) Successful in 18m1s
CI / integration_tests (pull_request) Successful in 22m52s
CI / docker (pull_request) Successful in 21s
CI / coverage (pull_request) Successful in 10m58s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m25s
Audited 587 existing Behave feature files to identify scenarios with poor data variation (hardcoded or repetitive single values). Prioritized high-impact candidates: ULID validation, NamespacedName validation, project name validation, and skill schema name validation. Created 5 new feature files using Behave's native Scenario Outline + Examples tables as the data variation mechanism (since blocking issues #2760 TestDataFactory and #2765 Centralized Fixture System are not yet implemented): - features/data_variation_plan_ulid.feature: 33 scenarios covering valid/invalid ULID formats, boundary lengths, illegal characters (I/L/O/U), legacy names, and CLI command validation - features/data_variation_namespaced_name.feature: 32 scenarios covering valid names, special characters in namespace/name components, boundary lengths - features/data_variation_project_name.feature: 35 scenarios covering invalid special characters, valid formats, path resolution - features/data_variation_skill_name.feature: 39 scenarios covering invalid names, tool refs, MCP transports, include names - features/data_variation_edge_cases.feature: 81 scenarios covering empty/null values, boundary lengths, special characters, and invalid input types across all four domains All 592 features pass (14636 scenarios), typecheck passes with 0 errors Key design decisions: - Used Behave Scenario Outline + Examples tables as the Behave-native data variation approach (pending #2760 and #2765) - Created new additive feature files rather than modifying existing ones to avoid breaking existing tests - Verified each scenario against actual implementation behavior before including in Examples tables - Removed pipe characters from table cells (Behave table delimiter conflict) - Used "a ValueError should be raised" for Pydantic ValidationError assertions (Pydantic ValidationError IS a ValueError) Impact: - Improves data variation coverage in critical test domains without restructuring the existing test suite - Keeps changes isolated and additive to minimize risk to current tests - Maintains alignment with ongoing infrastructure work (factory/fixture system) while providing immediate gains ISSUES CLOSED: #2772
100 lines
7.6 KiB
Gherkin
100 lines
7.6 KiB
Gherkin
Feature: NamespacedName validation data variation — Scenario Outline refactoring
|
|
As a CleverAgents developer
|
|
I want NamespacedName validation tests to cover a wide range of inputs
|
|
So that edge cases and boundary conditions are systematically exercised
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Refactored from plan_namespaced_name_validation.feature using Scenario
|
|
# Outline. Each Examples table row exercises a distinct code path.
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Valid namespaced names — all should parse successfully
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: NamespacedName.parse() accepts valid namespaced names
|
|
When I parse the namespaced name "<full_name>"
|
|
Then the parsed namespace should be "<namespace>"
|
|
And the parsed item name should be "<item_name>"
|
|
|
|
Examples: Standard valid names
|
|
| full_name | namespace | item_name | description |
|
|
| abc123/my-action | abc123 | my-action | namespace starts with letter |
|
|
| local/a123-action | local | a123-action | name starts with letter |
|
|
| my-org/my-action | my-org | my-action | kebab-case namespace and name |
|
|
| x/y | x | y | single-char namespace and name |
|
|
| org/tool-v2 | org | tool-v2 | name with version suffix |
|
|
| builtin/read-file | builtin | read-file | builtin namespace |
|
|
| local/my-skill | local | my-skill | local namespace |
|
|
|
|
Examples: Names with numbers (not at start)
|
|
| full_name | namespace | item_name | description |
|
|
| org123/tool456 | org123 | tool456 | numbers in middle of both parts |
|
|
| a1b2c3/d4e5f6 | a1b2c3 | d4e5f6 | alternating letters and numbers |
|
|
| ns/name-with-123 | ns | name-with-123 | numbers at end of name |
|
|
|
|
Examples: Boundary length names
|
|
| full_name | namespace | item_name | description |
|
|
| a/b | a | b | minimum length (1 char each) |
|
|
| ab/cd | ab | cd | two chars each |
|
|
| abc/def | abc | def | three chars each |
|
|
| my-namespace/my-tool-name | my-namespace | my-tool-name | medium length names |
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Invalid namespaced names — special characters in namespace
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: NamespacedName.parse() rejects namespace with special characters
|
|
When I parse the namespaced name "<full_name>" expecting an error
|
|
Then a ValueError should be raised
|
|
|
|
Examples: Special characters in namespace
|
|
| full_name | description |
|
|
| bad@ns/name | at-sign in namespace |
|
|
| bad ns/name | space in namespace |
|
|
| bad!ns/name | exclamation in namespace |
|
|
| bad#ns/name | hash in namespace |
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Invalid namespaced names — special characters in name component
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: NamespacedName.parse() rejects name component with special characters
|
|
When I parse the namespaced name "<full_name>" expecting an error
|
|
Then a ValueError should be raised
|
|
|
|
Examples: Special characters in name component
|
|
| full_name | description |
|
|
| ns/bad name | space in name component |
|
|
| ns/bad@name | at-sign in name component |
|
|
| ns/bad!name | exclamation in name component |
|
|
| ns/bad#name | hash in name component |
|
|
| ns/bad/extra | extra slash (three components) |
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Invalid namespaced names — via constructor
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: NamespacedName constructor rejects invalid namespace formats
|
|
When I construct a NamespacedName with namespace "<namespace>" and name "<name>" expecting an error
|
|
Then a ValueError should be raised
|
|
|
|
Examples: Invalid namespace formats
|
|
| namespace | name | description |
|
|
| bad@ns | valid-name | at-sign in namespace |
|
|
| bad ns | valid-name | space in namespace |
|
|
| bad!ns | valid-name | exclamation in namespace |
|
|
| bad#ns | valid-name | hash in namespace |
|
|
|
|
Scenario Outline: NamespacedName constructor rejects invalid name formats
|
|
When I construct a NamespacedName with namespace "<namespace>" and name "<name>" expecting an error
|
|
Then a ValueError should be raised
|
|
|
|
Examples: Invalid name formats
|
|
| namespace | name | description |
|
|
| local | bad name | space in name |
|
|
| local | bad@name | at-sign in name |
|
|
| local | bad!name | exclamation in name |
|
|
| local | bad#name | hash in name |
|
|
| local | bad/name | slash in name |
|