freemo
36b3212607
CI / typecheck (pull_request) Successful in 1m2s
CI / security (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 46s
CI / lint (pull_request) Successful in 3m21s
CI / build (pull_request) Successful in 19s
CI / helm (pull_request) Successful in 23s
CI / unit_tests (pull_request) Successful in 7m6s
CI / e2e_tests (pull_request) Successful in 18m1s
CI / integration_tests (pull_request) Successful in 22m52s
CI / docker (pull_request) Successful in 21s
CI / coverage (pull_request) Successful in 10m58s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m25s
Audited 587 existing Behave feature files to identify scenarios with poor data variation (hardcoded or repetitive single values). Prioritized high-impact candidates: ULID validation, NamespacedName validation, project name validation, and skill schema name validation. Created 5 new feature files using Behave's native Scenario Outline + Examples tables as the data variation mechanism (since blocking issues #2760 TestDataFactory and #2765 Centralized Fixture System are not yet implemented): - features/data_variation_plan_ulid.feature: 33 scenarios covering valid/invalid ULID formats, boundary lengths, illegal characters (I/L/O/U), legacy names, and CLI command validation - features/data_variation_namespaced_name.feature: 32 scenarios covering valid names, special characters in namespace/name components, boundary lengths - features/data_variation_project_name.feature: 35 scenarios covering invalid special characters, valid formats, path resolution - features/data_variation_skill_name.feature: 39 scenarios covering invalid names, tool refs, MCP transports, include names - features/data_variation_edge_cases.feature: 81 scenarios covering empty/null values, boundary lengths, special characters, and invalid input types across all four domains All 592 features pass (14636 scenarios), typecheck passes with 0 errors Key design decisions: - Used Behave Scenario Outline + Examples tables as the Behave-native data variation approach (pending #2760 and #2765) - Created new additive feature files rather than modifying existing ones to avoid breaking existing tests - Verified each scenario against actual implementation behavior before including in Examples tables - Removed pipe characters from table cells (Behave table delimiter conflict) - Used "a ValueError should be raised" for Pydantic ValidationError assertions (Pydantic ValidationError IS a ValueError) Impact: - Improves data variation coverage in critical test domains without restructuring the existing test suite - Keeps changes isolated and additive to minimize risk to current tests - Maintains alignment with ongoing infrastructure work (factory/fixture system) while providing immediate gains ISSUES CLOSED: #2772
86 lines
6.5 KiB
Gherkin
86 lines
6.5 KiB
Gherkin
Feature: Project name validation data variation — Scenario Outline refactoring
|
|
As a CleverAgents developer
|
|
I want project name validation tests to cover a wide range of inputs
|
|
So that edge cases and boundary conditions are systematically exercised
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Refactored from validation_test_fixtures.feature (Section 4) using
|
|
# Scenario Outline. Each Examples table row exercises a distinct
|
|
# validation path or boundary condition.
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Invalid project names — special characters
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: Project rejects names with special characters
|
|
When I try to create a project with name "<invalid_name>"
|
|
Then a project validation error should be raised mentioning "alphanumeric"
|
|
|
|
Examples: Names with special characters — punctuation
|
|
| invalid_name | description |
|
|
| my@project | at-sign in name |
|
|
| my/project | forward slash in name |
|
|
| project! | exclamation mark in name |
|
|
| project#tag | hash symbol in name |
|
|
| project%done | percent sign in name |
|
|
| project^up | caret in name |
|
|
| project&more | ampersand in name |
|
|
| project*star | asterisk in name |
|
|
| project+plus | plus sign in name |
|
|
| project=equals | equals sign in name |
|
|
| project.pipe | period in name |
|
|
| project:colon | colon in name |
|
|
| project;semi | semicolon in name |
|
|
| project<less | less-than sign in name |
|
|
| project>greater | greater-than sign in name |
|
|
| project?question | question mark in name |
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Invalid project names — empty and whitespace-only
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Project rejects empty name
|
|
When I try to create a project with empty name
|
|
Then a project validation error should be raised
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Valid project names — accepted formats
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: Project accepts names with alphanumeric characters, hyphens, underscores, and spaces
|
|
When I create a project fixture with name "<valid_name>"
|
|
Then the project should be created with that name
|
|
|
|
Examples: Valid project name formats
|
|
| valid_name | description |
|
|
| my-project_v2 final | hyphens, underscores, and spaces |
|
|
| simple | plain alphanumeric name |
|
|
| project123 | name with trailing numbers |
|
|
| my project | name with space |
|
|
| project-name | hyphenated name |
|
|
| project_name | underscored name |
|
|
| Project Name | mixed case with space |
|
|
| a | single character name |
|
|
| abc def ghi | multiple words with spaces |
|
|
| project-v1-final | multiple hyphens |
|
|
| my_project_2024 | underscores with numbers |
|
|
| 123project | name starting with numbers |
|
|
| My Project Name | mixed case with multiple spaces |
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Edge cases — boundary conditions
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario Outline: Project path is always resolved to absolute
|
|
When I create a project with relative path "<rel_path>"
|
|
Then the project fixture path should be absolute
|
|
|
|
Examples: Relative path formats
|
|
| rel_path | description |
|
|
| relative/path | simple relative path |
|
|
| ./local/path | dot-relative path |
|
|
| ../parent/path | parent-relative path |
|
|
| subdir/nested/deep | deeply nested relative path |
|
|
| just-a-dir | single directory name |
|