aditya
8da7bd1e56
CI / lint (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 26s
CI / security (pull_request) Successful in 22s
CI / quality (pull_request) Successful in 16s
CI / integration_tests (pull_request) Successful in 4m17s
CI / build (pull_request) Successful in 16s
CI / unit_tests (pull_request) Successful in 7m45s
CI / docker (pull_request) Successful in 44s
CI / coverage (pull_request) Successful in 5m38s
233 lines
12 KiB
Gherkin
233 lines
12 KiB
Gherkin
Feature: Action YAML schema validation
|
|
As a CleverAgents user
|
|
I want to define actions via YAML configuration files
|
|
So that I can create validated, reusable plan templates
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Valid schema scenarios
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Load a minimal valid action YAML
|
|
Given an action YAML string with only required fields
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config name should be "local/simple-action"
|
|
And the action config strategy_actor should be "openai/gpt-4"
|
|
And the action config execution_actor should be "openai/gpt-4"
|
|
|
|
Scenario: Load a valid action YAML with all optional fields
|
|
Given an action YAML string with all optional fields populated
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config reusable should be true
|
|
And the action config read_only should be false
|
|
And the action config state should be "available"
|
|
And the action config estimation_actor should be "local/estimator"
|
|
And the action config review_actor should be "local/reviewer"
|
|
And the action config apply_actor should be "local/applier"
|
|
And the action config invariant_actor should be "local/invariant-resolver"
|
|
|
|
Scenario: Load the simple example YAML
|
|
Given the action YAML file "examples/actions/simple.yaml"
|
|
When I validate the action schema from file
|
|
Then the action schema validation should succeed
|
|
|
|
Scenario: Load the invariant-heavy example YAML
|
|
Given the action YAML file "examples/actions/invariant-heavy.yaml"
|
|
When I validate the action schema from file
|
|
Then the action schema validation should succeed
|
|
|
|
Scenario: Load the read-only example YAML
|
|
Given the action YAML file "examples/actions/read-only.yaml"
|
|
When I validate the action schema from file
|
|
Then the action schema validation should succeed
|
|
|
|
Scenario: Load the estimation-actor example YAML
|
|
Given the action YAML file "examples/actions/estimation-actor.yaml"
|
|
When I validate the action schema from file
|
|
Then the action schema validation should succeed
|
|
|
|
Scenario: Load the inputs-schema example YAML
|
|
Given the action YAML file "examples/actions/inputs-schema.yaml"
|
|
When I validate the action schema from file
|
|
Then the action schema validation should succeed
|
|
|
|
Scenario: Optional fields have correct defaults when omitted
|
|
Given an action YAML string with only required fields
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config reusable should be true
|
|
And the action config read_only should be false
|
|
And the action config state should be "available"
|
|
And the action config invariants list should be empty
|
|
And the action config arguments list should be empty
|
|
|
|
Scenario: Action with arguments validates correctly
|
|
Given an action YAML string with typed arguments
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config should have 2 arguments
|
|
And action argument 0 name should be "target_coverage"
|
|
And action argument 0 type should be "integer"
|
|
And action argument 1 name should be "test_command"
|
|
And action argument 1 type should be "string"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Invalid schema scenarios
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Missing name field produces a clear error
|
|
Given an action YAML string missing the "name" field
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "name"
|
|
|
|
Scenario: Missing strategy_actor produces a clear error
|
|
Given an action YAML string missing the "strategy_actor" field
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "strategy_actor"
|
|
|
|
Scenario: Missing execution_actor produces a clear error
|
|
Given an action YAML string missing the "execution_actor" field
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "execution_actor"
|
|
|
|
Scenario: Missing definition_of_done produces a clear error
|
|
Given an action YAML string missing the "definition_of_done" field
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "definition_of_done"
|
|
|
|
Scenario: Missing description produces a clear error
|
|
Given an action YAML string missing the "description" field
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "description"
|
|
|
|
Scenario: Invalid namespaced name without slash
|
|
Given an action YAML string with name "invalid-name-no-slash"
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "namespace/name"
|
|
|
|
Scenario: Invalid namespaced name with special characters
|
|
Given an action YAML string with name "local/bad name!!"
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "namespace/name"
|
|
|
|
Scenario: Invalid argument type
|
|
Given an action YAML string with an argument of type "array"
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "type"
|
|
|
|
Scenario: Invalid state value
|
|
Given an action YAML string with state "draft"
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "state"
|
|
|
|
Scenario: Invalid argument name
|
|
Given an action YAML string with an argument named "123bad"
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "identifier"
|
|
|
|
Scenario: None YAML string raises ValueError
|
|
Given a None action YAML string
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "None"
|
|
|
|
Scenario: Empty YAML string raises ValueError
|
|
Given an empty action YAML string
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "empty"
|
|
|
|
Scenario: YAML that is a list instead of a mapping
|
|
Given an action YAML string that is a list
|
|
When I validate the action schema expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "mapping"
|
|
|
|
Scenario: None file path raises ValueError
|
|
Given a None action YAML file path
|
|
When I validate the action schema from file expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "None"
|
|
|
|
Scenario: Non-existent file raises FileNotFoundError
|
|
Given the action YAML file "examples/actions/nonexistent.yaml"
|
|
When I validate the action schema from file expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "not found"
|
|
|
|
Scenario: Directory path raises ValueError
|
|
Given the action YAML directory path "examples/actions"
|
|
When I validate the action schema from file expecting failure
|
|
Then the action schema validation should fail
|
|
And the action schema error should mention "not a file"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Invariant normalization scenarios
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Empty invariants are dropped
|
|
Given an action YAML string with empty invariant strings
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config invariants list should be empty
|
|
|
|
Scenario: Duplicate invariants are de-duplicated preserving order
|
|
Given an action YAML string with duplicate invariants
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config should have 2 invariants
|
|
And action invariant 0 should be "No secrets in code"
|
|
And action invariant 1 should be "All tests must pass"
|
|
|
|
Scenario: Invariant whitespace is trimmed
|
|
Given an action YAML string with whitespace-padded invariants
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And action invariant 0 should be "Keep it clean"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Key normalization scenarios
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: camelCase keys are normalized to snake_case
|
|
Given an action YAML string with camelCase keys
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config strategy_actor should be "openai/gpt-4"
|
|
And the action config execution_actor should be "openai/gpt-4"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Environment variable interpolation
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Environment variable interpolation in YAML values
|
|
Given the environment variable "TEST_ACTOR" is set to "openai/gpt-4"
|
|
And an action YAML string using env var "${TEST_ACTOR}" for strategy_actor
|
|
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config strategy_actor should be "openai/gpt-4"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Round-trip serialization
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Schema model serializes to dict correctly
|
|
Given an action YAML string with all optional fields populated
|
|
When I validate the action schema
|
|
Then the action schema validation should succeed
|
|
And the action config model_dump should contain key "name"
|
|
And the action config model_dump should contain key "strategy_actor"
|
|
|